F P G A   

 Limerick(9) - Cocotb Tutorial: Test Bench Structure 4/21/2023

Hello, everyone. Welcome to the nineth episode of FPGA limerick. In this episode, we will take a deep look into the test bench structure of cocotb.

As we know, cocotb stands for Coroutine Cosimulation Testbench. So before we start, we might ask ourselves: What is coroutine?

In multi-task scenario, coroutine can be viewed as a collaborative non-preemptive thread. The following shows two coroutines calling each other, with yield at every step. And the execution order will always be 1, 2, 3, 4, 5

In cocotb, the coroutine is marked by putting “async” in front of the function name. And the yield is done with “await” command. We will soon see the details when it comes to the test bench next.

The Classic Way

If you read the official document of cocotb, it uses Makefile to organize the test bench. And for each coroutine, it will be decorated @cocotb.test(), like the following example:


async def test_xxx(dut):


However, what we will recommend is something different by taking advantage of the cocotb-test framework.

Using cocotb-test Framework

The cocotb-test is a framework based on pytest. Pytest is a Python test framework that supports features like parameterized testing, assertion, which makes unit test and regression test much easier. 

Also, as the test gets more and more complicated, the test bench will import more third party packages with various versions, and the environment setup gets complicated as well. To deal with that and make the test portable across different platform, we will use tox to setup the virtual environment.

So to put them together, here is what we will use as our test architecture:

With the above architecture, we will need:

1. a file called tox.ini to setup the virtual environment
2. Python files with the file names staring as “test”. This is a requirement from pytest to look for unit tests. And we will put our test bench in those python files.

So let’s look at the tox.ini first:

git clone --depth 1 --branch v1.2.7 https://github.com/PulseRain/FpgaLimerick.git

This file is largely boilerplate code, with the following part that can be customized:
1. The required packages and version, such as cocotb 1.7.2
2. The environment variable. For questa sim, the LM_LICENSE_FILE needs to be setup properly.

After you type in tox in the WSL command line, the first thing it will do is to download those required packages into a local folder called .tox.

Once the packages are all in place, the pytest will look for the test_xxx.py files under its sub directories. And as a convention, we will put all of our python test code in the tb folder. Now let’s take a look at the tb/test_nco.py

1. Test Parameters

For each DUT, it usually has some parameters (or Generics in VHDL’s lingo). And the test_param will list all the possible parameter combinations that we would like to test. For example, in the Hello World example, we would like to test 3 sets of parameters:

2. Test Prepare
Please note in this step, we will provide a function name starting with “test”. And it will also be decorated by the @pytest.mark.paramtrize to sweep all the possible parameters set in step 1.

Please also note that this function is not marked by “async”. So it is just not a coroutine.  What it does is the following:
a) Optional: If SpinalHDL is used, SBT will be invoked to regenerate the Verilog/VHDL based on the test parameter
b) Compile the Verilog/VHDL source code
c) Start the simulator

In this step, Pytest will spawn a new process to communicate with the simulator. 

3. Test Bench / Test Factory
There are mainly two sub-steps in step:
a) Prepare a class for the test bench, such as the class TB() in the Hello World example. This class is basically an implementation of the functional simulation approach we mentioned in previous episodes:

Please note that all the key pieces in the above diagram are implemented as coroutine in class TB() (with async mark in front of the function name)

b) Test Factory
The Test Factory will feed the Parameters into the current test run instance. Please note that the parameters from Step 2 can to be passed to this step currently, as they are run in separate processes. The way we deal with this is to save the parameters in step 2 into a local file, and read it out in this step.

And the test factory will work with a coroutine that launches the test bench, as shown in the run_test_nco function.

    Posted by FPGA Limerick at April 21, 2023


Post a Comment

<< Home  





*) Legal Disclaimer
*) Introduction


*) GitHub Repo
*) YouTube
*) reddit
*) SpinalHDL
*) cocotb

*) FCC Wireless
*) PAPA System