The cocotb-test is a framework based on pytest. Pytest is a Python test framework that supports features like parameterized testing, assertion, which makes unit test and regression test much easier.
Also, as the test gets more and more complicated, the test bench will import more third party packages with various versions, and the environment setup gets complicated as well. To deal with that and make the test portable across different platform, we will use tox to setup the virtual environment.
So to put them together, here is what we will use as our test architecture:
With the above architecture, we will need:
1. a file called tox.ini to setup the virtual environment
2. Python files with the file names staring as “test”. This is a requirement from pytest to look for unit tests. And we will put our test bench in those python files.
So let’s look at the tox.ini first:
git clone --depth 1 --branch v1.2.7 https://github.com/PulseRain/FpgaLimerick.git
This file is largely boilerplate code, with the following part that can be customized:
1. The required packages and version, such as cocotb 1.7.2
2. The environment variable. For questa sim, the LM_LICENSE_FILE needs to be setup properly.
After you type in tox in the WSL command line, the first thing it will do is to download those required packages into a local folder called .tox.
Once the packages are all in place, the pytest will look for the test_xxx.py files under its sub directories. And as a convention, we will put all of our python test code in the tb folder. Now let’s take a look at the tb/test_nco.py
1. Test Parameters
For each DUT, it usually has some parameters (or Generics in VHDL’s lingo). And the test_param will list all the possible parameter combinations that we would like to test. For example, in the Hello World example, we would like to test 3 sets of parameters:
G_CLK_RATE = 100MHz, G_OUTPUT_RATE = 24MHz
G_CLK_RATE = 100MHz, G_OUTPUT_RATE = 10MHz
G_CLK_RATE = 50MHz, G_OUTPUT_RATE = 99KHz
2. Test Prepare
Please note in this step, we will provide a function name starting with “test”. And it will also be decorated by the @pytest.mark.paramtrize to sweep all the possible parameters set in step 1.
Please also note that this function is not marked by “async”. So it is just not a coroutine. What it does is the following:
a) Optional: If SpinalHDL is used, SBT will be invoked to regenerate the Verilog/VHDL based on the test parameter
b) Compile the Verilog/VHDL source code
c) Start the simulator
In this step, Pytest will spawn a new process to communicate with the simulator.
3. Test Bench / Test Factory
There are mainly two sub-steps in step:
a) Prepare a class for the test bench, such as the class TB() in the Hello World example. This class is basically an implementation of the functional simulation approach we mentioned in previous episodes:
Please note that all the key pieces in the above diagram are implemented as coroutine in class TB() (with async mark in front of the function name)
b) Test Factory
The Test Factory will feed the Parameters into the current test run instance. Please note that the parameters from Step 2 can to be passed to this step currently, as they are run in separate processes. The way we deal with this is to save the parameters in step 2 into a local file, and read it out in this step.
And the test factory will work with a coroutine that launches the test bench, as shown in the run_test_nco function.
Post a Comment