Testing and verification isn’t as sexy as adding new features, but for complex products like the PhyWhisperer-USB, proper testing is crucial to ensure that the product works as expected, as soon as you receive it.

Since the beginning of this project we’ve spent lots of time on testing. Not leaving testing as an afterthought helps prevent delays in the schedule.

With PhyWhisperer-USB we’re doing several things to ensure we roll out a product that works as expected:

1. Re-use of existing code and components:

For communication with the host computer, PhyWhisperer-USB uses a processor from the same Atmel SAM3U family that the ChipWhisperer uses. PhyWhisperer SAM3U runs much the same firmware as the ChipWhisperer SAM3U.

Furthermore, the communication interface between the PhyWhisperer SAM3U and its FPGA is the same as the one on the ChipWhisperer.

2. Dedicated simulation testbench for the new FPGA Verilog code:

Most of the new code written for PhyWhisperer is FPGA Verilog code.

A simulation testbench and a regression test suite were developed in order to:

• aid in the development (fixing bugs that show up in simulation is so much easier than fixing them on the FPGA)
• provide a quick and automated way to ensure that no new bugs are introduced as the design evolves
• support constrained random verification -- there are just too many options and behaviours to reasonably cover with directed tests!

A simulation testbench also has a bunch of benefits that makes development easier. For example, a bug fix can be validated in simulation in mere seconds, whereas generating a new FPGA bitfile takes several minutes.

All you need to run the regression test suite is a few open-source packages. No commercial tools (e.g. Vivado) required! For more technical details see the README on the project page.

3. Clean FPGA implementation:

The goal is to have a clean FPGA implementation report and we’re getting there.

4. Testing with real targets:

Simulation is great, but the rubber only hits the road with the real PhyWhisperer rev-03/04 board and real targets.

This is the last phase of testing, and the previous steps set things up for this phase to go as planned. The final rev of the boards (-04) just arrived, and we’re currently building up the prototype (to be released to manufacture).

Real PhyWhisperer boards are used throughout development; this helps ensure that the simulation models used in step #2 are valid and accurate.

Already there are Jupyter notebooks that can run on the real hardware.