Unit testing in embedded systems: 3 myths and an automated tip

The vast majority of medical devices with electronics contain some sort of embedded processor running firmware. There are a range of modern techniques that can be used to facilitate better reliability of embedded firmware.

One that I’m a big fan of is unit testing. Embedded engineers opposed to unit testing will raise a few common objections.  After seeing firsthand how unit testing can improve medical device firmware quality, I decided to write this blog debunking three common myths and sharing my experience using an automated unit testing tool.

3 Common Myths of Unit Testing

1. Our firmware is very simple, we don’t need unit testing.
2. We can’t unit test microcontroller code, it’s too close to the hardware.
3. Unit testing will add unnecessary cost burden to our project.

What is unit testing?

Unit testing is now a widely accepted software engineering practice. However, unit testing is severely under-utilized in the world of embedded firmware because of a few myths. Unit testing involves (usually automated) testing of small software “units” in a much larger program. The benefits of unit testing are well-known: looser coupling between software modules, simplified regression testing, and greatly improved code quality. Most software engineers working in high-level languages accept that automated unit tests are a critical part of software development.

Our firmware is very simple, we don’t need unit testing.

Medical devices firmware is rarely “simple.” It’s true that there are very simple applications for embedded microcontrollers: as timing elements, for example, or as devices which execute very basic logic operations. In such a case, tests are usually unnecessary. That said, truly “simple” firmware, in my experience, is rare. Most embedded firmware actually serves a large range of requirements.

If your firmware is doing anything more complex than sequencing LEDs or reading an analog value to raise a warning indicator, you can almost certainly benefit from having good automated test coverage. Finally, simple firmware often does not stay simple. Scope creep affects firmware just as much as high-level software and new requirements discovered later in the project can dramatically increase code complexity.

We can’t unit test microcontroller code, it’s too close to the hardware.

I would argue that if your hardware interface code is so tightly coupled to your application logic that they are inseparable, you have failed to ensure loose coupling between modules. At minimum, most hardware register read/writes should be wrapped in some sort of abstraction layer. This allows them to be correctly mocked by a test framework. If performance is a concern, these functions can be marked as inline which removes calling overhead while still keeping the code clean and separate. All that said, there are methods to make unit testing frameworks mock register interaction correctly if required.

Unit testing will add unnecessary cost burden to our project.

The cost burden of unit testing is a matter of debate. Industry consensus on the issue is that unit testing has a higher up-front cost yet it results in much fewer defects in the finished software. Shipping with fewer defects is very desirable in embedded firmware. Unlike a web application where you can quickly push to production with a few keystrokes, updating software on embedded processors often requires technician visits, or a trip back to the factory. Recalling an order to re-flash units and fix a critical bug can be very expensive, both in dollar terms and in damage to reputation.

Unit testing also results in software which is much easier to maintain and add features to throughout the software lifecycle. There are situations with “simple” medical device firmware where unit testing is an unnecessary cost burden. However, in most cases unit testing provides a tangible benefit in the form of cleaner software and cost savings as new requirements are discovered and integrated into the existing codebase.

How to Unit Test Effectively: Ceedling

So, now that we know that unit testing is a valuable activity for embedded firmware, how do we go about it? One of the best automated unit testing tools I have found to date is called Ceedling, from ThrowTheSwitch.org. It is a free open-source build system designed for embedded projects written in C that integrates the Unity unit test framework, the CMock interface mocking utility, and the CException exception handling library.

If you have good separation between modules (defined as a C header file), Ceedling makes developing and running tests a breeze. CMock integration allows it to scan the header files of all dependent modules and generate appropriate mock functions for your tests, completely decoupling your modules from each other and allowing you to test them in isolation. You can run it either on your host development machine, or in a hardware simulator. It is highly configurable and does not require integration with your IDE. You simply point it at your source files and start writing tests. Its command-line nature makes it easy to integrate with continuous integration systems. If you also use continuous integration systems, you can integrate tests into your build workflow.

Conclusion

Despite common objections, embedded firmware benefits from unit testing just as much or more than high-level software applications. Modern firmware is increasingly complex, especially with the proliferation of connected medical devices. Unit testing helps medical device manufacturers’ ship firmware with fewer defects, reducing the risk of field visits and expensive recalls. It’s never been easier to add unit testing to embedded firmware, and there is a clear case for doing so in medical devices. Why not start today?

Image: StarFish Medical

Peter Kazakoff is a StarFish Medical Electrical Engineer. A recent graduate from the University of Victoria. Peter uses unit testing on a variety of medical device projects.