TL;DR

• AI coding can speed up error detection and testing but must be validated like all medical software.
• Strict processes—verification, inspection, version control, testing, and cybersecurity—apply to AI-generated code.
• AI coding assistants add value when tasks are small, contained, and easily reviewed.
• Risks include undocumented code, hallucinated functions, and maintainability issues.
• Human oversight remains essential to ensure safety, compliance, and sustainability.

Many developers have tried using AI to generate code, often called “Vibe Coding”. Sometimes, the results are nothing short of amazing. Other times, the results are mixed, or worse. On the positive side, AI can identify issues faster than developers alone. We have used it to generate unit test templates, decipher compiler errors, describe the functionality of SOUP software, generate code comments and many other small innocuous tasks.

However, we have also seen AI make recommendations for API calls that don’t exist, or produce code that doesn’t compile. One of the worst problems created by AI was when a developer asked AI to generate code comments: The AI decided to uncomment an old variable which shadowed a global variable and set the results of a calculation to 0. Oops.

So, while not perfect, it appears that AI can be a new powerful tool in the software developer’s toolbox. The question is whether it’s appropriate for programming medical device software, where every line of code counts.

There is no substitute to writing your own code when it comes to understanding software functionality. But software is not simple, and it is impossible to know all the code for even a small, embedded device. The layers of libraries and SDK’s often hide the true complexity of software. In the medical device space, we have tools to address that reality. What sets medical device software apart from other types of development is the rigorous process required to ensure the software does not harm patients. These processes also present the key to understanding where AI can fit in with medical device software development.

The reality is this: AI-generated code should be treated no differently than any other piece of software.

AI-Generated Code Must Be Validated in Medical Devices in the following ways:

• Verified to meet requirements: Quality documentation and requirements tracing validate the software functionality against safety and functional requirements.
• Inspected for suitability:  Code reviews, static analysis and dynamic analysis are used to inspect the details of the software.
• Version controlled – Stringent version control processes ensure AI has not tampered with other functionality or re-written items without oversight. Branching strategies, manual (human) diff reviews and strong code comments are all tools that can be used to inspect, and control commits.
• Tested for malfunction – Stress testing, fuzz testing, and user trials uncover latent bugs by accelerating usage statistics.
• Cybersecure: Prove that the software was developed using proper cybersecurity controls, and ongoing vulnerability detection is designed into the products lifecycle.

Where AI Coding Adds Value in Medical Device Software

So where do AI assistants provide value? As any assistant in the real world, the value comes from acceleration of the process, that bump in the right direction, that tap on the shoulder to say, “look over here”. These assistants can take an error log and provide new ways to look at them, or various places to go search for solutions to an error. Or perhaps they can start off a function in a more elegant way or help provide guidance on limiting the number of variables passed around.

Furthermore, AI can help write small helper functions that would otherwise be mundane for talented developers; the smaller the function the easier it is for the developer to review to meet regulatory requirements.

Keeping the request to the AI small and contained results in many benefits for medical device development including:

• Code reviews of AI produced code is an easy way to maintain quality of the product and catch any hallucinations before they make it into the main branch to be caught through verification.
• Code reviews also ensure the language model has considered maintain and sustainability of the code produced by AI models.
• Reduction of context to large language models reduces the potential “IP leak” to large language models hosted by major corporations such as OpenAI, Microsoft or Google

Risks and Limitations of AI Coding in Medical Devices

If you’ve followed along this far, you can probably predict the main concern: tampering with existing functionality while making other changes.

The guidance we follow really starts at this: do not allow AI to change code without the software developer’s knowledge. Furthermore, code that an AI assistant generates that the human developer doesn’t understand should be discarded, even if it “works for the intended use case”. If a developer doesn’t understand the code, it’s (a) not maintainable, and (b) might do or not do things that the developer doesn’t “see” in the code.  Just like any included software, the code generated by AI needs to be vetted and validated, with the biggest difference is the AI code doesn’t come with any documentation.

The Path Forward for AI Coding in Medical Devices

Many medical devices are responsible for keeping their patients safe from its standard operating procedures, from errors made by technicians, nurses or doctors, straight back into how it was designed by the manufacturer.  As AI Coding assistants continue to evolve, it’s possible there will be more ways for AI to assist in the creation of medical device software. The key to using AI in medical device software is to stay true to the existing processes, regulations and design gates that are already in place. Always ensure a trained human software developer reviews AI output to confirm it is safe, effective, and compliant with regulatory processes.

Related Resources