Why formative test fidelity is important to simulate realistic use for medical devices

The degree to which users, environment, and test articles are realistically simulated for a medical device formative test (I.E. Formative Test Fidelity) has a significant impact on the integrity and quality of the usability study data. Spending the time and effort to meet the recommended level of fidelity helps ensure that data collected from the formative test will be impactful and valuable to the design team.

Study Participants

A medical device is designed according to the target patient and user population. Recruiting and using study participants based on the intended patient and/or user population is integral to collecting reflective and actionable usability study findings. Formative test fidelity will in part validate that the system has been designed to meet these defined user and patient profile needs. Ensuring participants reflect key profile characteristics such as age, demographic, socioeconomic status, education, and training is imperative to evaluate the system’s user interfaces effectively. These characteristics govern an individual’s behaviour, perception, attention, and cognition, and guide how they interact with and utilize the system.

Study Environment

A study environment that closely resembles the intended use environment will provide formative test fidelity and facilitate the most realistic interaction between the patient and/or user of the device. For a formative test, an environment which can accommodate this need, while being readily available for use within the desired timeline and budget, is most desirable.

Preparing ancillary and supporting materials within the environment is recommended. Lighting conditions, ambient noise, and distractions should all be taken into consideration. Environmental characteristics (including available materials) play a significant role in how the user perceives a situation. A participant is better able to get into a mindset, reflective of the intended use scenario, when the environment best mimics the actual use scenario. Providing the participant with a better simulated environment relieves them of having to rely on their imagination during completion of the scenario. The individual is able to act based off instinct and any previous experience, rather than dedicating effort into imagining the circumstance.

Study Materials and Formative Test Fidelity

A formative test may be designed to study one, a collection, or all of the user interfaces of a system. When a formative test is being planned and executed, there should be a prototype of the device and its user interfaces that is reflective of the intended embodiment of the system to ensure formative test fidelity. A prototype that closely resembles the intended embodiment will allow for a more informed analysis. A study that uses a lower fidelity representation of the system may be subject to test artifacts. Often over-looked components of a formative test are the training materials and user documentation. It is advantageous to evaluate these materials iteratively prior to the Summative Usability Validation through formative testing, to ensure the summative validation is successful and generates minimal findings. Having representative user documentation available during a formative test (instructions for use and quick reference guide) will inform on the quality of the materials, as well as the manner in which they are presented.

In summary, the intent of a formative usability test is to inform the design of the system. However, the formative test fidelity to which the environment, participants and materials are simulated to replicate a realistic use scenario has significant impact on the quality of the data and findings that are collected. Spending the effort and time preparing and attending to these characteristics is valuable to the design team, and impacts the resulting product’s usability.