Medical device designers understand why the phase gates in the hardware design process are necessary to keep medical device development and system integration efforts on track and moving forward. But with the introduction of complex software into the product development equation, staying true to the gated development process can be difficult. 

Medical device designers are accustomed to thinking of software as malleable and easily changed—after all, we get updates to operating systems and applications almost daily. This assumption can lead to the attitude that software requirements don’t need to be as fully developed at the beginning of a project since “it’s just software” and can be easily changed later.

Although it is true that a software change late in the project may not be as disruptive as tooling changes or an ASIC mask modification, there can be severe costs associated with a software change. From a schedule and labor standpoint, the effect is often much greater than a cursory glance at the change would imply. Solving late-breaking problems with software changes should not be blindly accepted as the most efficient and least costly path, especially considering the added complexities of hardware and software interactions. 

This article examines the effect of seemingly simple changes at different stages in the medical device development process and introduces strategies to avoid costly changes at the end of development.

Initial architecture and requirements definition

During the initial planning and architecture phase of a project, the team is defining what the device will be. The requirements definition should be considered from a holistic standpoint to make architecture decisions appropriate for the device. It is vital to spend time in this phase defining requirements and planning an appropriate and flexible architecture.

Requirement changes during this phase are the least problematic—design inputs are fluid and the final direction of the product is converging.

Make sure the requirements that affect system architecture choices are established with high confidence and are proven, via breadboard if necessary, prior to exiting this stage of development. This critical step builds a strong foundation for the rest of the project.

Detailed design work

After the system requirements and architecture are defined, the design is implemented. In the medical software environment, firmware (or embedded software) must be developed in increments that allow the critical mechanical and electrical systems to be integrated and tested by engineering before the software systems are completely developed. Because some parts of the system will be developed early in the phase, requirements that affect those subsystems will significantly influence change from the first day of development. 

For example, if device drivers are being developed early in the phase and the display is changed from a six-character LED display to an LCD display, the impact of change will be relatively high. In contrast, if a change is requested on the order of the user interface graphical screens, there may be no cost or schedule impact on medical device development since the detailed design of the user interface (UI) has not yet been implemented. 

Understanding and planning the order of feature implementation and visualizing system integration strategy throughout the development life cycle is critical.

Engineering testing and formal test development

No one wants to change requirements after the bulk of the development effort is complete, but sometimes it can’t be avoided. For example, it may be difficult to predict whether certain features produce a usability problem until system usability testing is done. 

Requirement changes at this phase of medical device development can be costly, but the only remedy is that the development team test the system early and often to prevent changes to requirements late in the process. A change in a software requirement may only result in a two-hour software change, but the overall effect could be a one week schedule slip if, for example, it results in additional code reviews, updates to formal verification and validation test protocol documents, or revision of risk management documentation. 

Further unintended consequences could arise from the new software implementation, causing additional impact to schedule and development costs. A robust testing strategy and the timing of this testing is critical to containing change impacts.

Formal verification and validation testing

Requirement changes during formal product verification testing are even more detrimental than changes during engineering qualification testing. Changes need to go through the formal review process, and get regression tested, as well as justified to FDA that the impact is understood and contained. The quick two-hour software change during the design phase could easily turn into a multi week effort for the entire team to formally justify the change by updating verification protocols and performing regression testing to ensure that the changes do not affect the safety of the product. 

This late change may also catch the eye of subsequent audits or any field adverse event investigation, further increasing the possible impact of change.

The 10 essential components of medical device development strategy

Understanding how software requirement changes affect different points in medical device development does not prevent changes from occurring; changes are a natural aspect of the development process. A development team with the best intentions may still find a usability issue to correct during the final testing phase of the program or need to adjust a feature implementation due to shifting market conditions. However, overarching strategies can be used to reduce the probability of late changes in the software requirements. 

1. Fundamentally, convince the team that “It’s just software” is not the right attitude 

This attitude might work in a nonregulated environment where software updates are easy. It absolutely does not work when the software could affect patient clinical outcomes or when the final product could be in the marketplace for years without updates. If software changes are considered by the team to be as costly as hardware changes, they will perform software-critical testing early and bring problems forward more quickly.

2. Develop complete requirements 

Spend as much or more time developing software requirements early on as you would on hardware requirements. Some software requirements will flow out of the hardware requirements, but do not let this delay defining the software requirements. Examine all of the subsystems affected by system-level requirements, including software. 

For example, if external communication or the ability to upgrade the device’s software remotely is desired, think about what the device really needs to do on a requirements level for the entire system. A requirement that states “communicate externally” could be implemented in many ways, but if the product needs to “communicate wirelessly to a base station transmitting the patient data once a day,” defining this early will produce the right system architecture.

3. Use breadboards and test often 

Although breadboards may seem to add time to the schedule without much benefit, good breadboard development and ad-hoc module testing is invaluable to understanding how the high-risk subsystems in a product drive the necessary software requirements and subsequent implementations. 

A well-designed breadboard removes requirement and development risk early in development, while also serving as a software development platform and enabling initial software module testing.

4. Bound the requirements that cannot be defined 

Concurrent product development sometimes means that requirements cannot be fully defined as early as is desirable. However, from a high level, acceptable performance is definable. 

For example, it may not be known yet if a sensor needs to be read at 1 kHz or 50 kHz, but it is known that it will need to be faster than 10 Hz and not as fast as 200 kHz. If schedule constraints do not allow the system design decisions to wait for a breadboard test to resolve the requirement, create bounded initial requirements. This ensures that the system is designed efficiently for known needs, and project risks associated with unresolved requirements can be prioritized for resolution.

5. Involve the software development team in product architecture selection 

Take time to fully understand the drivers and trade-offs behind different architectural decisions. If a processor is chosen based on the initial requirements, understand what requirements drove its selection over a processor that is either more or less capable. If the requirements that drive the decisions are understood, it will be easier to add risk mitigation for the requirements that are not yet stable, or achieve cost savings if a driving requirement can be adjusted. 

For example, if a graphics module has been chosen to meet current requirements but will not be able to support a potential user interface requirements change, it is important to understand that risk during the process of defining the system architecture instead of after usability testing. A system that is designed to be more capable than the defined requirements can be a reasonable risk-mitigation strategy if there is uncertainty in product direction.

6. Stick to requirements that drive the system architecture 

Once the system architecture has been locked down, it is vital to stick to the critical requirements that drove that architectural choice or be prepared to sidetrack the medical device development effort while a new architecture is chosen.

For example, if the processor was chosen because it met the current low-power usage requirements but did not have the ability to communicate over Ethernet, adding a requirement to communicate over Ethernet could be devastating to the project schedule and would force a rearchitecture effort. Architecture-influencing requirements should be explicitly identified, understood, and agreed on.

7. Decide feature implementation order with purpose 

Develop the software in increments that map to the functionality needed by the hardware to perform engineering integration and usability testing. The entire team should work together to help the software team determine what features are necessary for which prototype builds to test the high-risk subsystems as early as possible. Once these features are determined, order implementation of the features to mitigate requirement risk. 

This is especially important if the unresolved requirements seem peripheral to the main function of the device. The software team may prioritize features on what carries the highest software development risk but not necessarily the highest system development or requirements risk. Determining early that a key usability feature does not function the way you envisioned may delay the resolution of the development implementation, but will lower the overall project risk.

8. Encourage incremental testing releases 

Get the software in key stakeholders’ hands as soon as is reasonable so requirements can be validated. It may not be possible to have a functioning set of software before the initial hardware engineering tests are performed, but leveraging breadboards and having someone outside of the software group testing the software functionality on a regular basis should catch requirement changes. Early usability testing is also valuable in validating requirements and should be considered when planning the features available in the incremental releases.

9. Allow time for UI iterations and usability testing 

Add time in the schedule for UI feature changes that flow from the usability testing performed during development. Early usability testing rarely completely validates a UI design, meaning features need to be adjusted. Deliberately schedule time for the changes so that the development team and key stakeholders expect changes and are prepared for them. 

However, one strategy to avoid a change free-for-all is to determine the amount of time you are willing to allocate to changes and then make a ranked list of the desired changes. The development team can then start at the top of the list and stop when the time runs out.

10. Once in verification testing, only consider critical software changes. 

Once the product enters full system formal verification testing, it is easy to look at a completed product and think “It would be really nice if….” The it’s-just-software attitude can creep back for software changes and the total costs associated are forgotten. Software changes at this stage need to be justified to the same extent as a mechanical change. It may not mean a tooling change or inventory churn, but it will mean reopening the development process. 

This will then require code reviews and regression testing and add the risk of introducing an unexpected failure that takes time to resolve. Running engineering-level verification testing on the complete system before committing to verification hardware and performing verification dry runs before software finalization will go a long way towards catching changes before formal verification.

Leverage strategy to mitigate risk and maximize value

Changes are inevitable during product development. However, understanding and managing the effect of software requirement changes at the different stages in the development process enables smoother development implementation and integration, and allows for appropriate risk mitigation activities throughout. Understanding the impact of requirements on the system architecture, developing with purpose, and testing early and often helps validate requirements during the design process and will minimize the effect of changes during medical device development.

Don’t fall into the it’s-just-software trap. Use these observations during your next development effort to get the most efficient use of resources, ensuring your investment will return the highest value and best meet the intended clinical outcome.

If you need assistance with product development and testing, let’s connect!

###

[This article was originally published in MD+DI]