The reliability of the embedded software used in medical devices and the risk associated with it has become a vital concern. IEC 62304, “Medical device software – Software life cycle processes,” has thus emerged as an internationally recognized mechanism for the demonstration of compliance with relevant local requirements. Part 1 of this article (MDB, October 2017) examined the development of detailed requirements and associated design of medical devices specified by IEC 62304, culminating in a detailed software design in accordance with clause 5.4 of the standard and shown in context in Figure 1. Part 2 details the process applicable to the implementation of that design in source code, the verification and validation of that code, and the ongoing maintenance of the system after release.
5.5 Software Unit Implementation and Verification
Next comes the process of translating the detailed design into source code. To consistently achieve the desirable code characteristics, coding standards should be used to specify a preferred coding style, aid understandability, apply language usage rules or restrictions, and manage complexity. The code for each unit should be verified using a static analysis tool to ensure that it complies in a timely and cost-effective manner.
Verification tools largely offer support for a range of coding standards such as MISRA C and C++, JSF++ AV, HIS, CERT C, and CWE. The better tools will be able to confirm adherence to a very high percentage of the rules dictated by each standard, whereas more lightweight tools will be unable to detect the subtler violations. More comprehensive solutions will also support the creation of and adherence to inhouse standards from both user-defined and industry standard rule sets.
IEC 62304 also states that the manufacturer shall establish strategies, methods, and procedures for verifying each software unit. Where verification is done by testing, the test procedures shall be evaluated for correctness. Among the acceptance criteria are considerations such as the verification of the proper event sequence, data and control flow, fault handling, memory management and initialization of variables, memory overflow detection, and checking of all software boundary conditions.
Unit test tools often provide a graphical user interface for unit test specification, which is used to create tests according to the defined specification and to present a list of all defined test cases with appropriate pass/fail status. The ability to automatically present the graphical presentation of control flow graphs, and to create test harnesses, stub functions, and cover for missing member or global variables means that unit test execution and interpretation becomes a much quicker and easier process, requiring a minimum of specialist knowledge. By extending the process to the automatic generation of test vectors, the tools provide a straightforward means to analyze boundary values without creating each test case manually. Test sequences and test cases are retained so that they can be repeated (regression tested), and the results compared with those generated when they were first created.
Thorough verification also requires static and dynamic data and control flow analysis. Static data flow analysis produces a cross reference table of variables, which documents their type and where they are utilized within the source files or system under test. It also provides details of data flow anomalies, procedure interface analysis, and data flow standards violations.
Dynamic data flow analysis builds on that accumulated knowledge, mapping coverage information onto each variable entry in the table for current and combined datasets and populating flow graphs to illustrate the control flow of the unit under test.
5.6 Software Integration and Integration Testing
Software integration testing focuses on the transfer of data and control across a software module’s internal interfaces and external interfaces such as those associated with medical device hardware, operating systems, and third-party software applications and libraries. This activity requires the manufacturer to plan and execute integration of software units into ever larger aggregated software items, ultimately verifying that the resulting integrated system behaves as intended.
Integration testing can also be used to demonstrate program behavior at the boundaries of its input and output domains and confirms program responses to invalid, unexpected, and special inputs. The program’s actions are revealed when given combinations of inputs or unexpected sequences of inputs are received, or when defined timing requirements are violated. The test requirements in the plan should include, as appropriate, the types of white box testing and black box testing to be performed as part of integration testing.
Testing tools need to have the capability to provide dynamic structural coverage analysis, both at system test level and at unit test level, which is a mechanism to show which parts of the code base have been exercised during testing. The coverage data derived from unit and system test can be combined to provide the most effective way of working for the particular needs of a development project. A common approach is to operate them in tandem, so that, for instance, coverage can be generated for most of the source code through a dynamic system test and complemented using unit tests to exercise defensive code and other aspects.
Testing of changed source code can again be performed using regression testing. It is advisable to validate test cases as a matter of course and perhaps automatically, to ensure that any changed code has not affected proven functionality elsewhere.
The requirements for structural coverage metrics like statement, branch, condition, procedure/function call, and data flow coverage varies depending on device classification, and all are provided by both the unit test and system test facilities within tools such as the LDRA tool suite.
5.7 Software System Testing
Testing at the system level requires the manufacturer to verify the software’s functionality by verifying that the requirements for the software have been successfully implemented. Software system testing demonstrates that the specified functionality exists in the system as it will be deployed, and that performance of the program is as specified.
Software system testing is in many ways the ultimate integration test, and comments relating to integration testing still apply here. Functional (black box) testing with no code coverage can also be performed although it might be desirable to use white box methods to more efficiently accomplish certain tests, initiate stress conditions or faults, or increase code coverage of the qualification tests.
Software system testing tests the integrated software and can be performed either in a simulated environment, on actual target hardware, or on the implemented medical device.
Software System Maintenance
These devices require maintenance when out in the field, and that also involves risks that must be tracked, managed, and mitigated in accordance with the processes and procedures laid out in the standard, as Figure 2 illustrates.
A high percentage of medical device software defects are introduced after product release, and many are related to the application of inappropriate software updates and upgrades. For that reason, the software maintenance process is considered to be as important as the software development process. Clause 6 of the IEC 62304 standard spells out details for the software maintenance process, which has much in common with the software development process.
Subclause 6.3.1 states that the manufacturer shall apply the software development process as mentioned in clause 5 (5.3, 5.4, 5.5, 5.6, 5.7, and 5.8), or an established maintenance process to implement the modifications. Either way, it is important to manage changes in accordance with subclause 7.4.1, which requires that changes be analyzed to prevent introducing any harmful causes that could contribute to a hazardous situation. Furthermore, those changes are to be assessed to determine whether additional software risk control measures are required.
Given the similarity of the required processes before and after release, and assuming successful deployment of high assurance software in the first place, it makes sense to continue to apply the same tool chain as was used in development.