Risk management for IVDs-part 3

Risk management for IVDsPart 3: Reducing and controlling risks to patientsDonald M. PowersThe first two articles in this series discussed the significance of risks fromIVD devices, emphasized the importance of risk management planning,and explored ways to apply the risk analysis principles of ISO 14971.1,2This third article covers the risk reduction and control phase of ISO14971.3Controlling Risks to PatientsA visit to a modern clinical laboratory provides ample evidence of the risk control practices that IVD manufacturers have employed: electronic monitoring systems, integrated procedural controls, fail-safe error messages, automated calibration, refrigerated onboard reagent storage, bar coded labels, and explicit warning stickers. Whether such risk control features originated from planned risk management activities or corrections of unexpected failures, many of these features were commonplace well before risk management came into vogue.4 Regardless of the motivation, their main purpose is to reduce the risks to patients from incorrect IVD test results.As stated in the preamble to the U.S. quality system regulation, FDA expects manufacturers to reduce risks to an acceptable level.5,6The European Union’s IVD Directive goes further by requiring manufacturers to reduce risks as far as possible, and to select risk control measures in the following order: inherent safety by design, protective measures in the device or the manufacturing process, and information for safety.7The previous articles in this series discussed ways that IVD manufacturers can identify hazards and estimate their risks.2 If risk analysis determines that the risk from a possible failure mode is negligible, further risk reduction is not necessary. For the remaining risks, ISO 14971 outlines a logical sequence of risk control stages that manufacturers must follow to conform to the standard (see Figure 1). These stages are intended to be checkpoints to promote systematic decision-making and ensure that IVD manufacturers document their risk management decisions for future reference. A justification for each decision must be included if the rationale is not obvious.Figure 1. Risk control stages. Source: ISO 14971:2000.Analyzing Risk Control OptionsThe first stage is a comprehensive analysis of risk control options, which enables IVD manufacturers to select the most appropriate control measures for each specific risk. Manufacturers can do little to reduce the severity of harm, since that depends mainly on the medical actions taken in response to the IVD test results. For most IVDs, manufacturers can only reduce the probability that harm will occur.ISO 14971 adopted the three-level hierarchy of risk control options from the IVD Directive, thereby placing the emphasis on mitigating risks through design whenever practicable. To show that the risk control measures were selected according to the prescribed priority hierarchy, IVD manufacturers must document that the options available to reduce each risk were identified, classified according to the hierarchy, and evaluated for technical and economic feasibility.Several standards from the International Organization for Standardization (ISO; Geneva), the International Electrotechnical Commission (IEC; Geneva), the European Committee for Standardization (CEN), and the Clinical and Laboratory Standards Institute (CLSI; Wayne, PA) address safety aspects of IVDs. A few standards (e.g., IEC’s new usability standard for medical devices) integrate elements of the risk management process.8 When appropriate,IVD manufacturers should consider such standards as options to control identified risks, since conforming to a recognized standard generally conveys a presumption of risk acceptability.Inherent safety by design for an IVD assay means its performance characteristics and reliability meet the medical requirements of its intended use. The components that affect specificity, precision, stability, and traceability are the keys to improving analytical performance, while sound hardware and software design features can preventinstrument-generated spurious results and enhance the uptime of electromechanicalsystems. In addition, automation can eliminate the potential for human error, bar coding can ensure the proper identification of patients, samples, or reagent lots, and attention to human factors can simplify procedures and eliminate causes of error.Protective measures reduce the probability that incorrect results will be reported to physicians or patients. Examples of protective measures include fail-safe software, which results in error messages instead of incorrect results, and fault-tolerant systems, which continue to operate correctly after a fault occurs. Instrument-based controls are now commonplace in IVD systems, and quality control testing has been integrated into many assays.9 Other examples of protective measures are in-process tests that detect defects introduced in the manufacturing process and final-release tests that verify conformance to performance specifications.Information for safety is the labeling, which includes operating instructions and other information to avoid risks. Historically, IVD manufacturers have relied heavily on labeling to help clinical laboratories avoid generating hazardous results. However, adding new instructions or precautions without first attempting to eliminate the failure causes or to incorporate protective mechanisms is contrary to the philosophy of ISO 14971. The standard reminds manufacturers that ―information for safety is the least preferred me thod of risk control measure, to be used only when other risk control measures have been exhausted.‖ Risk management experts allow a risk reduction of no more than one level on the probability scale from labeling or training. Although mandatory training programs designed to reduce risks can sometimes achieve greater risk reduction, training alone is seldom effective in eliminating errors.10IVD manufacturers often overestimate the ability of laboratory quality control (QC) procedures to detect incorrect test results. When delegating risk controls to clinical labs, design engineers must make assumptions with caution. While a manufacturer’s QC recommendations may provide helpful advice, there is no assurance that labs will follow them. In order to qualify as a protective measure, a QC procedure would have to be an integral part of the assay procedure. The IVD guidelines in annex H of the second edition of ISO 14971, which has recently been advanced to a final draft international standard, make it clear that an IVD manufacturer’s recommended QC procedures, as well as recommended confirmatory tests, are only considered information for safety.11Warnings also have limited effectiveness as a risk reduction measure and should be considered as a last resort. The pres sure to avoid calling attention to an assay’s shortcomings in a package insert has all too often led to vague, innocuous statements that would not pass an objective test for effectiveness. The revised ISO 14971 will include a new annex titled―Information for Safety and Information about Residual Risk,‖ which provides guidance on using information for safety as a risk control measure and promoting true awareness when disclosing residual risks.11Implementing Risk Control MeasuresThe risk control implementation stage begins once an IVD manufacturer decides which options yield the greatest risk reduction. Often, more than one risk control approach is needed. Manufacturers have to verify each risk control measure twice, first to document that it was implemented correctly, and then to verify or validate its effectiveness in reducing the risks to the desired level.IVD manufacturers should set up risk management documentation to ensure clear traceability from each identified hazard and hazardous situation to the associated risk control measures. While manufacturers are able to verify some hardware- and software-based risk controls individually through fault injection techniques, a means of testing and evaluating fault-tolerant systems,12 most will demonstrate the cumulative risk reduction from multiple control measures during design validation.Evaluating and Disclosing Residual RisksRisks are rarely eliminated completely. Risks that remain after a manufacturer has implemented all risk control measures are called residual risks, which must meet the acceptability criteria in the risk management plan. Confirming that the residual risks from each hazardous situation have been reduced to an acceptable level is a pivotal checkpoint in the risk control process.If the acceptability criteria are met, then manufacturers must determine what information to include in the labeling to disclose the residual risks. They must also consider the comprehension level of end-users and the detail that users need in the labeling to make informed decisions. If the acceptability criteria are not met, then risk reduction efforts continue until further reduction is not technically or economically feasible. If the risk acceptability criteria are still not met, an IVD manufacturer must either abandon the project or justify it based on a risk-benefit analysis.ISO 14971 recognizes that end-users need to be aware of the residual risks in order to manage them. Such disclosure involves providing the information necessary to understand the residual risks and minimize exposure to the hazards.In IVD labeling, the information in the ―Limitations of the Method‖ section is an example of residual risk disclosure. This part of the labeling contains information such as drug interferences that the IVD manufacturer could not eliminate. Disclosure of drug interferences is not a risk control, since laboratories do not have any practical means to monitor the presence of drugs in specimens. However, in the spirit of ISO 14971, IVD manufacturers should opt to disclose these residual risks only after their efforts to eliminate the interferences prove unsuccessful. The information in the ―Performance Characteristics‖ section is another example of residual-risk disclosure. With such information, a lab director can make informed decisions about whether a particular assay will be useful for its intended medical purpose.The details about how IVD manufacturers control specific risks are becoming more important with the advent of equivalent quality control.13,14 This concept allows a laboratory to build on a manufacturer’s risk controls and design a holistic quality control program. This requires knowing not only the circumstances that can lead to incorrect results but also the internal control mechanisms that can detect or suppress them.15,16Although in principle an IVD manufacturer is responsible for deciding what information is necessary to include in the product labeling, in practice much of the content has already been decided by labeling regulations and standards.17–20 Several examples of prescribed information for safety are intended to help laboratories avoid common use errors (see Table I).Table I. Information for safety to avoid potentially hazardous use errors. Source: ISO/DIS 14971:2005,annex H.Performing Risk-Benefit AnalysisA risk-benefit analysis is required when efforts to reduce residual risks to acceptable levels have failed, yet the IVD manufacturer believes the product is still worthwhile. Unfortunately, while methods for estimating risks with reasonable confidence exist, a standardized approach to estimate the benefits has not been developed. ISO 14971 acknowledges thatrisk-versus-benefit decisions are largely matters of judgment by experienced and knowledgeable individuals. Some guidance for conducting risk-benefit analyses is included in annex D.6 of the draft second edition of ISO 14971.Risk-benefit analyses of individual residual risks are difficult and provide at best only enough information to decide whether to proceed with a design project. Having reduced each individual risk in an iterative manner until it is either acceptable or irreducible, any residualrisks that remain outside the acceptability criteria require a decision whether to proceed with the project. If it is obvious that an individual risk is irreducible and will jeopardize an IVD manufacturer’s ability to meet the overall acceptability criteria or to provide a product with medical benefits that outweigh its overall residual risks, the development project may be stopped. Outside clinical consultation often helps in making this judgment. If the answers are not obvious, the project will usually continue.Addressing Risks Resulting from Risk Control MeasuresOccasionally, implementing risk control measures creates new hazardous situations. For example, a proliferation of warning labels can confuse laboratory operators and desensitize them to hazard warnings. Similarly, an overly complicated set of instructions may be difficult to follow without errors. There are also abundant examples in the recall literature in which actions intended to reduce risks inadvertently created new hazards. IVD manufacturers must review the results of the risk control phase to avoid increasing risks.In some cases, IVD manufacturers may introduce a new hazardous situation intentionally as the lesser of two risks. For example, an instrument programmed to give an error message when a fault occurs is trading off the risk from a delayed result against the risk from a possible incorrect result. The actual degree of risk depends on how the test results are used. For self-testing glucose monitors, if a diabetic were not able to perform blood glucose tests in a timely manner because the meter malfunctioned, there is a chance of accidental overmedication with insulin, leading to hypoglycemic shock. While an error message may seem clearly preferable to an incorrect glucose result, manufacturers must estimate the risks created by a delay in availability of the results and evaluate them against the risk acceptability criteria.On the other hand, a delay in the availability of routine hospital laboratory test results while an automated instrument undergoes repairs is less likely to cause serious harm. Labs anticipate the need for service calls and are expected to have backup plans for lab tests that are urgent. Acceptability of the risks from delayed results is determined in each case by a risk assessment.Verifying Completeness of Risk ControlA verification checkpoint in the risk control process was added as a safeguard against possible overlooking of any necessary risk control measures in the rush to launch a new product. IVD manufacturers are required to verify that they have adequately addressed all hazardous situations identified in the risk analysis phase and any new ones created by the risk control measures.ISO 14971 also requires manufacturers to record the results of all risk control activities in the risk management file, including each decision to take or not take action, every review andverification check discussed above, and the results of the risk reduction activities. This requirement reinforces the need for IVD manufacturers to establish a logical documentation system at the outset, with the necessary traceability to show that they have addressed all hazardous situations and reduced all risks to acceptable levels.Evaluating Residual-Risk AcceptabilityEven after all the individual risks have been mitigated, there is still the question of whether the overall risk of using the IVD device will be acceptable. After documenting the reduction of each individual residual risk, an IVD manufacturer compiles the information in a way that reveals the overall residual risk. The overall residual risk is the risk that remains after a manufacturer has implemented all risk control measures. The risk of using the device may not be acceptable because of the cumulative probability of serious harm from manylow-probability failure modes. In that case, an IVD manufacturer must apply further risk control measures to make the overall residual risk acceptable.Estimating overall residual risk is more challenging than estimating risks from specific device failures, and up to now, little guidance has been available to help IVD manufacturers in evaluating it. A discussion of several possible approaches has been added to the second edition of ISO 14971 as annex D. Two tools described in this annex are particularly applicable to IVDs. In event tree analysis, multiple individual risks from a specific sequence of events can be considered together to determine if the overall residual risk is acceptable. In fault tree analysis, the combined probability of harm from several hazardous situations can be estimated from their individual probabilities. The output of a cause-consequence analysis, a blend of fault tree and event tree analysis, is a diagram for documenting and communicating relationships between risks and their initiating causes.21For well-understood IVD assays, confirming that the overall residual risk is acceptable by performing a detailed risk-by-risk comparison to a comparable assay on the market may suffice. If manufacturers use this approach, they must ensure that information about risks of the comparable assay is up-to-date. Current product labeling and performance data can often be obtained from manufacturers’ Web sites or from clinical laboratories. Peer-reviewed data from product evaluations and clinical experience are found in laboratory journals. In addition, a wealth of adverse-event experience, recalls, and summaries of product safety and effectiveness is available on the FDA Web site.22Another approach especially useful for evaluating the overall residual risk of new IVD systems is to engage application specialists (i.e., laboratory technologists) who are familiar with using similar devices. An important caveat is that they must not have been directly involved in developing the device being evaluated.The application specialists evaluate the system in a representative clinical lab environment to confirm the acceptability of its overall residual risk, specifically considering the characteristicsrelated to safety identified in the risk analysis (e.g., performance, usability, and reliability). In addition, the specialists review the operating instructions and other essential labeling information to identify instructions that are difficult to follow, excessive reliance on warnings, and conflicting requirements. Based on their professional knowledge and experience, they would judge whether the overall residual risk of the system is acceptable.What if the overall residual risk is judged not acceptable according to the criteria in the risk management plan, and further risk reduction is not practicable? An IVD manufacturer has to decide whether the potential medical benefits outweigh the overall residual risk.Risk-benefit analyses of overall residual risk are not as common for IVDs as they are for life-sustaining medical devices. Manufacturers must spend considerable energy in developing a defensible comparison of overall residual risk with the overall benefit that the IVD device provides. Quantifying the medical benefits of an IVD assay with an unacceptable potential for incorrect results is difficult. When the overall residual risk falls outside management’s risk acceptability criteria, IVD manufacturers usually abandon plans to commercialize the device.If an IVD manufacturer pursues a risk-benefit analysis, the decision process is similar to that used for analyzing the individual risks. If the evidence supports a conclusion that the medical benefits outweigh the overall residual risk, the overall residual risk is judged acceptable. Otherwise, the manufacturer has no other choice but to abandon the product.ConclusionA key element of the last IVD design review prior to a new product launch is a comprehensive review of all the risk management activities that have been undertaken. An IVD manufacturer must verify three things: proper execution of the risk management plan, acceptability of the overall residual risk, and t he entire organization’s readiness to carry out its ongoing risk management duties during the life of the device. The documented summary of this review becomes the high-level risk management report required for conformance to ISO 14971.The revised ISO standard no longer requires the risk management report to document traceability of hazards to risk controls. This requirement was eliminated because it caused the final report to become unwieldy, particularly for complex systems that require extensive risk analyses. Nevertheless, IVD manufacturers must still demonstrate traceability in the risk management file. A guidance document from the Global Harmonization Task Force presents an example of an efficient risk management summary table that IVD manufacturers can use to document how the individual risks were controlled for a given device, and the status of implementation and verification of the risk control measures.23 Such a document will be an invaluable tool for continuing to manage risks through a product’s life cycle.The next installment of this series will discuss the production and postproduction monitoring stages of the risk management process, and explore how these aspects of the riskReferences1. DM Powers, ―Risk Management for IVDs, Part 1: Planning and Documenting the Risk Management Process,‖ IVD Technology 12, no. 2 (2006): 28–33.2. DM Powers, ―Risk Management for IVDs, Part 2: Assessing Risks to Patients from Incorrect Test Results,‖ IVD Technology 12, no. 3 (2006): 24–31.3. ISO 14971:2000, ―Medical Devices: Application of Risk Management to Medical Devices‖ (Geneva: International Organization for Standardization).4. DR Wallace and DR Kuhn, ―Lessons from 342 Medical Device Failures,‖ National Institute of Standards and Technology Web site (Gaithersburg, MD: 1999 [cited 23 March 2006]); available from Internet:/wallace/hase99.pdf.5. Federal Register 61 FR:52601–52662, October 7, 1996.6. FDA, ―Guide to Inspections of Quality Systems,‖ FDA Web site (Rockville, MD: August 1999 [ci ted 23 March 2006]); available from Internet: /ora/inspect_ref/igs/qsit/QSITGUIDE.PDF.7. ―Directive 98/79/EC of the European Parliament and of the Council of 27 October 1998 on In Vitro Diagnostic Medical Devices,‖ Official Journal of the European Communities L331 (1998).8. IEC/CD 62366, ―Medical Devices: Application of Usability Engineering to Medical Devices‖ (Geneva: International Electrotechnical Commission, 2004).9. FD Lasky, ―Technology Variations: Strategies for Assuring Quality Results,‖ Laboratory Medicine 36, no. 10 (2005): 617–620.10. ME Maddox, ―Human Factors: Designing Medical Devices to Minimize Human Error,‖ Medical Device & Diagnostic Industry 19, no. 5 (1997): 166–178.11. ISO/DIS 14971:2005, ―Medical Devices: Application of Risk Management to Medical Devices,‖ 3rd ed. (Geneva: International Organization for Standardization).12. MC Hsueh, TK Tsai, and RK Iyer, ―Fault Injection Techniques and Tools,‖ Computer 30, no. 4 (1997): 75–82.13. MB Bandy, ―CLIA Quality Control Requirements,‖ IVD Technology 11, no. 2 (2005): 24–29.14. DM Powers and T Roscoe, ―QC for the Future,‖ IVD Technology 11, no. 9 (2005): 22–27.15. L Ochs, ―QC for the Future: CLSI Standard Development and Option 4 Proposal,‖ Laboratory Medicine 36, no.10 (2005): 639–640.16. DM Powers, ―Laboratory Quality Control Requirements Should Be Based on Risk Management Principles,‖ Laboratory Medicine 36, no. 10 (2005): 633–638.17. Code of Federal Regulations, 21 CFR 809.10.18. EN 375:2001, ―Information Supplied by the Manufacturer with In Vitro Diagnostic Reagents for Professional Use‖ (Brussels: European Committee for Standardization).19. EN 591:2001, ―Instructions for Use for In Vitro Diagnostic Instruments for Professional Use‖ (Brussels: European Committee for Standardization).20. ISO 18113, ―Clinical Laboratory Testing and In Vitro Diagnostic Test Systems—In Vitro Diagnostic Medical Devices—Information Supplied by the Manu facturer (Labeling)‖ (Geneva: International Organization for Standardization).21. Center for Chemical Process Safety, Guidelines for Hazard Evaluation Procedures with Worked Examples, 2nd ed. (Hoboken, NJ: Wiley, 1992).22. ―Databases on the FDA Website,‖ FDA Web site (Rockville, MD: [cited 23 March 2006]); available from Internet: /search/databases.html.23. ―Implementation of Risk Management Principles and Activities within a Quality Management System,‖ Global Harmonization Task Force Web site (Brussels: 2005 [cited 23 March 2006]); available from Internet: /sg3/inventorysg3/sg3n15r82005.pdf.Copyright ©2006 IVD Technology。