Work groups seek to identify metrics to ensure quality is maintained throughout the total product life cycle
The Medical Device Innovation Consortium and Xavier Health have come up with suggested metrics to help medical device manufacturers improve the quality of their products and help the U.S. Food and Drug Administration enhance the consistency and quality of their inspection assessment. The proposed metrics address medical device quality from R&D, to product assembly or manufacturing, to their use in the hospital or other healthcare facility.
While work continues on that process, MDIC has also released metrics that may help supply chain executives, clinicians, value analysis professionals and others compare the quality of similar products from multiple manufacturers.
Founded in 2012, MDIC is a public-private partnership created to advance medical device regulatory science throughout the total product life cycle, and to enhance trust and confidence among stakeholders. The organization works in the pre-competitive space to facilitate development of methods, tools, and approaches that enhance understanding and improve evaluation of product safety, quality, and effectiveness.
Case for Quality
In 2014, the FDA awarded a grant to the non-profit Medical Device Innovation Consortium to launch an ongoing “Case for Quality” initiative, designed to foster broad-based collaboration and improve the tools and methods used in advancing device quality.
The roots for Case for Quality lie in an October 2011 report by McKinsey & Company for the FDA titled “Understanding Barriers to Medical Device Quality,” explains Dwight Abouhalkah, program manager, Case for Quality, who is on loan to MDIC from Johnson & Johnson.
“Research showed that the interaction between industry and the FDA is fairly complex, with multiple challenges,” says Abouhalkah. “The Case for Quality was created to facilitate ongoing dialogue that will ultimately benefit patients, providers and the medical technology community.”
Case for Quality shifts the medical device culture away from compliance to regulations, to “higher sustained product quality, for the benefit and improvement of patient outcomes,” he says.
For manufacturers
The FDA and Xavier Health brought their Quality Measures Initiative under the MDIC Case for Quality initiative in late 2014 with a working group comprised of manufacturers, consultants and FDA officials. “The goal of this initiative was to identify ways for industry to proactively and predictively measure the risk to its own product quality, which would therefore enable industry to focus on improving product quality commensurate with the need,” reports MDIC in a recently published report. These metrics are the first in a series of recommendations coming this fall from the MDIC’s Case for Quality initiative.
Convening at scheduled forums through 2015 and 2016, the Case for Quality participants worked to identify quality-related metrics intended to answer three key questions:
- How can manufacturers improve their design, engineering and manufacturing processes to ensure the reliability, safety and effectiveness of their devices?
- Are there early, quality-related signals or red flags that manufacturers can identify and correct in real time?
- After a device is marketed, is there adequate monitoring and feedback to alert manufacturers about problems either in product design or their manufacturing processes?
Once developed, these metrics would be piloted by volunteer organizations in order to analyze their effectiveness to predictively measure risk to product quality.
“Ideally, metrics assist in the detection of underlying root causes that need to be addressed in order to prevent recurrence,” said MDIC in its report. “The ultimate goal of a robust metrics program is continual improvement throughout the total product life cycle, such that the root cause of the failure is taken back to the earliest stages of development as possible in order to improve the outcome for any current and future product.”
From pre- to post-production
The three metrics suggested by the working group concern the pre-production, production and post-production processes.
- Pre-production: “If the research and development process is conducted with rigor, the rate of change to the product and/or process during device transfer should be minimal,” reports MDIC. As a result, the pre-production metric is designed to track the number of changes that occur during the transfer stage – that is, at the point in which products switch from the concept stage to production – that were triggered by product and/or process inadequacies. Assessing this metric enables organizations to track the frequency and volume of changes that could possibly have been avoided by a more robust research and development system.
- Production: The production metric calculation that was chosen is the Right-First-Time (that is, no defects) metric that many organizations already track, according to MDIC. Rather than use the metric solely to assess the efficiency of the production process, however, the working group recognized that by triaging the root causes so as to isolate those related to product and process inadequacies, manufacturers could continue assessing the effectiveness of their development processes.
- Post-production: The work group found a number of post-production metrics to be important in painting a holistic picture of product performance on the market, so multiple indicators are included in the final metric that are commonly tracked by organizations: service records, installation failures, complaints, medical device records, recalls by number of units involved, and total number of recalls.
The FDA is exploring the use of the three MDIC metrics as well as additional metrics that may specifically address product quality in the FDA’s upcoming metrics pilot initiative.
Tools for providers
The initial work of the Case for Quality has focused on the FDA and industry, says Abouhalkah. But MDIC has included providers in the mix.
For example, one of the Case for Quality’s working groups is investigating developing a “maturity model,” that is, a standard model that providers could use to identify which device firms have a mature quality system, demonstrating an ability to reliably develop and manufacture high-quality medical devices. That information could help providers make product decisions, as public and private payers institute risk-based payment programs based on quality metrics, such as reduction of healthcare-acquired infections.
A second group – the Product Quality Outcomes Analytics (PQOA) team — released a report in September on the feasibility of developing a tool to provide value analysis committees Consumer-Reports-like ratings of the quality of medical devices from multiple manufacturers. (See related article.)
Editor’s note: More information about the MDIC Case for Quality initiative is at mdic.org/cfq/
Scorecard for providers
How about a tool that would let providers compare the quality and effectiveness of similar products?
Clinicians, supply chain executives and value analysis committees are often at a loss when trying to compare the quality and effectiveness of Manufacturer A’s products with those of Manufacturer B. Comparative data isn’t available, at least not in an easy-to-access or easy-to-use format.
The Medical Device Innovation Consortium’s (MDIC) Case for Quality Product Quality Outcomes Analytics (PQOA) working group released a report in September on the feasibility of developing a tool to provide value analysis committees Consumer-Reports-like ratings of the quality of medical devices from multiple manufacturers.
Stakeholders, such as hospital value analysis committees, require accurate and complete data to make educated decisions to improve patient access to high quality devices, says the team in its report, “Feasibility and Effectiveness of Analytics for Medical Device Product Quality Outcomes.” Three challenges face those trying to gather data on medical device quality:
- Lack of unbiased, relevant and available data.
- Need for consistently applied performance measures and analytical methods.
- Lack of a secure process or operating model to enable and encourage individual companies to be fully transparent about product quality.
The project team defined the following quality domains for measuring product quality:
- Safety: Device does not compromise the clinical condition or the safety of patients, or the safety and health of users.
- Effectiveness: Device produces the effect intended by the manufacturer relative to the medical condition(s).
- Reliability: Device system or component is able to function under stated conditions for a specified period of time.
- Patient Satisfaction: Device is perceived to meet or exceed patient expectations of usability and outcome.
- Usability: Device minimizes the risk of user errors by patients or clinicians.
- Availability: Device is available to fill first request orders.
- Compatibility: Device is compatible with related devices or drugs, the user environment or relevant standards.
By the end of 2017, the MDIC team hopes to have well-documented system for accessing and sharing device quality data. “If this goal is reached, third-party data analysis teams could use the methods developed to consistently provide accurate information about device quality,” said the report’s authors.
To view the report, “MDIC Case for Quality Program: Feasibility and Effectiveness of Analytics for Medical Device Product Quality Outcomes,” go to http://mdic.org/wp-content/uploads/2016/08/Final.MDIC-Product-Quality-Outcomes-Analytics-Report.pdf
Real-world evidence
How well do medical devices work in the field? What effect – if any – do they have on patient outcomes?
Post-market information like that is hard to get. But the U.S. Food and Drug Administration recently awarded the Medical Device Innovation Consortium $3 million in seed funding to try to find a way.
The funding will establish the Coordinating Center for the Medical Device National Evaluation System for health Technology, or NEST. The initial phase will include demonstration projects piloting methods for tracking medical device data and patient-reported outcomes through the use of real-world evidence.
“This use of real-world evidence to support product approvals/clearances of public health importance has the potential to shift premarket data collection to the postmarket setting and to meet postmarket data collection commitments through a modern system that leverages electronic health information generated in the clinical and home setting,” according to MDIC.
“There is great potential in using real-world data to foster innovations in medical device technology that will lead to optimized outcomes for patients and improved quality of life,” MDIC Board Chairman Mike Minogue, president, CEO and chairman of Abiomed was quoted as saying. “In addition, real-world evidence will help measure the cost-effectiveness of these new technologies.”