10 questions to ask human factors vendors in the medical device industry
Getting a life-changing medical device or drug product to market can be challenging – balancing cost, timescales and quality. Over the past 10-15 years, the medical device industry has put an increasing emphasis on human factors or usability engineering (HF/UE). Applied correctly, HF can offer an enormous return on investment as well as adding a competitive advantage to your products once they reach the market. Applied poorly, HF is one of the leading causes for regulatory submission failure, which can lead to lengthy and costly project delays.
A growing demand for regulation has led to more and more businesses entering the HF market to serve the needs of companies like yours.
To help you choose an HF vendor to suit your specific needs, we’ve come up with 10 questions every medical device and pharmaceutical company should think about relating to vendors before they contract them. The answers to these questions will help you decide which company will get your product to market quickest and minimise risks downstream.
1. What proportion of the vendor’s work is in medical devices / pharma?
Lots of companies, consultancies and agencies offer HF expertise. Often, these services form part of a broader design or engineering consultancy. Others will be dedicated HF agencies, focused exclusively on HF services.
As well as the diversity of the teams within the organisations you look at, it’s important to take note of the markets and sectors each vendor works in. Historically, many product design agencies have spread business risk across multiple sectors, including consumer goods, industrial, transport, defence, and medical. Companies working across sectors will typically dip in and out of keeping up to date with relevant industry standards and guidance. Whereas those that work exclusively in one industry will eat, sleep and breathe the standards.
Cross sector agencies do have their place, and drawing influence from external markets to the one you’re working in can help cross-pollinate innovation. However, when it comes to focusing on a rigorous, compliant, design development process, it can be better to work with genuine experts in that field to ensure you’re not left with nasty surprises at the time of submission.
2. How much HF experience in your specific field should your chosen vendor have?
The field of medical devices and combination products is vast, so a vendor simply having experience of the medical device sector is sometimes not enough for your project. You’ll want to make sure you’re working with people who have direct relevant experience in similar or related device categories. Those who have worked on a home use, single use drug delivery device, for example, may not have knowledge or experience of complex surgical environments. It is important to select a team with the right people for your project.
3. Does the vendor have a quality management system (QMS)? And is it robust enough to minimise risk, but flexible enough to allow you to work how you need to?
The British Standards Institute (BSI) state: “A medical device manufacturer’s quality management system is the foundation for maintaining regulatory compliance, driving improvement, effectiveness and achieving stakeholder confidence.”
Noncompliance presents a major roadblock to your device’s lifecycle, not to mention the cost burden of rework. Patient safety is dependent on the quality and consistency of medical products. A QMS provides a higher level of confidence in the ability to consistently achieve and maintain compliance with regulatory requirements.
If you’re wondering whether it is important for an HF vendor to have a QMS, such as ISO 13485, when they are unlikely to be the legal manufacturer of a medical device, consider the following. The purpose of the QMS is to apply a risk-based approach to processes and procedures to help ensure the product (or service) being delivered is fit for purpose and meets the customer needs, is free from defects, and is a high quality product. Now ask yourself the following to help frame the value of a certification and robust QMS in your vendor’s business:
- If the vendor does not have a QMS, how are the following factors managed?
- Identification and qualification of your needs (think of these as design inputs)
- Management of appropriate resources and infrastructure required
- Staff competency and training
- Selection and evaluation of suppliers (third parties)
- Management and traceability of client assets (devices)
- Data analysis
- Identification of non-conformities, opportunities for improvement, and trending of customer feedback
If the factors above are not important to your project, then it would be reasonable to select an HF vendor who does not have a QMS in place. However, if you wish to seek confidence that the vendor has control over the processes and procedures they will be using for your project, it makes sense to select one with a QMS.
4. How does the vendor manage competency and capability in relation to training, inductions, compliance and the size of its team?
Competency and capability are major factors that will inform your decision when choosing an HF vendor. There’s a lot of truth in the saying “people buy from people”. It’s important you select your HF vendor based on an informed decision about the people you’ll be working with. Larger organisations may have account managers or sales teams who you’ll work through. Smaller agencies are likely to offer a more intimate relationship and a direct line of communication between you and the people running your project.
5. How well does the vendor understand current market regulations and health authority expectations?
When you decide to outsource a service, you’ll want to make sure the people you’re contracting are knowledgeable and up to date with the latest regulations and expectations. A vendor who exclusively works in the medical or pharma industry will have a good awareness of the requirements as these will be the foundation of all the projects that the team undertakes.
It’s a good idea to work with a partner who has an in-house professional regulatory consultant or partners with one. This will ensure you’re well supported to navigate the ever-changing landscape for compliance.
6. How does the vendor manage potential project risks?
Medical device development is a risk-based process and human factors should be no different. Your vendor should be able to demonstrate their approach to risk management in their work, both from a day-to-day project management perspective and a study risk perspective. This should include a dedicated project manager for the work as well as a project reviewer.
Vendors should also be aware of risks during studies such as data breaches and GDPR compliance, participant wellbeing and other elements that could jeopardise timelines or budget. Check in with potential suppliers and ask them to talk you through their project risk approach.
7. How does the vendor perform data analysis, verification and reporting?
Would you have confidence in a vendor who says formative studies are cheaper as they are quicker and less robust than a validation? Ask yourself, how much confidence does this offer for the design decisions that will be made from their data?
Working in the medical device sector, you’ll be familiar with the notion of “if it’s not written down [documented] then it doesn’t count”. With this sentiment it is easy to understand the importance of a well-structured and rigorous reporting approach for HF activities.
It’s common for formative studies to be considered “quicker” and “dirtier” than validation studies. That said, you may wish to ensure your HF vendor doesn’t forego a robust analysis process when analysing and reporting the results of a formative study. After all, formatives are your opportunity to make mistakes with the design, learn how humans might interact with your device, design out potential risk and build robust mitigations and controls. All of these goals rely on building a thorough and detailed understanding of the root causes that lead to use errors during formative studies, so you can make sure design developments genuinely solve the issues, leading to a high quality, safer design.
Our advice would be to invest in robust formative studies to maximise the return on investment and pave the way for a smoother validation study. Remember, robust doesn’t need to mean slow and lengthy.
If you’ve watched any HF studies being performed, you’ll recognise that often the vendor will setup multiple cameras to ensure the sessions and interactions with your devices are recorded. Have you ever wondered what happens to these recordings and how they are used? We’d recommend asking whether your HF vendor plans to watch back the videos as a matter of due course. Data verification is crucial to ensure what was seen on the day has been documented as a concise, accurate narrative. At Rebus we routinely review all videos during the data analysis process to ensure the data being reported is reliable and a single source of truth.
8. Does the vendor have design development experience and the capability to provide commercially grounded recommendations and consulting advice?
Unfortunately, HF can sometimes be tarred with a brush of “hindsight is wonderful”. Many historical case studies emphasise a lack of due HF process, where systems have been designed without an appropriate level of consideration for human capabilities. For example, Chernobyl, Three-Mile Island, the poignant case of Elaine Bromiley, all reflect on instances where systems and processes have lacked HF thinking prior to deployment or implementation.
Any good practitioner will therefore champion the role of HF as providing a proactive set of tools and methods which can help to prevent potential use-error. A key consideration for selecting HF vendors and expertise here relates to the ability not only to regurgitate HF principles and best practices, but to turn them into commercially feasible recommendations that are appropriate for your intended manufacturing constraints, production volumes, target users, and use environments.
We’d recommend seeking HF vendors whose teams hold applied knowledge of working within medical device manufacturing companies to help ensure HF is being applied to your product in a commercially sensitive manner, while maintaining regulatory compliance. Knowledge of product processes, manufacturing lead-times, material properties, distribution and supply chains can all help provide informed recommendations that will provide a holistic perspective.
9. How flexible will a vendor be to change requests?
If you’ve worked in medical device development for even just a short time, you’ll know and appreciate that by virtue, development projects rarely go to plan.
When selecting and partnering with an HF vendor, contracts, master service agreements and statements of work are necessary and form a baseline of all legal standings. Nonetheless, as part of the selection process you should be asking your HF vendor how they manage change and deviations from a plan.
Consider your degree of confidence in the timescales you’re requesting deliverables to be provided, and the number of interdependencies those timescales have with other departments, vendors, and stakeholders. Scope creep is an important metric, but there can be merit in selecting a vendor who wishes to work with you, rather than against you. Yes, scope creep happens, but there may be instances where it’s allowable and remains within the overall intent of the existing contract / statement of work (SOW). There are other instances where it is less palatable for either party.
The key consideration will be your vendor’s willingness to work with an element of permissible give and take, versus those who wish to instigate formal change control at every slight deviation and crossroads along the way.
10. Will the vendor outsource any activities to other companies, contractors, or organisations?
As with many organisations, there will likely be some element of the project that your vendor will need to outsource to a third party. For example, it could be printing of study materials, IRB / ethical approval, recruitment of participants, online / live streaming, study venue hire, and in some cases a vendor will outsource the actual conduct of the study or elements of the study to sub-contractors or freelancers. To echo the sentiment of the first point, there is certainly merit in working with experts for specific services.
As we’ve discussed in previous articles, HF work can make up a very significant portion of your development budget and often sit on the critical path to product launch. It’s therefore vital you select the right partner for your project with the care and due diligence you give your whole development programme. We hope that these tips help you in making selections for the right partner on your future projects.
Written by Ian Culverhouse, Director at Rebus.