Health care research: Stimulating simulations

Editor's note: Tom Donnelly is vice president, head of medtech at research firm Branding Science. Michael Gundrum is managing director, North and Latin America, at MTS the Wet Lab Company.  Samantha Swanson is research manager at Branding Science. 

When simulating procedures with medical devices it is important to keep as close to reality as possible, whether you are gathering insights for commercial teams or performing human factors studies for R&D teams. Reading a description of a device and seeing an image can be useful for some marketing research but it does not provide the same level of feedback as hands-on testing in the use environment by the intended end user. 

How do you conduct such projects? This article will describe tips and tricks in conducting simulated medical device procedures by providing: an overview of human factors and the use of wet labs; guidance on logistics for global projects; and advice on data collection, analysis, reporting and deliverables.

Market understanding

Most readers are focused on helping commercial teams with marketing research, such as gathering market understanding, communications testing and brand-growth research. Such work is done for different types of industries, including consumer brands, finance, business-to-business and health care. When gathering insights for medical devices and technology (medtech), human factors (HF) studies may also be done to help guide their design for R&D teams. 

Human factors focuses on the interactions between end-users and medical devices in the intended use environment. FDA guidance says that many use errors cannot be anticipated until the device is used in simulated and observed situations.1 While HF should be conducted in as close to the real procedures as possible, sometimes it is not ethical or possible to conduct device testing during the real procedure. In such cases, you might opt to do a simulated procedure instead. 

There are different types of human factors studies, such as aiding the iteration of the design of a self-injection device, examining the safety of medication storage, usability of an electronic health record system or ways to make operating rooms safer. 

Wet labs

Conducting a medical procedure that is close to real life is called a simulated procedure. If you are using an artificial model of a human organ (e.g., eye, brain), it is a dry lab, whereas if you are using real animal tissue or hazardous material, it is a wet lab. Wet labs are conducted for a variety of reasons. Surgeons need hands-on training before they operate on a patient. When a new device is approved for use, companies set up exhibitions at conferences to allow surgeons to try it in a simulated setting. In the context of this article, we are focused on gathering feedback on a device either for marketing or R&D. 

There are a few reasons when you might want to use a wet lab for research: 

• there is no other way to simulate the procedure;
• the feel of the device or procedure is dramatically different between artificial and animal models; 
• the outcome of the procedure differs when animal tissue is not used.

There are different approaches to conducting wet labs and it is important to consider the logistics of each when doing global research. Should you conduct research in a lab/office setting or at a research facility? Research in a R&D lab or office setting has certain advantages (Table 1). It is convenient to be at the manufacturer’s location. You save money by not having to rent space. Access is easy because you are in control of the space. The research can be conducted more quickly because travel and shipping of equipment are not needed. 

However, there are important disadvantages. There is no viewing room behind a one-way mirror. It is not double-blinded research, which can cause privacy and/or bias issues. Observers are obvious so respondents might be less candid than they normally would be. The client team chatter may be overheard by respondents. A higher incentive may be required for respondents to come into a place that they normally are not used to for research studies and recruitment would be constrained by the geographic location of the office. 

Conducting the research in a standard central-location viewing facility offers several advantages. There is a viewing room with a one-way mirror. The research can be double-blinded. Respondents are more candid and less biased. The team can discuss findings in the background and suggest adjustments. The geographic region(s) can be selected to meet recruitment needs and research can be conducted in multiple locations if desired. 

Of course, there are some disadvantages. The research will take longer due to time for travel and shipping equipment. The costs are higher because you have to rent the facility, ship equipment and the team has travel expenses. There are also logistical challenges securing the facility for the desired research dates and coordinating shipments. 

HF lab vs. standard MR facility

There are two types of research facility you could choose: a specialized human factors lab or a standard marketing research facility. A specialized lab may provide an easier venue for the research. In some cases, the facility is already set up for a configuration of a hospital room, an operating room or a doctor’s office. The disadvantage is that when you go from location to location, the setup may not be consistent. In addition, you still need to set up any special equipment that you are testing. Such facilities can be more expensive and harder to find than standard marketing research facilities. 

Another approach, which has several advantages, is retrofitting standard central-location facilities. You can use any central-location facility and duplicate your setup across all markets, not just domestically but also globally. The setup can be customized to your project needs. Important disadvantages include: the research preparation (getting furniture and equipment set up as needed) can be more extensive; and the needs may vary by location, so researchers need to know the configuration of each facility to be prepared for customization. 

Setting up wet labs

Figure 1 shows one way a wet lab may be set up. In this illustration, there is a surgeon providing her qualitative feedback. The moderator is conducting the interview and making specific observations of the procedure. The moderator might take pictures at select moments. There is a wet-lab technician helping during the interviews to make sure that all the materials (e.g., animal tissue, microscope, surgical equipment) is managed properly. In addition to the camera that records the entire room, you can use a second video camera to zoom in to get a close-up of the procedure. 

Figure 2 shows equipment used for a cataract study, including a phacoemulsification machine (phaco), a monitor, surgical instruments, an artificial head to hold the eye model and disposables needed for the procedure. Of course, a microscope used in the OR is too large to transport. Instead, you can use a smaller version with the same-quality microscope that can attach to a normal table. You can get one that has a HDMI camera which allows a video feed. A monitor in the room allows the moderator to see the same view as the surgeon. The same feed can be viewed in the backroom, as well as recorded for later analysis. Figure 3 shows an example of a retrofitted standard viewing facility for a simulated brain surgery. The surgical instruments are set out on the table. The surgical device is connected to a monitor that the respondent can watch while doing the surgical procedure. 

Animal tissue vs. artificial model

One key decision is whether an artificial model or animal tissue will be used in the simulated procedure. There are advantages and disadvantages of each, which are easier to articulate using a specific example: cataract surgery. 

Wet labs in ophthalmology tend to use porcine eyes because they are very similar to human eyes. The advantages of using them are that the surgical outcome and the feeling of the surgery are very similar to surgery on the human eye. There are disadvantages that need to be considered. The freshness of the tissue is critical because the eyes become cloudy over time. Also, they need to be kept cool in transport and refrigerated at the facility. In addition, the acquisition can be challenging, especially outside the U.S. and EU5. The advantage of the artificial eye is that there are no logistic problems in terms of acquisition, shipment and storage. You can have them individually produced so they can be customized to the study. Some clients like to have artificial eyes specially made for their device. However, the essential disadvantages are the less-realistic outcomes and feel. In fact, surgeons may refuse to simulate surgery with an artificial eye because they are not used to using it.

Logistics 

There are many factors to take into account for the logistics of conducting simulated surgery, especially wet labs. It takes substantial time, money and upfront planning. Researchers must consider the geographic locations (e.g., U.S., EU5, other), equipment needed, type of model (i.e., animal vs. artificial). The research team has to determine if it is better to try to source things locally (e.g., animal tissue, surgical equipment) or ship globally. Some countries (e.g., Brazil and Japan) require at least 30 days to get through customs. You need detailed documentation with the shipment explaining what the contents are; that it is for market research or human factors testing and is not for sale, etc. In some countries or regions (e.g., Latin America) it is difficult to transport devices and equipment with you given that it may be seen as potentially sold for human use. If multiple team members are going to the research location, consider having each one carry the devices so that there are backups if an individual is not permitted to bring the device in transport.

The research team should create a master list of everything that is needed, taking into account how many respondents there are, how many times the procedure is performed, who is sourcing the item and how these may differ by geographic location. You should include who will be at each location, what the dates are and what the facility needs are.

In some cases, the tissue needs to be ready for the part of the surgery when the medical device will be used. In the example of cataract surgery, there are multiple steps in the procedure. If the device you are testing is needed at a later step in the surgery, you have the option of limiting the research to that step. As shown in Figure 5, there are different approaches to the preparation of the animal tissue for conducting wet labs with this example: ship pre-prepped, prep on-site or respondent preps. 

With the first option when you ship pre-prepped, the animal tissue is sourced and one surgical technician can perform the initial steps of the procedure. The tissue can then be packaged with ice packs and shipped to the research facility overnight. 

Advantages to shipping pre-prepped:

• You do not need the equipment used for preparing them (e.g., phaco) at each research location.
• Interview time is reduced because you are focused on only one part of the procedure. 
• There is a standardization of the tissue which is prepared by one surgeon. 

Disadvantages to shipping pre-prepped:

• There will be tissue degradation over time.
• Typically, international shipping of animal tissue is not possible.
• It takes time to get and prepare the tissue.
• Logistically, it is challenging to coordinate the research times with the preparation and shipment times. 

With the second option when you prep on-site, you have the prepping equipment at the research location as well as the animal tissue. A surgical tech preps just before the research takes place. 

Advantages to prepping on-site:

• Allows for the tissue to be fresh, which is ideal for the outcomes.
• Reduced interview time because the work is done prior to the interview.
• You have standardized tissue because one surgical tech prepares all of the tissue consistently. 

Disadvantages to prepping on-site: 

• Although it saves on interview time, it adds to the total research time. 
• You need an extra room at the facility where the surgical tech does the prep work. 

With the third option, the respondent preps the tissue as part of the research interview. There are advantages to this approach. The tissue is fresh. It is most realistic, as the surgeon performs surgery as usual, from beginning to end. You may gain additional insights from this part of the procedure, even if it is not directly relevant to the device being tested. The surgeons use their own surgical technique, which can make the next step in the procedure more in line with their standard surgeries vs. starting midway with another surgeon’s setup.

There are some disadvantages. You need the prep equipment, which increases costs and increases the logistical burden. The interview is longer and less focused on the part of the procedure of interest. The recruitment can be more challenging due to the need for a longer interview. 

While all three approaches are viable, wet-lab technicians prefer shipping pre-prepped because there is less equipment to manage on-site and the procedure is more focused. Researchers prefer prepping on-site with one surgical tech because there is consistency, the tissue is fresh and the research is focused. The third approach is most realistic when the respondent performs the entire procedure. The outcomes are closer to what would happen in an actual surgery.

Selecting respondents and method

Who is interviewed and how the research is conducted will depend on the study objectives and the device being tested. For health care practitioners (HCPs), example criteria might include current users of the device, perhaps with different levels of experience, and/or potential users of the device with no prior experience. For example, when testing a cataract surgery device, criteria would include the number of cataract surgeries performed in a given time period, as well as availability and willingness to participate in a wet-lab study. If the part of the procedure being studied includes a surgical technician (tech) or a nurse, you might consider having a duo interview that includes both respondent types. In fact, you might recruit a surgeon-tech pair who work together so that the interaction is more realistic. 

If the device is used by the patient, it will not typically be conducted as a wet lab. Instead, you might perform a usability study with a model of the body part that involves the device or use some other type of simulation, such as injecting an orange. As with the HCP example, you could include current users with varying experience and/or potential users with no prior experience. You might also be interested in patients who are willing to pay for a premium vs. only for the basic products. 

Tips on data collection, analysis, reporting and deliverables

When designing HF studies, you must prepare a rigorous study protocol that includes the objectives, study requirements, methodology details, analysis details, reporting of results, study material, all of the equipment and the animal tissue/artificial models used. This is standard for HF work with the R&D team but differs from the type of work done for a commercial team. When thinking about the data collection, you also must consider various control parameters that might affect the procedures, such as the lighting, the temperature, the type of model (i.e., animal vs. artificial), the research approach (e.g., pre-prep vs. respondent prep) and the instruments used (e.g., familiar vs. novel to respondent). Typically, we’re striving for a double-blinded study. It is advised that you scrub logos on equipment. You can request that respondents do not show their name or institution (e.g., on their scrubs or their badge). Having the research at a third-party location helps make the research double-blinded and minimizes bias. 

During the research, there are different types of data collected. The moderator makes specific observations and can provide ratings. There may be a knowledge task for the respondent after they have learned a new procedure. Respondents can provide qualitative feedback and ratings during the interview. All of this information can be included in the analysis and reporting.

One important aspect of device research is recording the procedure and allowing the research team to view live remotely. On a recent medical device testing project for brain surgery, Curator Video was used, which includes an iPad and tripod. Unlike standard video, which provides a bird’s-eye view, this enabled a close view of the device use. The videos are stored on a platform for easy search via AI transcripts. Videoclips can be captured for later reporting and the team can provide tags for later searches. 

In a recent project, we used two Curators. One was focused on the hands of the surgeon and the other was focused on the display that they were watching. The video can be up close and is high-quality, which is important for analysis and reporting. 

As with all qualitative research, moderation skills are key. Questions should be properly framed so you are not leading the respondent. When interviewing surgeons, you also need a soft touch, especially asking about errors. Sometimes respondents are unaware of their errors. As always, you have to carefully consider your use of ranking and rating scales (e.g., even vs. odd). You should consider if the respondent should rate each device individually vs. making comparison ratings between two devices. A moderator sheet is recommended, to make it easy to jot down ratings and notes live for quicker reporting. Having team instructions is helpful when you have multiple teams going out to different locations. These might include a schematic of the lab layout, instructions for how to use the equipment, tips on how to ensure recordings are done and parameters for backroom conduct. 

Reporting style is typically tailored to the individual client’s need. In insights work, the norm is marketing-style PowerPoints, perhaps with video clips and infographics. For R&D teams, you might provide more formal reports and protocols that are very detailed in Word. The analysis described in an R&D report may seem more like a quantitative research report vs. the standard MR qualitative report. 

Close to reality

When conducting research with medical devices, it is advised to simulate the procedure as close to reality as possible. Getting hands-on feedback in the use environment by the intended end-user is vital to gathering insights for commercial teams and performing human factors studies for R&D teams. Doing so is often more time-consuming, more expensive and more logistically challenging. Is your goal a medical device that is safe and effective and which meets the end-users’ needs better than your competitors? If so, this added effort and cost can result in a highly valued medical device. 

References

1 https://www.fda.gov/medical-devices/device-advice-comprehensive-regulatory-assistance/human-factors-and-medical-devices