So what’s the problem?

What’s to say respondents haven’t slightly or even significantly misinterpreted questions and answered accordingly? How do we know they haven’t struggled with recall and taken a wild guess? Maybe they felt low in confidence throughout the survey, or simply disliked elements of the survey experience.

How is this relevant to medical device market research?

In the world of medical device market research, respondents’ reactions to specific elements of a device, reflections on day-to-day therapy decisions or comments on the intricacies of marketing materials are explored in-depth. The way that people engage with and interpret subject nuances are therefore critical, as they have a significant impact on how people respond during surveys.

What’s the solution?

Cognitive interviews are a tried and tested tool that help researchers evaluate and refine survey questions before final launch, to ensure the survey explores and measures the constructs in the way that the researcher intends.

How do Cognitive Interviews work?

In a nutshell, cognitive interviews expose and explore 3 key elements concerning how a respondent engages with each survey question, namely;

1. How the question is interpreted
2. The circumstances and thought process used to interpret and answer the question
3. Difficulties encountered when interpreting and answering the question

Face-to-face interviews or web-assisted telephone interviews are utilised to take potential research participants through the survey in real-time. Different cognitive interviewing techniques can be used, such as ‘thinking aloud’ techniques which encourage participants to describe their understanding of the question and their thought process with regard to their answers. Researchers can also use in-depth qualitative probes to understand how the participant engages with and responds to each question.

Despite the pros and cons associated with each approach, the end result remains the same; clarity regarding problematic questions and clear insight regarding how questions, answer options and the survey’s flow and timing, can be changed to elicit more accurate, considered responses.

Why are cognitive interviews not always used to pre-test surveys?

Despite their obvious benefits, researchers don’t always factor cognitive interviews into their survey design. But why?? For some researchers and clients, awareness remains a barrier, having not previously heard of, or used this powerful qualitative method. For others, cost is an issue. For cognitive interviews to offer value, a minimum sample of 10 participants is required (and max. 25). Depending on the survey length, cognitive interviews can last anywhere from 30 to 60 minutes; all of which result in an increase to the total project cost.

While cost effectiveness remains a high priority for both research agencies and clients alike, any trade-off between price and quality should be considered with ‘eyes wide open’ to fully understand the business advantages that might be risked. If the survey in question requires a significant sample size or involves a complex methodological approach, cognitive interviews represent money well spent.

Written by Alper Hulusi