Respondents can’t answer questions they have never been asked.
Editor’s note: Vera Macedo is a B2B market researcher at NewtonX, Portugal, with five years of experience in market research. Macedo holds a bachelor’s in psychology and a post-graduate degree in marketing management. Find Macedo on LinkedIn. Ben Elliott is a senior research strategist with 10 years of experience in research. Elliott holds an MBA and a dual bachelor’s degrees in psychology and sociology. Find Elliott on LinkedIn.
There is a flaw baked into how most B2B surveys are built, and it sits right at the beginning of the process, before a single respondent has been recruited. The researcher builds a list. Drivers, attributes, barriers, perceptions. Whatever the study is measuring, the researcher decides in advance what the options are. They draw on past studies, stakeholder input, category knowledge and educated guesswork. Then they hand that list to respondents and ask them to engage with it.
What respondents can’t do, structurally, by design, is tell you about anything that wasn’t on it. The study's ceiling is fixed before fieldwork starts.
“At the beginning ... before you reveal the topic or brand or prototype, the waters are still pure. That's an invaluable chunk of time to get feedback or emerging trends that's unbiased, more real. Use that chunk of time – if you don't have a question, come up with one." - Todd Horvitz, Head of Insights, Business Computing and Software, HP
This is frame bias. And while it exists across all survey research, it is sharpest in B2B, especially with genuine domain experts.
The expert respondent problem
Researchers cannot always anticipate the full universe of what an expert respondent might consider relevant.
Consider research into a niche B2B purchase decision. You have recruited, at significant cost and effort, a chief information security officer evaluating zero-trust architecture vendors. Then you share a list built by a generalist researcher.
The knowledge gap between what a domain expert knows and what a researcher can presuppose is often enormous. And it’s not because researchers are not rigorous but because being an expert in research methodology is not the same as being an expert in cybersecurity procurement or enterprise finance systems. The researcher can design an excellent study but miss something critical simply because the field isn't their own.
Why the usual fix doesn’t work
The most common response to this concern is the open-ended question at the bottom of the survey: “Please share anything else you wish to expand on.” This is better than nothing, but it is not a solution.
By the time a respondent reaches an open-ended field at the end of a structured survey, they have spent several minutes navigating the closed-ended options the researcher designed. Those options have become anchors. The vocabulary of the instrument has primed how the respondent thinks about the topic. The open-end at the bottom gives the researcher the feeling of having left a door open. In practice, respondents rarely walk through it.
“The risk isn’t wrong answers, it’s getting precise answers to the wrong question. The frame you set at the start is the truth you end up with.” - Melanie Courtright, Chief Strategy Officer, Sago
The real solution is well understood: mixed methods. Run a qualitative discovery phase with interviews, open exploration and genuine listening before building the structured instrument. Let respondents teach you the vocabulary before you ask them to rank it. This is the gold standard.
Mixed methods have been a core part of how we approach complex B2B research. When the timeline and budget allow for it, it is the approach we recommend. But the honest reality of the industry is that it is not always possible. Discovery phases add time and cost, and they force a client conversation that starts with, “We won’t have structured data for another few weeks.” This is a difficult conversation when timelines are already tight.
The question we have been working on is whether there is a way to recover some of what mixed methods provide within a quantitative instrument, at a pace clients need to stay current.
What we do today: An adaptive approach to research studies
An adaptive approach is entirely possible within a quantitative study. The key is treating the process with honesty, not as a single seamless dataset, but as a structured fielding design where different attributes carry different base sizes, and where reporting is calibrated to reflect that. Done transparently, it isn't a limitation to apologize for.
The process runs as follows.
Establish audience quotas proportionally from the first complete, ensuring the correct balance of individual opinions across the full sample from the outset.
Open-ended first, for up to 50% of the sample
For the questions most vulnerable to frame bias – typically attribute importance, decision drivers or perception batteries – the first portion of the sample receives an open-ended version rather than a structured list. The rest of the survey runs as normal. This phase is the discovery engine. Its job is not to produce quantitative ratings but to produce a respondent-generated attribute set that is honest about what the category actually contains.
Responses are coded continuously as fieldwork progresses by identifying recurring themes, clustering similar language and translating discovered concepts into clean, parallel answer options. AI meaningfully accelerates this step, surfacing patterns across responses faster than manual review alone. A researcher reviews and refines the output before anything enters the instrument. As new themes emerge at meaningful frequency, they are added to the evolving attribute set. When open-ended responses stop producing new themes, the attribute set has stabilized and the discovery phase closes.
Close the open-ended and move to pure quantification
Once the attribute set has stabilized — typically around the 50% mark — the open-ended question is retired. The remaining sample completes a fully closed-ended instrument built from what respondents themselves surfaced. Every respondent in this phase sees the complete final attribute set and produces fully comparable ratings data.
Treating attributes as their own sub-groups
Because the attribute set evolved during fielding, not every attribute was present for every respondent. Attributes present from the beginning of the quantitative phase carry the largest base sizes. Attributes that emerged and were added mid-field carry smaller ones. Rather than treating this as a flaw to design around, the approach embraces it explicitly: each attribute is analyzed against its own base, with confidence intervals reported accordingly.
This is not unlike rotated battery designs or version testing, where different respondents engage with different stimuli and results are reported at the sub-group level. The attribute-level finding is valid within its own base. What it requires is transparency in how results are presented.
Two tiers of output
This naturally produces two reporting tiers, each honest about what it can and cannot claim.
The first is a full-sample ranking limited to attributes present for the entirety of the quantitative phase. These carry the maximum base size, the tightest confidence intervals and fully comparable ratings across every respondent in the cohort. This is where driver analysis, regression-based importance modeling and relative ranking are appropriate. Every respondent rated the same list, so the comparisons are clean.
The second is an extended attribute report covering attributes that were observed but didn't make it onto the final, short list of choices (either because they were mentioned only a couple of times or because they were just outside of the study's main focus), and therefore carry smaller, explicitly flagged bases. These are reported directionally rather than definitively. They tell the client what else the market surfaced, with appropriate caveats about confidence. In many cases, a directional read on an emerging attribute is exactly what a client needs to decide whether it warrants a dedicated follow-up study.
Presented together, these tiers give clients something neither a pure qualitative nor a pure quantitative study delivers on its own: a structured, scalable measurement built on a framework the market actually recognizes, with a transparent accounting of what each finding is worth.
What this makes possible
The practical implication is that no discovered attribute is discarded. An attribute that emerged late in fielding is not excluded from the report because its base is smaller than ideal, it is reported at the confidence level its base supports. The client sees the full picture of what respondents surfaced, with the statistical weight of each finding clearly attached. Early emerging attributes get definitive analysis; late-emerging ones get directional signals. Together they map the complete territory.
An adaptive approach is totally possible. The survey design uses existing survey tooling to build the attribute set iteratively, rather than imposing it upfront. It isn't automated yet, and it takes researcher effort between stages, but it works, and it produces attribute sets that feel meaningfully more honest than those built from assumptions alone.
Where we want to go: AI-enabled frame building by respondents
AI is now capable of doing most of that work. Large language models can read and cluster qualitative responses at a speed and consistency that no analyst team can match. AI can identify emerging themes, propose closed-ended option language and flag when the attribute set appears to have stabilized, all without a researcher having to open a single spreadsheet.
What this makes possible, for the first time, is a survey that builds its own framework as it runs. Not in discrete cohorts managed by a research team, but continuously, each respondent’s open-ended input feeding into the structured choices presented to the next. The instrument learns in real time. The frame is built by respondents, not assumed by researchers, and the entire process happens at the speed a quantitative study is expected to run.
The human researcher does not disappear from this process. Oversight and quality control both matter. Decisions about when a theme is coherent enough to become a structured option, or whether an AI-generated option introduces its own bias, require judgment that should not be fully automated. But the ratio of researcher effort to research quality shifts dramatically. The manual work that currently makes adaptive design expensive and slow becomes, largely, a review task.
“Sample quality is foundational to any research. This raises the stakes further. When respondent input shapes the instrument in real time, cleaning data on the back end is too late. It will have already built a misleading frame. The output is only as good as the input.” - Karine Pepin, Co-Founder, The Research Heads
Taking frame bias seriously
The fully automated adaptive workflow is something researchers are actively developing, and the quality controls needed to run it with confidence at scale are still being refined. This article is an argument for why the industry should be taking this problem more seriously than it currently does.
When you recruit a domain expert and invest in accessing someone who knows their category in a way no generalist researcher can replicate, the least you can do is ask them questions that are open enough to receive what they actually know. Handing them a researcher-built list is not the best use of that access. It has just, until recently, been the only option.