Both in search of the same thing
Editor's note: Linda H. Hwang is a former senior user experience analyst at Usability Sciences, an Irving, Texas, firm.
Market research professionals who have embraced user experience (UX) research glean value and insight that conventional market research methods cannot deliver. All too often market research professionals, based on commonly-held fallacies, dismiss user experience research in favor of more traditional marketing research.
Here are seven of the most common fallacious beliefs, along with arguments by market research veterans on how marketing and UX research can and should coexist and even complement one another.
Fallacy #1: We can get away without user feedback
Who needs to talk to users, anyway? We know they’ll love our product, it’s so fantastic. After all, we love it, and we’re the experts, right?
Not so fast: Plenty of products have failed because consumers could not figure them out. A very memorable example is the 2013 disastrous launch of the healthcare.gov site. Although usability issues made up only a fraction of the site’s highly publicized problems, it was obvious to everyone that gathering user feedback had not been a part of its development process.
Ignoring users’ needs is an understandable, but avoidable, mistake. “Web interaction design teams kind of live in a bubble; sometimes they forget they’re designing for real people,” says Kerry O’Hara, Nationwide’s director of research and marketing strategy. Through a process of eliciting users’ raw commentary and their natural behavior, usability testing can pierce that bubble and keep the focus on the customers.
Additionally, usability testing is best performed early in the design and development stages. Several studies estimate that the cost of fixing defects discovered late in the software development process was exponentially higher than if they were resolved at an earlier stage.
Avoid replicating healthcare.gov’s epic (and costly) fail by getting user feedback on your products early and often to avoid being trapped in a bubble.
Fallacy #2: UX research is very narrow in focus
Many people (and not just those in the marketing field) perceive UX research as consisting only of usability testing. Although usability testing does constitute a large portion of UX research, there’s more to this research than product-based tasks.
For example, “omnichannel” testing literally takes testing outside of the lab to capture the holistic consumer experience in a variety of digital and physical environments. These types of field studies inevitably encounter real-life factors that can’t be anticipated or replicated in a usability lab and yet reveal important findings and insights.
It’s vital for all researchers to realize that the scope of UX research is growing and there is value in “thinking outside the lab.” Marketing researchers, including Teri Hughes at Intel, are already doing this: “Sometimes, somebody will bring up usability and say, ‘I don’t know if I need it’ but they may be thinking about it in a very narrow sense; I think the whole space of usability/user experience is much broader than it ever used to be, because of how technology has evolved. It touches everything we do today,” Hughes says.
Fallacy #3: UX research is expensive
Compared to quantitative marketing studies with samples of hundreds of participants, the cost of usability testing is generally more affordable. Since usability tests typically require a much smaller number of participants (for more on the fallacy about the need for a large sample size, see #4 below), the outlay for expenses for recruiting, user compensation and researchers’ time can be a fraction of what a marketing study costs.
Ed Podraza, business insights at Allergan, points out that “usability research is very affordable relative to other research methodologies or compared to the overall investment in your campaign.”
It’s worth the time to get rate quotes from UX professionals to see how affordable testing is, before you assume UX is too pricey for your budget.
Fallacy #4: A large sample size is necessary for meaningful data
To achieve statistical significance in the data, market research studies typically require sample sizes of hundreds of consumers to flush out the outliers and ensure the results closely align to the total population.
Usability studies differ from quantitative studies in their focus: They diagnose problems. Quantitative studies generate measurements, so a larger sample size increases the accuracy of the measured quantities. Since usability tests produce qualitative insights, it is possible to discover a large fraction (85 percent is the frequently-cited percentage) of a product’s severe problems with a smaller number of users, usually in the range of five to eight. Beyond five to eight users, the likelihood of discovering additional severe problems decreases and the money that would be spent on a larger sample size can be better utilized on a usability test of the next (corrected) version of the product.
The smaller sample sizes for usability testing can be difficult for market research practitioners to accept. “I was one of the non-believers,” says Alicia Schmid, Michelin North America’s digital consumer experience manager. “I knew we didn’t need hundreds of participants but when I was told eight to 12 users, I was like, ‘What?!!’ Now when I am pitching usability testing internally, I tell them we will run eight participants; after six participants you will have all you need and then we can tackle lower-priority areas.”
Fallacy #5: Focus groups are the best way to gather user qualitative feedback
Focus groups are a popular method for collecting feedback in a quicker time frame than usability testing (a two-hour focus group session can include up to 10 participants, versus individually testing all 10 for an hour each over several days). But there are differences in the type of feedback gathered from these two methods that need to be considered. In a marketing focus group, users are typically asked for their responses to advertising messages or campaigns or new product development. In contrast, usability testing focuses on whether or not consumers can easily use a product.
“If I am asked to research a new advertising message, I’ll often run two to three focus groups per segment to diagnose it,” says Gary Martin, vice president of Travis Research Associates. “But a Web site is different. One of the best ways I have seen to research the efficacy of a Web site is to run in-depth interviews utilizing the usability testing methodology.”
The intimacy of watching users interacting with the product can reveal areas that cause difficulties or confusion (as well as effective functionality and design) and can also inspire pragmatic solutions from the entire project team.
The other fundamental difference between focus groups and usability tests is the depth and quantity of the participants’ feedback. In focus groups, the Type As may dominate the discussion and the Type Bs and Cs may not contribute much, if at all.
In usability testing, however, participants cannot hide in a group; each is tested in a one-on-one session. Participants are asked to perform tasks on a Web site or product and are observed as to whether they succeed. Instead of participants’ opinions, usability testing seeks to capture exactly how they use the product targeted to them.
Fallacy #6: Anybody can moderate a usability test
It is a common belief that moderating a usability test does not require preparation or specialized skills. All the moderator has to do is just ask some scripted questions or one can get a focus group moderator to pinchhit for a usability test.
Not true, says Michelin’s Schmid: “The usability moderator is very different from the focus group moderator. Usability moderators have to have a technical understanding of Web sites that focus group moderators don’t have to have, in order to make good recommendations. A usability practitioner’s recommendations are based on best practices, not just on the feedback heard from users. Usability testing is an expertise.”
Acquiring the skills necessary to moderate a usability test and recommend effective solutions requires many hours in the lab. You can conduct usability tests with untrained moderators but you’ll get more reliable data and avoid numerous pitfalls using seasoned professionals.
Fallacy #7: A company’s Web site is solely a marketing vehicle
It is easy to fall into the trap of thinking a Web site is a tactic to disseminate marketing messaging online and leave it at that. Web sites are often developed within the same paradigm as print collateral. Many times, Web sites are online versions of their current print campaigns.
Consumers view Web sites differently, as Allergan’s Podraza learned. “Usability testing made me realize the Web site is a conduit to communicate with our customers; if the Web site experience itself is suboptimal, it’s diminished the value of the investment we make in driving customers to it. Web sites are two-way interactions. Consumers visit your site for a reason. It is all about getting the right message to the consumer at the right time and in the right manner.”
To consumers, a company’s Web site is an extension of itself, a touchpoint; a subpar experience on the Web site is equivalent to poor customer service. It’s been shown through numerous site analytics that when users bounce off of a site because of a frustrating experience, they won’t go back.
On the other hand, a good or superlative user experience conveys positive messages to consumers: It engages – rather than repels – them. Although it may not initially seem like the case to marketers, a product’s user experience is a significant element of the brand’s voice and good UX can coax visitors to linger and ultimately convert (and even become repeat customers).
Customers expect Web sites to be a portal to the companies behind them. If the site lacks access to customer support or a support chat feature, visitors interpret this omission as a clear signal of the company’s lack of receptivity to communicating with its customers. Keep your lines of communication with customers open and visible; they will appreciate it.
Win-win
The experiences of the marketing experts cited above attest that marketing and UX research do not have to be considered an either-or choice. By first understanding how UX can work in tandem with marketing and then applying what you learn from a UX study to improve the user experience, you will gain on both fronts. And that’s a win–win.