Tapping into the usability dimension

Editor’s note: Karen Seidler-Patterson and Michael J. Patterson are principals of Usable Solutions, LLC, a firm with offices in New York City (212-588-0297) and Colorado (719-487-0563).

As the dust settles from the stampede to get on the Web, companies are taking a more reasoned approach to their online product development, with well-placed attention on the quality of the customer experience. As numerous companies have already painfully learned firsthand, if you neglect the customer experience, your customers will take their clicks elsewhere.

Creating a positive customer experience is critical to any online product success, whether we’re talking about business-to-business, business-to-consumer, or internal products. What defines the customer experience? Certainly, a product filled with bugs will make your customers unhappy. However, over and above basic technical performance issues, it is the usefulness and usability of the product that ultimately defines the customer experience.

Usefulness refers to whether the product has the information, features, and functions that the user needs. It answers the customer’s question: “Would I want to use this site?” Usability, on the other hand, has to do with whether or not the site communicates its content and functionality to the user and how well a user can effectively access and use the site’s information and features to perform a task. Usability corresponds to the customer’s question “Can I use it?” Each dimension is critical to the customer experience and if one is missing, a company may fail to achieve its Web business objectives and experience erosion of its customer base.

Traditional market research and observational techniques, such as focus groups and contextual inquiry, can be used to discover what would make a product useful. There is also no shortage of companies offering an array of techniques to evaluate Web site usability. In our experience, there is simply no substitute for directly observing customers trying to use your product through usability testing.

In a usability test, representative users are observed performing realistic tasks with a Web prototype or online product. Usability testing exposes inconsistencies between how the users expect to interact with the product and how they actually must interact with it. It also exposes the obstacles preventing users from successfully completing transactions, accessing product content, or performing other desired tasks.

Usability testing is about what people actually do, not what they say they do. This differentiates it from focus group studies, which are often inappropriately used to elicit usability information, or from online research methodologies that rely on self-reporting. Observing the usability test allows the product team to make a connection with their customers and see their users in action with their designs. The impact of directly witnessing usability tests often transforms initially reluctant product team observers into ardent usability advocates.

Usability testing is well-suited to informing online product design. It is easily adapted to any stage in the Web development process, allowing the product team to evaluate the usability of early, paper prototype concepts, low- or high-fidelity prototypes, or live products and competitor sites. The ability to assess designs throughout development can prevent product teams from wasting critical resources on the wrong design paths and help guide design choices at critical product development junctures.

The nuts and bolts of usability testing

Usability studies do not need to be expensive or elaborate to provide high-quality insight. However, they do need to be well-planned and executed. All effective usability tests start with a test plan identifying goals and the measurements that will be used (e.g., instances of user frustration, number of successful task completions). Quantitative metrics are sometimes of interest, for example when benchmarking is the goal or where shaving time off performing tasks is critical to save money or lives. However, quantitative studies usually require larger numbers of participants and may sacrifice rich verbal insight into relevant issues.

In qualitative usability studies, participants are asked to think aloud while performing realistic task scenarios. Thinking aloud helps to expose user expectations, problems, and frustrations with the product. The participants’ actions are also observed (e.g., which navigation route is being taken to do a task, what do participants do when they can’t find needed information). It is critical to carefully observe the participants to ensure that what they are saying syncs up with their actual behavior. We often see a discrepancy in which participants say one thing but do another.

These incongruities are particularly valuable in a usability test because they point out things that are unclear or confusing to the user. In a recent usability test of an online brokerage application, for example, a test participant exclaimed “Wow, that was easy!” when, contrary to his own intentions, he unwittingly sold a stock instead of buying one. This disparity between attitudes and actions is a strong argument in favor of usability testing over usability evaluation methodologies that are wholly grounded in self-reporting.

Task scenarios are selected to represent the typical or critical tasks for which customers will be using the product. These tasks are often identified during earlier phases of research. Task scenarios often include both directed tasks, in which the participant is asked to perform a specific task, and self-directed tasks, which target user-chosen goals or browsing behavior.

A few participants go a long way…

You don’t need many participants to uncover the major usability flaws, although the participants do need to be representative of your target user group(s). For a qualitative usability study, most experienced usability specialists know that using about five participants per target group is sufficient to provide meaningful data. With more participants, you often see diminishing returns on findings, in addition to glazed eyes of observers in the backroom. You need to ensure that the major user groups are represented, even those whom you suspect will differ in the ways they will use your product. For instance, when we tested an online benefits management tool that would be used by both general employees of a company and the company’s HR benefits administrators, we recruited participants from both populations.

Testing in the lab and beyond

There are a number of choices when it comes to where to conduct testing. Most commonly, usability studies are conducted in a focus group-type facility (either with or without a usability lab setup). The two-way mirror setup at such facilities allows the product team to observe test sessions firsthand and in real time. A usability lab setup provides the additional capability of capturing a picture-in-picture view of the product (e.g., a Web page) simultaneous with a view of the participant, allowing observers to see and permanently record the details of the participant and what the participant is clicking on. While a nicety, a usability lab does add a layer of cost to the study and its absence will not compromise the quality of the test sessions.

A cost-effective alternative to running the usability study in a focus group facility is to conduct the test sessions in an office or meeting room. This approach is appealing when the number of observers is very small or when facility cost or availability is a serious issue. The downside is that you cannot accommodate many observers in real-time, although videotaping the test sessions (using either a simple video-camera setup or portable usability lab) can extend observer capabilities.

Sometimes it doesn’t make sense to divorce the user from the environment in which the product will be used and the critical social or environmental conditions cannot easily be replicated in a lab. In such cases, usability tests can be conducted in-situ, i.e., in the workplace or other environment in which the product will be used. While in-situ studies are somewhat more complicated to conduct and require giving up some control over aspects of the test sessions, the realistic backdrop can provide very rich usability data.

Involve the team in observation, analysis, and solutions

We find that involving the product team in some manner in the usability testing makes it more likely that usability issues will be addressed in the redesign. Team members can observe test sessions, take notes, and help collate findings by listing observed issues from their notes and describing them with examples and/or performance measures from across participant sessions.

When identifying issues, it is critical to not just consider observations individually, but to see if there are themes or patterns that potentially point to a larger issue (e.g., is there a problem with the tab label or is the problem that the overall content has not been organized intuitively?). During analysis, it is also important to identify the usability strengths of the product, so that redesign solutions can further capitalize on them.

Once the team has identified the list of usability issues, prioritizing them is helpful. We often employ a simple scale that takes into account both the severity of the problem (i.e., the extent to which the issue will affect task performance) and its expected frequency. For example, a major problem would be one that prevents users from completing a task or causes significant delay or frustration. This would be a high priority fix.

With the list of usability issues and priorities in hand, the team can brainstorm solutions. The prototype or product is then redesigned or refined, based upon the insights.

Test early and iterate!

Undoubtedly, the most common remark we hear from the backroom during usability test sessions is “Why didn’t we test earlier?” Typically, companies wait until the product is virtually ready to launch to conduct their usability testing. While certainly any usability testing is better than none, by the time you have a highly functional prototype, it is often too late to address major usability flaws for the current effort, no matter what type of evaluation technique you are using. For instance, six months ago we were asked by a large insurance company to evaluate the usability of a highly functional prototype just weeks before its launch. Our testing identified some major flaws with the overall organization of the product content and the navigation. But the product’s information architecture was so far along that only button and link labels could be tweaked. The product launched and several months later, the company finds that it must now revisit the usability issues and the product’s information architecture (e.g., navigation, organization) because of such poor customer response.

Building usability into a product is fundamentally a process, rather than a single step. Usable products are most efficiently created through an iterative approach that incorporates the results of the usability evaluation into redesign or refinement solutions, and then tests the product again. We usually advocate quick iterative design-test cycles, to keep pace with the aggressive schedules typical of most online product development organizations.

Ultimately, the earlier in development you implement usability testing, the easier it will be for the product team to absorb and respond to the findings to create highly usable products which truly elevate the customer experience.