Editor's note: Suresh Subbiah is president, North America at Questback, a Bridgeport, Conn., enterprise feedback management firm. Darren Bosik is senior methodologist with Questback.

The global big data trend is driven by the business need to gather bigger research insights from the various types, and myriad sources, of data flowing into the business. Companies across all industries are betting on big data for their next strategic move, such as where to expand geographically and how to increase customer loyalty. Accelerating the movement is computer technology capable of analyzing terabytes and even petabytes of data. The data pile, no doubt, will only continue to grow.

Big data, it seems, is also big business. The median spending per company on big data projects in 2012 exceeded $25 million in the telecommunications and travel/hospitality industries, according to Tata Consultancy Services. Wikibon reports that total revenue earned by technology companies in 2012 from big data alone exceeded $11.3 billion for everything from computer software to consulting services.

The big data landscape is dotted with dozens of technology providers profiting from the trend. But as spending continues to grow, companies are increasingly challenged to implement their big data initiatives without difficulty. Survey results published in The Wall Street Journal showed that 80 percent of IT professionals involved in big data projects find it hard to secure talent to run the software; 76 percent are having trouble finding the right technology tools; and 73 percent do not understand the computing platforms. Oracle, in fact, did its own survey about big data and found that most companies are unprepared to tackle the big data challenge. While they understand the perceived benefits of big data analytics, 60 percent of executives rated their companies unprepared to leverage the data and cite significant gaps in people, process and tools while 97 percent say they need to make changes to improve the big data process.

All just hype

Executives are frustrated over how to manage big data processes and many companies are beginning to wonder whether the messages from technology providers are all just hype. In fact, Gartner published a Hype Cycle report focusing on big data that shows the industry is just reaching the peak of its hype right now, with vendors flocking to the market, customers getting overly excited and not understanding the technology and the expectations about what big data can do for an organization being overinflated.

Whether or not big data is all hype is really a function of how well you can embrace the challenge. Companies understand they have enormous amounts of data which they need to analyze but they have a bigger need to understand how to manage it. So far companies that have big data at the core of their business have yet to yield any meaningful results. Some companies, such as Research In Motion, Netflix and JCPenney, have seen setbacks even though they have invested heavily in big data. They are employing predictive analytics for customer behavior, data mining of customer purchases and data modeling for media spending, yet they all failed to listen to and focus on what is important to their customers.

These and other examples teach us that executives who make decisions without first listening to customers are doomed to failure. Although they have massive amounts of big data at their disposal and are engaged in predictive analytics and other data-driven initiatives, it boils down to the bare principles of listening to customer feedback for success – feedback through both tactical and strategic types of surveys for example, where you collect customer input about services, new products and brand awareness.

Matters much more today

It comes down to focusing on what we call value data instead of big data. Value data are the bits of customer-centric research information that are critical to the business processes of your company – the core of what is necessary to succeed. Value data matters much more today than big data because we are doing business in a new era: the age of the customer. We have transitioned from several other eras of business, starting with the age of manufacturing at the turn of the 20th century, when for nearly 60 years companies like Ford and Boeing ruled the business world. As consulting firm Forrester says, the age of the customer is driven by a consumer’s experience with a brand and their ability to make or break a brand’s reputation overnight through social media conversations. Value data is critical to creating a deeper understanding of customer behavior and their experiences with your company.

Leveraging value data requires a change in thinking about how data flows throughout the business. Within many companies today, all departments and functions are trying to pull from the pile of big data that sits in legacy systems and data warehouses. They attempt to make sense of the data that exists across multiple communication channels and internal departments, including HR, shipping, marketing and sales. All of this is leading to what we mentioned earlier: frustration and analysis paralysis.

The new value data infrastructure is one where data flows more smoothly throughout the enterprise. Each department winnows down the customer data points that are necessary to the functions of their organization within the larger enterprise. Reducing data waste by 30-40 percent gets you much closer to the metrics that matter, the data that can improve your workflows and the ability to allocate resources for more important business initiatives.

The value data that exists in your company is a function of the different types of market research data at your disposal, whether it be structured data (from customer surveys) or unstructured data (from social media). Technology platforms exist to help you capture, manage, distribute and analyze all of this information and narrow it down into the nuggets that matter – and even ensure that it flows throughout the enterprise in real time.

Scores of metrics

As an example of value data, first think of the multitude of data flowing into and out of the customer service call center. There are scores of metrics the call center can track to manage its business process and understand its success. These metrics include first call resolution, customer hold time, customer complaint volume and time to resolve complaints. If the call center manager were to do big data analytics by looking at the terabytes of information at its disposal, he or she might still be frustrated and time will not have been spent wisely. The value data process, however, focuses on the one or two metrics that matter to the call center, the KPIs that it really needs to succeed. This metric can be tied to its business process and is rooted in customer feedback.

Let us look at first call resolution (FCR), for example. Companies spend thousands of dollars trying to track this metric and get their numbers aligned with the business goals of improving customer service. FCR improvements are often correlated with other metrics, such as repeat call volume and customer satisfaction. Improvements in FCR can save money across the company. But the big work comes with gathering the data, analyzing it effectively and tracking the results.

The call center value data, however, can lie in one metric, which is achieved by narrowing the customer feedback down to a single question – one that has gained traction in recent years – the customer effort score (CES), which asks customers, “How much effort did you personally have to put forth to handle your request?”

CES is a good example of value data because it achieves a lot with very little effort. CES focuses on organizational changes in customer service and makes it easier for customers to have their needs fulfilled. In 2008, the Corporate Executive Board argued that what customers really want is to simply be given a satisfactory solution to their service issue. After conducting structured interviews with customer service leaders and a study of more than 75,000 customers, the Corporate Executive Board found that CES tops the charts with the highest predictive power for customer satisfaction. The research claimed that excessive levels of customer service (such as offering free products or services) will only make customers slightly more loyal to the brand and to achieve customer loyalty organizations must reduce the effort that customers exert to get their problem solved.

CES is an example of value data because it is a KPI that can highly predict business objectives (namely customer loyalty and satisfaction) without the dependence on other excessive amounts of data inputs. It is getting to the value data in a much more simplistic way – and it is eliminating other data points, such as customer satisfaction, repeat call volumes and customer time spent on calls.

It is essential in today’s age of the customer for businesses to manage customer satisfaction. Another example of value data that is driving the customer decisions of many companies is Net Promoter Score (NPS), a customer loyalty metric developed by Fred Reichheld, Bain & Company and Satmetrix. NPS is based on the fundamental perspective that every company’s customers can be divided into three categories: Promoters, Passives and Detractors. By asking one simple question – “How likely is it that you would recommend [your product or service] to a colleague?” – you can track these groups and get a clear measure of your product’s performance through your customers’ eyes.

Critical stage

Big data has reached a critical stage. The market is poised to grow to more than $50 billion by 2017 but more than 55 percent of big data projects fail. With so much opportunity coupled with hype and misinformation, we are in the midst of the big data Wild West. With value data, the opposite is true. To achieve breakthroughs in customer insights, value data can bring bigger benefits. Companies need to invest less in technology for managing big volumes of data and more in technology that allows real-time access to value data.

Once you have established your value data streams and their associated KPIs, it is best to create a living dashboard that can track metrics in real time and easily share the results throughout the enterprise with relevant executives. This technology also creates the means for gathering customer feedback – the feedback associated with value data points such as CES and NPS. Behavioral and attitudinal insights collected from qualitative and quantitative research from panels and online communities are a goldmine for building better customer profiles to grow loyalty. These are benefits you gain by deploying an EFM suite with features for developing surveys for computers or mobile devices, building online communities and panels and analyzing data from social media and questionnaires.

The big data trend won’t go away anytime soon. But by changing the focus to value data, companies can take a better step toward easily managing customer feedback and making faster, more relevant marketing research-based decisions.