Editor's note: Chris Dowsett is principal customer and market analytics officer, Queensland Government, Queensland, Australia. He can be reached at cjdowsett@gmail.com.
The growth of business intelligence, analytics and new research methods has created an environment where business leaders are inundated with an increasing number of data sources. This presents businesses with a lucrative opportunity as well as a functional challenge. Business leaders have unprecedented access to customer and market information but are struggling to avoid data paralysis, overcome data silos and maximize their investments in measurement.
There is a well-documented case for investment in new data sources such as big data, analytics, data science, text analytics and neuroscience alongside more entrenched intelligence-generating programs like market research. Studies by McKinsey have shown the financial benefit to organizations that invest in their research and data insights programs. For example, in 2011 McKinsey estimated that retail services companies could increase their operating margins by 60 percent through exploring data insights.
However, measurement frameworks and processes for integrating these new areas of insight have been missing from the conversation. Organizations may be armed with more data insights but recent research shows business leaders are struggling to integrate that knowledge into business processes. Only recently have industry commentators started to talk about the need to invest in management processes alongside research and analytics growth. Harvard Business Review, for example, recently published an article about the need for businesses to change their management style to better include data insights in decision-making.
Over the last three years, I’ve been working on a doctorate-level research project to understand how business leaders use data. Feedback collection involved speaking with hundreds of business leaders at director level and above. The research shows that leaders need a structured process to incorporate holistic data insights into business projects.
In this article, I introduce the new ICSAR Model for data use that came out of this research. The ICSAR Model is a framework that provides business leaders with a structure to build insights into their decision-making process. It draws on disciplines such as design thinking and business analysis to provide a simple but flexible framework that can be applied to business areas across everything from small projects to large, complex structures. The ICSAR Model is completely free to use under the Creative Commons (NC) License and I hope to keep improving the model based on user feedback as a form of open-source model.
Across several phases
The research into data use by senior business leaders involved collecting feedback across several phases as part an exploratory approach to the topic.
The first phase involved looking at existing literature and news articles to understand the factors that influence the value placed on different sources of market research data. From there I interviewed a number of senior business leaders ranging from directors to C-level executives including CEOs. I asked questions around how they used data, what data they regularly used and why. The questions covered all data sources and provided foundational insight that helped inform a follow-up survey of senior business leaders.
The survey was designed to look holistically at what data was used, how senior business leaders interacted with analytics and insights teams, how business leaders applied different data sources to decision-making and to understand how different types of communication influenced the way data was used.
The final phase involved conducting an in-depth case study with Intuit’s Small Business Group, responsible for the QuickBooks software product. The case study focused specifically on how business leaders at Intuit used social media data as part of their business intelligence work. I wanted to investigate how a newer data source like social media was being integrated into business feedback.
Four types of influence
I identified four types of influence involved when data is used in business activities from the research. These are:
- Organizational demographics and goals
- Project-specific goals and attributes
- Personal research and experience
- Time-based requirements
Organizational demographics and goals
A number of studies including my own have found evidence that the type of organization and the culture has a large influence on how analytics and market research is used. For example, Bednall and Valos (2005) found that Defender organizations were more likely to use confirmatory research to justify existing programs while more entrepreneurial Prospector organizations were more likely to use research to look for emerging trends. Organizational makeup can favor some data sources while limiting others even if another data source would be more useful.
Project-specific goals and attributes
I found that the type of project influenced the type of data used. In one sense this is good because insights are tailored to the project at hand but my research also shows there is a lack of triangulation across data sources and a lack of holistic approaches to program measurement. For example, there was a digital marketing program relying solely on digital analytics while one respondent talked about a customer service enhancement relying only on qualitative feedback. A more holistic approach might include qualitative feedback on the digital content alongside analytics to measure the overall effectiveness of the digital marketing.
Personal research and experience with data
Simply put, a business leader is more likely to give more weight and place higher value on familiar data sources. This means newer data sources or potentially more accurate data sources may be sidelined. For example, business leaders at Intuit struggled to incorporate social media feedback into customer care and product development. Survey respondents also noted challenges with getting business leaders to include newer data sources.
Time-based requirements
This is an area that is well covered in market research journals and will continue to be important as business inches closer to real-time decision-making. Once again, more accurate data sources may not be used when other data sources are available that can be accessed more quickly.
While undertaking the research with the senior business leaders, it was clear that there are barriers that prevent better use of different data sources. However there are discrepancies around the awareness that business leaders have about these. Some explicitly commented on barriers in their organizations while others seemed unaware of any challenges or influences. This may mean that a business leader thinks they are using data insights comprehensively yet won’t be aware of potential biases.
Intuit’s Small Business Group is just one example of an organization that has made significant investments in analytics and research. However their investment in this area isn’t being fully realized because they do not have management structures in place to manage both the growing number of data sources and the influences on senior business leaders.
Many organizations face challenges in this area. There is a growing variety of data sources for business teams to manage and four influences that impact how data is used in business decisions. The research shows there is an opportunity area for organizational development by implementing a process like the ICSAR Model.
ICSAR was created to encourage a holistic approach to using data and minimize the four influences by providing a framework for senior business leaders at organizations like Intuit and others. The model supports senior business leaders by providing a step-by-step framework for incorporating data insights into business activities and circumvent the influences outlined above.
The ICSAR Model is based on five steps. Each step includes inputs and actions – both of which are crucial to encouraging more holistic data use. The name ICSAR relates to each step in the model and these are briefly outlined here.
1. Initial insight
The model begins with an initial insight that starts a new investigation or analysis project. An analyst or researcher, monitoring different data sources and measurement projects, notices a potential change in behavior or a changing trend. For example, a group of customers might complain about a new product feature on social media. The social media monitoring picks up this topic and the analyst or manager begins a project to investigate whether this is an outlier or a bigger problem.
Inputs: Measurement frameworks and associated measurement programs. A measurement framework clearly outlines what data will be collected, key performance indicators and communication of the insights. They ensure businesses notice changes in their business or customer environment. Some organizations may not have these and will need to set them up.
Actions: Gathering the initial insight from the data.
2. Compare with inputs from all data categories to validate
Once the initial insight has been brought to light, the next step in the model is to compare that insight with inputs from each of the data categories. The data categories outlined as part of the model process are: unprompted feedback (e.g., social media comments), behavioral data (e.g., digital analytics) and prompted feedback (e.g., surveys or focus groups). (These categories and more examples can be found in the data categories table accompanying the ICSAR Model.)
The goal of this step is to minimize biases (project, personal or organizational) by encouraging business leaders to triangulate the initial insight with different categories of data inputs. Using at least one data source from each category in the table ensures that leaders take a more holistic view of the issue.
Inputs: The data categories table and data inputs from each of the three data categories.
Actions: Triangulate the initial insight with at least one data source from each of the three data categories.
3. Synthesize findings with existing organizational learning
Once the initial insight has been triangulated with data from each of the three data categories, the next step is to compare the findings with any existing organizational learning or knowledge base. The goal here is twofold: firstly to encourage organizations to properly document the findings from analysis projects; and secondly to ensure organizations continue to expand their knowledge to capture the full value of their research and analytics programs. Many organizations I researched had very poor management of data insights and didn’t actively leverage existing knowledge to inform future research. This led to a lot of redundant projects like repetitive A/B tests.
Inputs: Existing organizational learning hubs (if available), internal experts and/or insights library. Some businesses will need to improve this area and work to implement better central systems of documenting business-wide learning.
Actions: Compare the findings with any existing knowledge to leverage existing knowledge and enhance the value of investments in research and analytics.
4. Apply to business area or project
The next step is to apply the new knowledge and insights to the business area or to a project. This can be done as a complete change to the process or as a test-and-learn scenario such as an A/B test. The key here is that the insight moves to being put into action. As the organization prepares to make a change, it’s worth being clear about what metrics or methods will be used to evaluate if the change has improved the process.
Inputs: Feedback processes for activity improvements are the main input – that is, the process an organization uses for improving areas of the business. For example, a lot of organizations use scrums to build out product features with the engineering team. Others might use a cross-functional task force under a project manager.
Actions: Apply the insights to a project or business area and ensure it is measured appropriately to determine success of the change.
5. Review outcomes and list new knowledge
The final step in the ICSAR Model is to review and collate any feedback from the change or improvement. That means asking questions such as: What happened? How did the project perform in terms of its key indicators? Were there other insights gained? This reflection should result in new organizational knowledge, regardless of whether the change was successful.
Inputs: Measurement frameworks will be important here to clarify how the project will be evaluated, metrics to be collected, insights from key performance indicators and how the team will communicate findings to the wider organization.
Actions: Review the data and research to evaluate the project. List any new knowledge and communicate findings to the wider organization.
A cyclical process
The ICSAR Model is a cyclical process. This means that once Step 5 is complete, the process starts all over again with the next insight. The cyclical nature of the model means that an organization is continually improving, continually gaining new insights and capitalizing on investments made in the areas of analytics and research insights.
Want to read more about the ICSAR model and get more in-depth material? Go to www.designingdata.co where you can get downloads and sign up for e-mail updates about the model. You’ll also find a range of free resources including an introduction guide to the model, a guide to creating a measurement framework and the reference guide for the data category table. All of the materials are also free to use under Creative Commons (NC) license.
REFERENCES
McKinsey Global Institute (2011). “Big data: the next frontier for innovation, competition and productivity.”
Bednall, D., and Valos, M. (2005). “Market research effectiveness: the effects of organizational structure, resource allocation and strategic type.” Australasian Journal of Market and Social Research, 13(2), 11 – 27.