Your data matters
Editor’s note: Marcus Silversides is the head of data at Researchbods. This is an edited version of an article that originally appeared under the title “6 steps to building confidence in your data.”
The importance of having confidence in your data as a researcher goes without saying. Whether you’re reporting results, breaking bad news or otherwise, you need to be able to swear by your data.
You may be making recommendations for a new product, advising on messaging or comms or sharing a shift in consumer behaviors. Either way, your data will likely have a significant financial impact, so you need to be able to stake your life upon it (though no reasonable company will hold you to that!)
How many times have you given a client or stakeholder feedback knowing it’s not what they want or expect to hear? This could be the CMO’s pet project or a campaign that an entire department has already sunk many hours into. How can we build confidence in our data? How can we ensure our data has integrity?
Explaining data integrity
Data integrity refers to the accuracy, validity and completeness of data throughout its lifecycle. Data might lose integrity and become compromised through replication, alteration or data loss. Anything with the potential to change data after the fact could result in it being compromised. Data might even be compromised from the point of collection, through inappropriate collection methods and techniques or source selection.
How to ensure data integrity
1. Background research
This is a given. You need to know your subject to have any chance of identifying things that look out of place. General reading around a topic and comparison with other existing data sources can help give you an idea of what to expect, as well as highlight deviations and anomalies.
2. Developing partnerships with clients
They will have the background data, sector sales data and more. Don’t bank on it being volunteered, but you can bet your bottom dollar it’ll surface if the research data doesn’t meet their expectations. To this end, making sure you really become an extension of your clients’ internal teams and become intimately familiar with their business and what they do can help give you a ‘sixth sense’ when it comes to understanding their data, especially when it might be compromised.
3. Gather information
Following on from the previous point, don’t always expect useful information to be immediately provided. Press them for what you need. In the same way you might tell a client specifically and unequivocally what information and resources you need when briefing for a creative project, tell them what you need when briefing for data collection projects. See what results they’re expecting and don’t be afraid to question how well-founded it is. Ask for previous results, the context of the results and the methodologies used.
4. Monitor fieldwork
Ongoing monitoring of results and projects helps highlight when things might have gone awry. The earlier you catch any potential hitches in your data collection, the easier it is to address and the sooner you can do something about it. Look at your KPI’s so you can open a discussion up at the earliest opportunity if something isn’t working.
5. Have a solid reason for removing responses
Whilst it’s perfectly acceptable to remove or suppress interviews from fraudulent respondents or bots, there’s a clear line to be drawn between cleansing the data and suppressing unpalatable responses, e.g. unpleasant language may have been used in response to an open-ended question that you think your client will take issue with.
6. Be a confident researcher
You’re the expert! Your clients might know their industry inside out but you know data collection like nobody else does. After all, they came to you for your expertise. This, alongside diligently covering all the other points in this article means you’ve done all you can to ensure data integrity and that can only boost confidence in your data.
Handling fraudulent data
Be aware of what the causes are: Bots, professional panelists and “spammers” in general can compromise the data you collect and are all examples of responses you should consider removing. They might be trying to access more incentives or they might just be bored, but if you can spot these responses, you can omit them. Spotting these types of respondents can also help you remove them from future activities and avoid dishing out your precious incentives to participants who might not deserve them.
Build fraud safeguards into your activities: Build quality gates into your questionnaire and use the technologies at your disposal to combat fraud. It might take a little extra time in field and a few more man hours but ask yourself whether you’re prepared to compromise your data, and possibly even your reputation.
Don’t over-gamify: Don’t kid yourself that “gamification” of a survey is going to gloss over what is ultimately a long and boring survey. It’s been suggested that 10-15 minutes is a sweet spot for an online survey; too much over that and you run the risk of respondents becoming disengaged mid-survey and rushing through the rest of it. It’s a much harder ask to disqualify ever-increasing volumes of interviews from a survey when a pattern is emerging that’s predominantly driven by poor survey design. Treat your audience with respect and they will do the same when responding to your surveys.