Editor’s note: Ed Crowley is CEO of the Photizo Group, a marketing research firm based in Lexington, Ky.

My first job out of college in 1984 was to start up an online marketing research service for agriculture producers (FarmPulse). We recruited over 200 high-income farmers in over 20 states and provided them with Minitel terminals and an online service providing commodity pricing, weather and other market data (produced on a Radio Shack TRS-80 with multiplexers). All the farmer had to do was look on the service once or twice a week and answer an ad copy test questionnaire or some other short, focused questionnaire. In return they received free market and commodity information in almost real time. We could turn around an ad copy test or concept test in 24 hours with a quantitative sample. A similar study using traditional fielding methods could easily take months. This was breakthrough stuff!

At this time, state-of-the-art was considered to be CATI centers or mail surveys. CompuServe and Prodigy were just being launched. AOL wasn’t even online yet. I remember presenting this concept to a senior executive in charge of marketing research at a large agrichemical producer. I was told that I was an idiot. The only way to do marketing research was via CATI systems, focus groups, intercepts or good old pen-and-paper surveys. It was a pretty stinging rebuke for a kid fresh out of college. In all fairness, we were more successful using the service as a door opener than as a revenue producer.

Online changed the research landscape. I don’t think any researcher today would argue that online research is invalid. It made panels of all sizes more manageable. It increased the speed at which research is conducted. It really changed the process for conducting a research project from recruiting sample to fielding surveys to a new digital paradigm. And I believe the next paradigm is about to sweep away the industry as we know it.

Moving away from consumer reporting

Today, most research studies are based on the paradigm of asking consumers or businesspeople what they have done or will do. It’s asking them to recall, accurately, the behavior they exhibited, the results they achieved. And like it or not, it’s only as good as a person’s recall or their ability to accurately predict what they will do. I recently conducted a project for a client that involved surveying users of printers in large office settings. We had software which monitored the printers and told us exactly how much and what each person was printing. We also asked the individual, as part of a one-on-one interview, how much they were printing. Interestingly, the closest users could get was about 50 percent of their actual print volume. They guessed wrong by either underestimating by 50 percent or about doubling their actual volume. We are notoriously unreliable at accurately reporting on our own behaviors. Today’s paradigm says that’s OK – by using quantitative samples and analyzing with advanced statistics we can fix the data to make sure it’s good. Well – OK but only as good as the users are at reporting (which can be argued is not very good).

But things are changing. An immense amount of data is available from business devices (printers, computers, trucks, etc.) and consumer devices (phones, watches, smart homes, etc.) which are digitally enabled and connected. In fact, there are currently over 20 billion of these connected things (according to IDC) being used today, with the number passing 30 billion by 2020. Collectively, this is called the Internet of Things (IoT), generating over 403 zettabyes (one zettabye equals one trillion gigabytes) annually.

And here is the interesting thing: These devices are reporting on not what you think or say you are going to do but on what you actually do. And they are doing it in real time with a high degree of accuracy. So why would a brand or product manager need marketing research to understand how consumers are using their product which relies upon them accurately reporting their activities and intentions? Why wouldn’t they just monitor the data and see how consumers are using it?

Of course, privacy advocates are going to scream that this is an invasion of personal privacy. But the reality is, we sign away our rights to our data all the time. For instance, if you own an electric car, you may have signed away your rights to the data as part of the purchasing process since manufacturers can require this monitoring data to continually improve upon battery performance and other car performance characteristics.

The future of data 

I envision a future where your refrigerator tracks how much ice cream you ate for a midnight snack. It knows it’s you because of your digital watch, which communicated your unique ID to the Internet-connected refrigerator. And then, the refrigerator sends the time and amount of ice cream you ate to your electronic diet record. (Anyone use MyFitnessPal?) From there it is incorporated into your medical record so the next time you visit your doctor’s office he notices your recent weight increase and correlates this with the increase in midnight snacks. Because you signed up for the healthy option on your health care plan, your insurer has rights to your medical and diet record. The insurance company’s artificial intelligence system guides the doctor to issue a stern warning and remind you that if your weight increases by more than X percent you will be violating the terms of your healthy-option plan and your insurance premium will increase by 50 percent. Sound incredible? Every piece of this technology exists today.

Soon, companies will be able to target product marketing toward specific groups based on their actual usage profiles and IoT data. Longitudinal studies shift from studies of reported behavior to studies of actual behavior. So just like a paper-based mail survey is a hard sell today, in the near future selling that respondent-based study may seem just as antiquated.