Editor's note: Mary Mathes is the market intel director for Thriveplan, a Cincinnati-based insights and strategy firm. 

When we at Thriveplan were first thinking about how to design a big study drawing on passive data, a story came to mind that I had heard back when I lived in Washington, D.C. It’s a Cold War-era tale, maybe just an urban legend that tour guides tell visiting busloads of high school students every summer, but I still think it holds an important lesson for us as researchers as we approach the growing trend of using passively collected data in our work and what it can and can’t do for us. A cautionary tale, if you will. 

There was a small building in the center courtyard of the Pentagon back in the day. (There’s still a building there but this particular one no longer stands.) As the story goes, the Soviet Union had missiles – nuclear ones, no less – constantly trained on this building. They were convinced it was the entrance to an underground bunker with a top-secret meeting room, with the whole Pentagon essentially acting as a fortress around it. 

Why did they think this? Well, the Soviets had their own version of passively collected data: satellite imagery. Every day, their satellites would pass over and take pictures of the site and every day they would observe a pattern of behavior in those images: there were always military officers going in and out of this building. From this data, they concluded that clearly the building must be important. After all, it was smack in the center of the Pentagon. This information didn’t come from a human spy with an imperfect memory reporting what he or she recalled seeing, this was objectively what was happening. 

Here’s the thing: that building? It was a hot dog stand. There were always people milling around it because when the satellites passed over at the same time every day, it was most likely lunchtime. 

I tell you this story be...