When it comes to data, it may be easy to assume that size really does matter. There's nary a tech company going that hasn't attempted to jump on the big data bandwagon, with the common consensus being that in the search for truth, the more data points the better.
It's not an argument that convinces Kate Crawford, a principal researcher at Microsoft Research. As she points out in a newly published piece for the Harvard Business Review, massive data sets and predictive analytics do not always reflect objective truths. Sometimes big data is victim of bias.
Crawford points to the example of a study into Hurricane Sandy, which lashed the East Coast of the US in late October 2012.
In it, researchers examined more than 20 million tweets, combining it with data from Foursquare, to uncover a detailed picture of how people responded to the terrifying events. And in short, they stocked up on food before the storm hit, and went out partying after it was over. Or so the data tells us.
The problem is, most of the data came from Manhattan hipsters, where it might be assumed, use of Twitter was higher than average.
“Very few messages originated from more severely affected locations, such as Breezy Point, Coney Island and Rockaway,” notes Crawford.
Other big data studies have illustrated similar problems, such as Google's massive over-estimation of the number of US flu cases this winter. Google Flu Trends predicted 11 percent of the population would contact the illness, while the Centers for Disease Control and Prevention estimated the figure to be just six percent.
But while there may be many that would welcome less breathless excitement about so-called big data, Crawford's thinking doesn't seem the end of tooth-grindingly awful buzz phrases.
“We get a much richer sense of the world when we ask people the why and the how not just the "how many,” she wrote.
“We can move from the focus on merely "big" data towards something more three-dimensional: data with depth.”
Deep data. Whatever next?
Kicking Palantir off of AWS is among their demands, too
Rafaela Vasquez was watching The Voice at the time of the crash, new evidence shows
PUBG price slashed on Steam after selling more than 50 million copies - as daily player numbers plunge
Use the same password for every website? It might be time to change them all