How “Big Data” is changing the world

I was in Home Depot yesterday on an uncharacteristically rainy day and at the checkout, there was a rack of umbrellas there all set for spontaneous buyers. I didn’t even know that HD sold umbrellas but my first reaction was to think that some enterprising employee had the foresight to get that rack out there for the public at the right moment for a sale. But is that really what happened there?


Have you ever noticed that when there is a snowstorm in your area, that Wal-Mart always seems to have plenty of snow shovels on hand to meet the sudden demand? Is this the result of exceedingly sharp inventory management….or a big local warehouse full of snow shovels?


In both cases, the answer is probably no, and it is certainly no in the latter case. These fortuitous actions are the result of “Big Data”. Wal-Mart has a computer system that takes in weather data from all over the world, and along with known demand for snow shovels in specific stores when it snows and known inventory levels, an “algorithm” or a mathematical formula, takes into account the weather forecast for your area and if a snowstorm is coming, it initiates an appropriate shipment of snow shovels to your local store before the snow hits. Home Depot probably has a similar alert system.


The latter paragraph illustrates quite well the essence of “Big Data” but it has implications far greater and better for mankind than just making available enough snowshovels. Everything people do such as making credit card transactions or doing a Google search is recorded and aggregated into huge data pools that produce information on human behaviour. Google, for instance, has come up with a way to predict future outbreaks of the flu in any area of the world. It’s pretty simple in principle, they analyze the number of Google searches about “the flu” and if it spikes in an area, then they know that the health system is going to get hit with a lot of need for flu treatments.


Another example is a “Big Data” expert with no particular legal knowledge challenged some top lawyers to predict the outcome of the upcoming Supreme Court cases. Well, the mathematician won hands down because he was able to study previous patterns and design a predictive model that was eerily accurate…..and he didn’t know much about the law!


When you sit back and think about it, the possibilities are endless because there is more and more data available, and the analysis of the data is getting more sophisticated all the time. The “algorithms” are getting more and more precise and better able to predict human needs….even disasters.


The downside of this would be in poor modeling, or mathematicians making analyses and algorithms based on false premises and then make a bad prediction and cause a lot of needless fear and panic. Still, it looks like “Big Data” is here to stay and will change the way we live over the next few decades.

Let’s hope though that “Big Data” doesn’t become too much like “Big Brother”!


For more in depth and intelligent information on this, check out this article from Harvard University.





Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s