There is a new fallacy in business: Big data makes you make smarter decisions. Measuring and monitoring is one of the absolutes of leadership, and data is obviously critical in decision-making. Ann Winblad, the senior partner at Hummer-Winblad, has been cited as saying: "Data is the new oil." But like oil, it is both friend and enemy.
On the positive side, the proliferation of data provides insights into consumer behavior that we have never had before. It can be a goldmine to economic stimulation and profits. Plus, the right data at the right time can be a lifesaver—literally (more on that later).
As with oil, the misuse of data can backfire. The more you have, the more dangerous it can be. In fact, lots of data and information can actually make you dumber.
Let’s start with a simple fact. More than 1.8 zettabytes of information will be created and stored in 2011, according to the latest IDC Digital Universe Study. That’s enough information to fill 57.5 billion iPads. The Digital Universe Study says that over the next decade, the number of servers (virtual and physical) worldwide will grow by a factor of 10, the amount of information managed by enterprise data centers will grow by a factor of 50, the number of files the data center will have to deal with will grow by a factor of 75-plus—but the number of IT professionals in the world will grow by less than a factor of 1.5.
This means that we are putting a lot of data in the hands of very few. What if the data is misused, misinterpreted, or misunderstood?
The proper use of data at the right place and right time, for example, can lead up to driverless cars. The reliance upon data at the right time is critical, as some cars will have a safety device to "beam data like position, velocity, and acceleration to and from neighboring vehicles and infrastructure 10 times every second."
We have always had data. The capital markets depend upon data for accurate trading and timely information. Referencing the title of this post, Warren Buffet gets data—in fact, the same data is available to him and his team as is available to other investors. The difference is what his organization does with it.
Data does not help us make better decisions. It helps us make more informed decisions.
And companies and political campaigns are using data to strategically reach and influence us. Much of the credit for Obama’s victory was given to data science.
I noted that data could backfire. It backfires when it is in the hands of the wrong people. It is why we have the Health Insurance Portability and Accountability Act (HIPAA). Personal healthcare information is too sensitive and private to be openly available to the world.
In the case of protected data in healthcare and personal computers, we can set rules and boundaries for data access. It hopefully is protected and encrypted and only arrives in the hands of the intended. In the case of the proliferation of the zettabytes of data being created by the proliferation of mobile devices and more, it is open season for data. It can get into the wrong hands with the public, within companies, and within Wall Street.
More data does not mean the right information. It just means more information. And more information in the right hands can mean better decisions. More information in the wrong hands is data misuse or abuse. This is why we expect to see more hiring mistakes in data management for companies than any other position. The right person can empower the enterprise; the wrong person can poison it.
The data geek is now one of the most powerful jobs in corporations. It touches every aspect of a company—most importantly customer relationships. While its public face may be Nate Silver, its commercial value is, and has been, exemplified by Wal-Mart. Behind great decisions are the smartest people interpreting big data for big business.