A recent study published by Iron Mountain and PwC (Price Waterhouse Cooper) revealed that most companies, including mid-size firms and large global enterprises, don’t know what to do with the big data they collect.
The research, based on interviews with 1,800 senior business leaders in North American and European companies, found that “most organizations lack the required skills, technical capabilities and culture to truly gain the greatest advantage from their information. In fact, three in four businesses extract little or no advantage whatsoever.”
This study jives with comments made recently by Intel’s Jason Waxman, vice president and general manager of the cloud platforms group, as reported in Computerworld. He called out “the dirty little secret about big data: No one actually knows what to do with it.”
Too big, too fast?
The trouble with big data is, well, its sheer size. It’s too big for most companies to get their arms around. It’s like a 40-pound hamburger that no one can eat, despite all the juicy nourishment it might contain.
Another tricky aspect of big data is mobility. Big data is fast on its feet. It’s like collecting water in a leaky bucket. As new data pours in, obsolete data pours out. If you are fortunate enough to have data analysts on staff, they’re aiming at a moving target. So analysis must be constant, continual, timely and relevant.
Collection versus Analysis
And that leads us to the crux of the matter: analysis. Extracting value from big data requires it. Simply collecting the data is not enough. Interestingly, the Iron Mountain/PwC study identified a “misguided majority–three in four businesses (76%) that are either constrained by legacy, culture, regulatory data issues or simply lack any understanding of the potential value held by their information.”
In an article entitled “Data Science and Big Data: Two very Different Beasts,” the author, Sean McClure, makes the point that “collecting does not mean discovering.” He likens big data to raw material, such as iron ore, that must be refined and worked to produce a useful tool. Similarly, big data must be refined and worked to produce something that has value to the organization. That’s a role for Data Science, and currently, at least, data scientists are scarce in most organizations.
The Iron Mountain/PwC study found that among the respondents, “23% overall (23% in Europe and 21% in North America) lack the data interpretation skills and 23% overall (25% in Europe and 22% in North America) lack the insight application capabilities required to deliver a return on information.”
Making big data work for you
What happens when the culture, tools, and resources come together to do an effective job with the analysis of big data? In an article entitled “Navigating Big Data for Big Profits,” author Oren Smilansky cites the example of Target. Using analytics, Target was able to identify women who bought certain products with the likelihood of their being or about to become pregnant. This insight allowed Target to design coupons for expectant moms matching different stages of their pregnancy. Insights like these gleaned from analysis of big data led Target to grow revenue from $44 billion per year in 2002 to $67 billion in 2010.
There’s no doubt that big data has tremendous potential to produce insights that can add value to the enterprise. It’s also apparent that the path from data to insight will require the rigorous application of data analytics, calling for a cultural shift in most organizations. The Iron Mountain/PwC study revealed “only 4% of businesses can extract full value from the information they hold.” That’s a lot of room for growth.
Extracting value from big data is not confined to direct retailers like Target. Companies who derive the majority of their sales through the channel can now use third party channel data management firms to analyze point-of-sale, inventory and sales in/sales out data to improve the customer experience and grow revenue.