Big data certainly holds a lot of promise for many businesses. In fact, it is that promise that has propelled the concept of big data into the peak of its hype now, even while implementation has yet to go into mainstream.
The rise of big data represents a significant paradigm shift from the limitations of conventional Business Intelligence (BI) — “rearview mirror” reports and generally ‘dated’ information — to more current and predictive data analytics that promote insightful decision making. But with so much data from various sources available and with the capability to pull in and analyze these data into relevant information, why aren’t more enterprises turning to big data?
The truth is, for all its potential, many companies are yet to be convinced that an investment in big data platforms, such as Hadoop technology will be worth it.
Turning data into information
Data is good, and more data is even better. But we’re not talking about structured, transactional data stored in formatted company databases here. These days, the more relevant data may come from external sources outside the company — social media channels, microblogging sites, images, search queries, video and audio files, data from mobile devices, and more. This has sent enterprises scrambling to build huge server farms of massive computing power, and mobilized venture capital firms into investing millions of dollars in technologies that can capture and store petabytes upon petabytes of data.
So the fundamental challenge in big data isn’t so much in collecting the volumes of data dispersed over various sources as it is in transforming these data into meaningful information. To drive business forward, key decision makers need to have access to quality data, not the large quantities of it. Now while the problem may seem straightforward, the solution, i.e. big data implementation, is a technical and costly process. But not if you know where to put your resources.
The impact of ‘small data’
Many are under the impression that only enterprise-level companies can leverage big data but this isn’t exactly true. Applying the adage about not trying to boil the ocean, what SMEs can do is harness the power of the ‘small data’. This allows businesses who can’t go all out in building enterprise data warehouses to still tap into big data, without going overboard. For one, you may not need all that data, plus, you may not have enough skills within the organization to make sense of it all.
Starting small means essentially focusing on models which would entail the least amount of investment, would be simplest to deploy, or would bring in ROI and drive profits the fastest. Establish these business objectives at the outset and work backwards from there. If at all possible, use data that is already on hand, and combine this with externally-sourced data sets, but this time targeting specific outcomes.
Say for instance you’ve already identified customer service as an area that has great impact on your P&L. Then it’s time to revisit those call center records to find out who have been put on hold for long, scour social media sites to find out who are talking about their not-so-pleasant encounters with support services, identify what problems were brought to attention, compare records and see how many have stopped being your customers, and finally, plan on what you can do proactively to ensure less fallout and greater client satisfaction.
Measuring big data ROI
Big data payoff can be easier to measure in some industries more than others. When applied to the web and e-commerce for example, results can be in the form of increased visits, longer session lengths, and more sales or clicks. Financial services can also greatly benefit from big data, particularly where high frequency trading and risk calculation of large loan portfolios are concerned. In manufacturing, big data plays a big role in detecting product defects and boosting quality control.
Whatever the industry however, the key to working with big data is knowing the desired outcomes first, and then using technology to achieve these. So while investment in technology is a given, it doesn’t have to be that high if objectives and efforts are streamlined, and considering the availability of open source tools. The other substantial big data investment would go towards getting the skills and talent of trained thinkers who would help identify specific business outcomes, define processes, cherry pick the data sets where the most insightful information can be derived, and facilitate deployment.
It’s true that big data projects can be difficult, and the sheer magnitude of the data can be daunting. But for those who are clear with the value they can derive from it, those who can work out their big data strategy and get it right, the payback can be huge.