Big Data

Big Data: Don't spend money just to produce trivia

In all this sound and fury about Big Data, it's easy to forget that its actual intent is to help make a decision.

In the zeal to gain Big Data capabilities, one key element of the process is often overlooked, which causes Big Data efforts to result in little more than trivia. As with many new technology-driven capabilities, Big Data is awash in process, procedure, and the new and organizationally interesting. In all this sound and fury it's easy to forget the actual intent of Big Data-style analysis: helping to make a decision.


If you've ever taken a basic process improvement class, or spent time as a programmer, you've likely heard of the concept of the "black box," or more formally, the Inputs, Process, Outputs (IPO) diagram. The concept suggests that the essence of every business process should be to gather inputs, perform some action on them, and generate outputs. Conceptually simple and fairly obvious, but ensuring you generate the right outputs can be a challenge, especially when it comes to something like Big Data.

We often focus on the wrong outputs when it comes to data analysis, considering the output of the analysis process to be an answer to a question. We might create a Big Data analytic effort to determine which demographic buys a new product, and consider the output a conclusion that teenagers are the right target. This is the wrong output, however, and the results of most data analyses are little more than trivia unless there is a corresponding action associated with the analysis. The action, which should ultimately achieve one of the business's strategic objectives, is the true output.

Planning for outputs

As a key component of developing analytics, consider what actions will occur based on the results of the analytics. Presumably your company will start some new action, stop an existing action, or adjust the resources allocated to various processes. Consider whether you're equipped to make these adjustments. As an analogy, if I hire an architect to design a new house for me, but I don't have a plot of land in mind or the financial resources to execute the project, I'm embarking on a fool's errand that will leave me with a blueprint I'll never use, and wasted time and effort expended on its creation.

Another benefit of planning for outputs in the beginning of a Big Data project is that you'll identify the levers you'll likely adjust as a result of the analysis. As the analysis runs its course, you can actually begin to modify these levers and see the results in your analysis immediately. Contrast that with spooling up a Big Data effort, getting some analysis, then spending months identifying how to react to the analysis as the data grow stale, and the teams that performed the analysis move on to other activities. Just as the speedometer and odometer of your car inform your driving and would be useless when disconnected from the vehicle, so should your Big Data efforts directly inform action; otherwise you're spending money to produce little more than trivia.

About Patrick Gray


Editor's Picks