Cheat Sheet

Big Data For Dummies

From Big Data For Dummies by Judith Hurwitz, Alan Nugent, Fern Halper, Marcia Kaufman

Companies must find a practical way to deal with big data to stay competitive — to learn new ways to capture and analyze growing amounts of information about customers, products, and services. Data is becoming increasingly complex in structured and unstructured ways. New sources of data come from machines, such as sensors; social business sites; and website interaction, such as click-stream data. Meeting these changing business requirements demands that the right information be available at the right time.

Defining Big Data: Volume, Velocity, and Variety


  • Volume: How much data

  • Velocity: How fast data is processed

  • Variety: The various types of data




Understanding Unstructured Data

Unstructured data is different than structured data in that its structure is unpredictable. Examples of unstructured data include documents, e-mails, blogs, digital images, videos, and satellite imagery. It also includes some data generated by machines or sensors. In fact, unstructured data accounts for the majority of data that's on your company's premises as well as external to your company in online private and public sources such as Twitter and Facebook.

In the past, most companies weren't able to either capture or store this vast amount of data. It was simply too expensive or too overwhelming. Even if companies were able to capture the data, they didn't have the tools to easily analyze the data and use the results to make decisions. Very few tools could make sense of these vast amounts of data. The tools that did exist were complex to use and did not produce results in a reasonable time frame.

In the end, those who really wanted to go to the enormous effort of analyzing this data were forced to work with snapshots of data. This has the undesirable effect of missing important events because they were not in a particular snapshot.

One approach that is becoming increasingly valued as a way to gain business value from unstructured data is text analytics, the process of analyzing unstructured text, extracting relevant information, and transforming it into structured information that can then be leveraged in various ways. The analysis and extraction processes take advantage of techniques that originated in computational linguistics, statistics, and other computer science disciplines.

The Role of Traditional Operational Data in the Big Data Environment

Knowing what data is stored and where it is stored are critical building blocks in your big data implementation. It's unlikely that you'll use RDBMSs for the core of the implementation, but it's very likely that you'll need to rely on the data stored in RDBMSs to create the highest level of value to the business with big data.

Most large and small companies probably store most of their important operational information in relational database management systems (RDBMSs), which are built on one or more relations and represented by tables. These tables are defined by the way the data is stored.The data is stored in database objects called tables — organized in rows and columns. RDBMSs follow a consistent approach in the way that data is stored and retrieved.

To get the most business value from your real-time analysis of unstructured data, you need to understand that data in context with your historical data on customers, products, transactions, and operations. In other words, you will need to integrate your unstructured data with your traditional operational data.

Basics of Big Data Infrastructure

Big data is all about high velocity, large volumes, and wide data variety, so the physical infrastructure will literally "make or break" the implementation. Most big data implementations need to be highly available, so the networks, servers, and physical storage must be resilient and redundant.

Resiliency and redundancy are interrelated. An infrastructure, or a system, is resilient to failure or changes when sufficient redundant resources are in place ready to jump into action. Resiliency helps to eliminate single points of failure in your infrastructure. For example, if only one network connection exists between your business and the Internet, you have no network redundancy, and the infrastructure is not resilient with respect to a network outage.

In large data centers with business continuity requirements, most of the redundancy is in place and can be leveraged to create a big data environment. In new implementations, the designers have the responsibility to map the deployment to the needs of the business based on costs and performance.

Managing Big Data with Hadoop: HDFS and MapReduce






Laying the Groundwork for Your Big Data Strategy

Companies are swimming in big data. The problem is that they often don't know how to pragmatically use that data to be able to predict the future, execute important business processes, or simply gain new insights. The goal of your big data strategy and plan should be to find a pragmatic way to leverage data for more predictable business outcomes.

Begin your big data strategy by embarking on a discovery process. You need to get a handle on what data you already have, where it is, who owns and controls it, and how it is currently used. For example, what are the third-party data sources that your company relies on? This process can give you a lot of insights:

  • You can determine how many data sources you have and how much overlap exists.

  • You can identify gaps exist in knowledge about those data sources.

  • You might discover that you have lots of duplicate data in one area of the business and almost no data in another area.

  • You might ascertain that you are dependent on third-party data that isn't as accurate as it should be.

Spend the time you need to do this discovery process because it will be the foundation for your planning and execution of your big data strategy.

  • Add a Comment
  • Print
  • Share
blog comments powered by Disqus