Tuesday, February 18, 2014

Digital Analytics in National Security

In today’s world of rapidly growing technology, there are billions of communications happening everyday using a plethora of technologies to accomplish the same goal—to communicate with each other. There are friends talking to friends, businesses talking to businesses, and even terrorists talking to terrorists. Can collecting and analyzing this big data actually save your life or the ones you love?

Before we answer that attention-grabbing question, let’s look at a few numbers:

Google reported figures back in 2009 to the SEC that their storage capacity is well over 100 Petabytes (That’s 1,000 Terabytes), they have over 1 trillion indexed URLs (website names), and over 3 million servers with over 7 billion page views per day. Facebook has over 1 billion users with a storage capacity over 300 Petabytes. Roughly 2.5 billion items are shared each day (status updates, wall posts, photos, videos, and comments), 2.7 billion likes per day, and 300 million photos uploaded every day. That means its adding over 500 terabytes of new data every single day into their databases. One interesting fact about Facebook is that 35% of the world’s digital photography is posted to Facebook.[1] YouTube has over 1,000 Petabytes (1 Exabyte) of storage capacity with over 72 hours of content added each minute with over 4 billion views per day. Twitter’s numbers are also staggering with over 124 billion tweets per year or 4,500 tweets per second.[2]
Social media is filled with big data, and we haven’t even looked at text messages, phone calls, and emails. What about Skype, Facetime, Google Hangouts, and other video/text chat services? There are about 6 trillion text messages sent every year or 193,000 text messages sent per second in the United States with an average of 876 per person each year. I know what some of you are thinking… you know you send a lot more than that, and if you have teenagers then that number must be multiplied by at least 20. There are 2.2 trillion cell phone calls made per year, and to make things a little more interesting I have included a graphic of facts about the emails we send and receive[3]

With all this communications data, do you think it could be used to protect us from another terrorist attack? Yes, of course!

How can analyzing all that information protect this country?

It’s hard enough (costly) to collect all that data, but the real value comes from analyzing it into meaningful information that we can use to benefit our personal or business goals. I would like to focus on the implications of all this data in relation to national security. The current controversy about the collection of these communications by the National Security Administration has caused many people to feel uneasy about their right to privacy. It’s no secret that these communications are tracked, stored, and shared with other organizations. These databases collect a vast amount of data, but it is obviously impossible for people to look at all of it. Because of this big data, several complex models and algorithms scan the data based off specified tags and conditions in order to draw conclusions.[4] This is known as digital analytics. It would take several matching conditions, such as a criminal investigation, for a human being to actually look at your communications. Is it worth it to prevent a crime? Do you think the two men responsible for the Boston Marathon bombings were communicating about their intentions via text or any other technology prior to the event? I’m sure they did in one way or another. Were there communications intelligence gathered prior to September 11, 2001 that revealed the intentions of the attacks? Using the communications data to catch “red flags” will continue to keep people safe from such crimes.

Along with government agencies, private companies are collecting information on you also, as displayed in the video below:

Most companies collect specific data on you as you browse the web and/or communicate as do government agencies. Companies do it to target advertising to you and the government agencies do it to protect you from people wishing to do harm.


All these trillions of communications (web-based or not) are collected and stored to make the world a better place, to save the company’s marketing dollars, give you ads on things you actually might care about (I don’t want to see ads to buy the Twilight series on DVD), and to protect you from harm. Social media is very powerful and we should embrace it because it can provide real time information which can be beneficial for us to know before the media can deliver it to us. One interesting example of the power of social media is when a man tweeted about the raid to capture Osama Bin Laden before anyone else knew about it.
You can read his Twitter posts of the raid along with a short introduction here.

[2] Hunt, Ira. Chief Technology Officer for CIA. http://www.youtube.com/watch?v=GUPd2uMiXXg
[4] Information Week. “Military Uses Big Data for Spy Tech.” http://www.informationweek.com/big-data/big-data-analytics/military-uses-big-data-as-spy-tech/d/d-id/1109641?