Connect with us
Big Data 2017 Big Data 2017


Why 2017 Will Be the Year of Big Data

 11 min read / 

With the rise of data as a resource and a tool to be exploited by businesses for enhancing their functions and systems, analytics and targeted strategies incorporating data have become the key to conducting business effectively. Big Data, in particular, can provide extremely powerful business intelligence.

With an estimated 40 zettabytes (43 trillion gigabytes) of data to be created by 2020 (an increase of 300 times on the amount of data in circulation in 2005), at a rate of 2.5 quintillion bytes per day, data is a valuable resource.

In a world of 7 billion people, 6 billion have phones. Those phones harbour sensors, collecting data on location, usage, application data, internet utility and even whether someone is holding the phone. Understanding what data is collected, what it means and how to use to produce more sales (to name one example) is the next battleground for companies and firms alike. Everyone is both collecting data and mapping out their lives.

Companies are collecting this data from sources such as transactions, social media, and mobile devices in order to leverage it and adjust their policies accordingly. For example, cars which can have over 100 sensors, to monitor fuel, travel distance, engine health, average journey time etc. can enable car companies to monitor customer usage, tailoring the next car plan to their needs.

Big Data: The New Frontier?

The continued development of the internet and interconnected technologies is creating unfathomably large datasets. However, the challenge of Big Data is also what makes it so rewarding for business: if a company can crack Big Data, they can uncover trends, spot correlations and increase the effectiveness of their decision-making. The rise of Big Data has been brought on by the increased online storage space and an enhanced computing power, capable of dealing with larger quantities of data. With data being increasingly seen as a resource rather than a by-product of conducting business, companies are trying to extract more data from their customers.

In 2010, the Big Data industry was valued at more than $100bn, stimulating interest in its usage and was growing at almost 10% a year, prompting companies such as IBM, Software AG and Oracle Corporation to spend more than $15bn on department specialisation and Big Data talent acquisition. According to IDC estimates, the market for Big Data investments is set to grow from $16.55bn in 2014 to about $41.52bn in 2018, a compound annual growth rate of 26.24%. Furthermore, McKinsey & Company, in a survey of 714 companies, measured the returns on investment for Big Data users. Whilst reporting that past technology trends take time to return tangible results, there was an average increase in profits by 6% over five years.

Additionally, Big Data investment amounted to 0.6% of corporate revenues and ‘returned a multiple of 1.4 times that level of investment, increasing to 2.0 times over five years.’ As a result, the survey found that Big Data’s introduction has mimicked cycles close to those of previous technology and IT introductions, such as the computer investment cycle of the 80s. To this end, investing early in the cycle could return large results.

First movers are more likely to learn by trial and error, augmenting their current competitive advantage with home-grown, organic practices that help differentiate from other competitors. If the Big Data cycle continues in a similar fashion to previous tech cycles, being able to use the technology before it becomes mainstream will help establish companies as key entities in the sector.


  • AI and the Internet of Things

Undoubtedly, this year there will be advancements in AI technology. By exploiting advancements in big data analytics, processing power and clearer computer systems and networks, companies can use automation and AI to augment their current work processes and generate methods of long-term value creation. Moreover, as these technologies become more commonly used and used together, the integration between these systems in companies will see the benefits of all three create value for the company.

Together with the increasing amount of connectedness, from cars on a network to increasingly accurate sensors in phones, companies will need to become better equipped at data processing, storage and retrieval. Within this, same infrastructure used for the Internet of Things lends itself well to analytics as, by having everything connected and communicating, data can be collected from all parts of a network. However, in this instance, it is the need for strong infrastructure that will benefit Big Data, since companies will have to operate at speed to be able to remain competitive.

Furthermore, as these systems are used, the greater one’s understanding grows, thus allowing one to enhance those same systems. Big Data can be used to compliment AI systems, even allowing companies to utilise machine learning to program the AI on the fly. As a result, Big Data and AI, in 2017, will become more intertwined.

A further effect of advancing technology in Big Data, AI and machine learning is an increase M&A activity. Google, Apple and Microsoft are examples of major players that drove the trend in 2016, acquiring start-ups that have very little running time but host experts in AI and valuable technology. No doubt this will become more aggressive in 2017 as valuable start-ups become more attractive.

Withal, as the advancements continue, one will see AI and Big Data become commoditised by companies such as Google and Microsoft, meaning that it will be more accessible and become easy for developers to analyse larger sets. One can see this occurring in the rise of ‘Chatbots.’ In 2016, the foundations were laid for these AI and in 2017 one will likely see them reach greater levels of integration. Similarly, Big Data will see commoditisation as it becomes an integral part of larger business operations, particularly Business to Consumer, due to the effect of companies fearing that they are missing out and thus implementing data strategies.

Yet, with these advancements come further issues. Data production is speeding up and becoming more voluminous. The sheer volume of data generated could lead to data overload and companies must use overcome this. What’s more, according to Slava Koltovich, CEO, EastBanc Technologies:

“In 2017, we’ll see data become more intelligent, more useable, and more relevant than ever.”

This is because the cloud systems have entered the commoditisation phase alongside the ‘increasing democratisation of artificial intelligence which is driving more sophisticated consumer insights and decision-making.’ Consequently, Big Data will also follow suit in 2017, impacting business strategy and business models.

  • Information-Heavy Sectors

Specifically, sectors such as law, rely on a lot of data processing and comparatively small decision-making. Whilst AI and Big Data are garnering interest in this sector, if one looks closer, the systems are in fact being used to speed up ‘routine’ legal work such as research and work that requires very little decision-making such as writing a will.

Thus, the systems are replacing legal services and turning low-level legal advice into a form of packaged, low-cost modules for areas such as wills, contracts, pre-nuptials and non-disclosure agreements for the benefit of consumers. This is due to the fact that more work is required until AI can effectively be utilised for delivering legal services to clients with minimal human involvement. Thus, Roy Russell, CEO of Ascertus Limited, concluded that:


  • Big Data and Big Judgement: Just Noise?

However, the reality of Big Data outside large, often transcontinental organisations, is that smaller enterprises lack the experience, budget or infrastructure to efficiently and effectively deploy Big Data, meaning that rather than enhancing processes, it drains and hinders them instead. Across businesses, it was found that less than 40% of employees have the sufficient skills and mature experience to derive any useful insight from information collected from suppliers and customers. Investments in such Big Data analytics can be useless, potentially even harmful, unless company employees can incorporate that data into complex decision-making.

Since Big Data relies on employee skills, such analytic skills are concentrated in few employees. When a new form of analytics is deployed by a company, the firm will resort to hiring experts in the new field, to help initiate the new system. Yet, the trickle-down effect that companies expect to happen in reality, rarely occurs. These experts become the standard of analytics and very few begin to train methodology to other employees. As a result, companies can become stuck in the expert phase of operations.

To this extent, rather than targeting experts in the field, building departments at scale from the start can address the current rising challenge of expertise deficiency within firms. Rather than concentrating the expertise in the hands of a few and using a trickle-down effect, firms would be better building from the ground up, using workshops, training schemes and other meetings to disseminate key big data skills and complimenting broad employee knowledge with a department of specialised talent. In this manner, companies can avoid the polarising effect of expert hiring and instead ensure that the company is further harmonised. The effect of this will produce forward-looking strategies and maintain a steady lead on competitors.

The Big Data growth has been turned into a race due to the fear of being left behind. Companies do not want to be left to collect dust whilst their competitors seek new opportunities with experimental techniques. It is an investment and a gamble and this reasoning is also seen with law firms adopting AI systems, whilst still in their infancy, one can see large firms such as Clifford Chance and Dentons deploying these systems to aid their legal teams. More and more firms are adopting these techniques to stay competitive, even if the enhancement is minimal.

  • DIY or Outsource?

An ESG survey (July 2012) revealed that 66% of IT and business professionals responsible for their organisations’ data analytics strategies, technologies, and processes considered enhancing analytics a top five business priority. Yet, whether a company should develop their own Big Data segments or outsource is a question that will be facing executives in the coming years.

Firstly, a company needs engineers qualified to configure, allocate, and manage infrastructure. Not only does the firm require a stable, secure and usable internal infrastructure, but the expertise required to implement and use such a structure is costly. This infrastructure must meet the total needs for storage, processing, and life cycle management. If not, meeting the expectations of end users in both performance and availability will be a challenge.

Secondly, companies should not underestimate the costs of employee time required to evaluate, procure, test, deploy, and integrate a full stack of hardware and software. Using Big Data also requires in-house expertise to tune and optimise systems.

A further issue with implementing Big Data is that its effectiveness is limited to the model on which it is predicated. The starkest example is Big Data’s use during the latest Presidential election in the United States. Big Data was used to predict the results, with Forbes commenting that:

 “If you believe in Big Data analytics, it’s time to begin planning for a Hillary Clinton presidency and all that entails.’”

As a result of the model, the data collected was limited in what conclusions could be drawn, leaving out other factors that ultimately led to false records. Such a hurdle can be created by issuing the wrong model, which, if large enough, can be a dangerous issue for the companies using analytics.


Certainly, Big Data represents a tremendous opportunity for businesses to utilise and deploy data to augment corporate strategies. The new methods of analysis allow firms to sift through huge amounts of data in a manner that was never available to them before. Despite these obstacles identified above, the technologies available and being developed have the potential to disrupt and sustain businesses of all sizes.

To be successful, companies must be aware of emerging trends and developments and maintain careful balances of investments in human capital, skills and technology.  Further, companies that investing in Big Data must “becom[e] a magnet for cutting-edge talent,” in order for company executives to turn their “modest data-analytics gains into broader and more substantial ones.”

Have your say. Sign up now to become an Author!

Click to comment

Leave a Reply


Send this to a friend