Does Artificial Intelligence Threaten the Future of the Data Scientist?

Will the data scientist be required for much longer or can artificial intelligence handle much of what they do?

Data science continues to revolutionize how organizations conduct business as they search for better ways to reach and serve consumers as well as streamline operations and reduce costs.

But increasingly a new issue has emerged around data science: Are data scientists going to be required for long or can artificial intelligence handle much of what they do?

The answer is yes, data scientists will continue to be needed. But the evolution of AI mean changes in the nature of the job.

Worrying about losing jobs to machines in the data science field is not a farfetched concern. Artificial intelligence has reached the point where many of the tasks taken on by data scientists might one day be handled completely by machines.

In theory, that could put the power of sophisticated data research into the hands of so-called “citizen data scientists.”

And it’s coming fast. About 40% of all data scientist tasks will be handled by machines by 2020, according to projections from Gartner, Inc. The company also projects the amount of data analysis done by machines and citizen data scientists will eclipse that done by data scientists in 2019.

But that doesn’t mean there will not be a place for professionals in data science.

Already Taking Data Scientist Jobs

In some cases, the AI evolution is already happening.

For example, Microsoft research scientist Jennifer Chayes said at a Bloomberg event in San Francisco this year that the company’s Custom Decision Service program had already taken at least one human job.

She told the audience that one startup company that used Custom Decision Services was “really pressed” for money so they “got rid of their one data scientist because this worked so much better than their data scientist.”

Experts certainly think that it won’t be long before much of data science is automated. A recent KDnuggets survey asked experts when “most” expert-level data science tasks currently performed by data scientists would be handled by humans.

More than 51% said it would happen in less than 10 years. Perhaps more telling, 23% said it would happen in less than 5 years and more than 5% said it’s already happened.

However, more than 18% said it will never happen.

What AI Can Do

Part of what is driving the popularity of AI and machine learning is the complexity of data science and analysis. Most companies have bought into the idea of using data. However, while there is ample amounts of data, sorting through it all to develop actionable recommendations can prove difficult.

AI essentially offers the promise of handling this for businesses, leading them to the promise land of finding customers, offering them a product in the way they want it and doing it all with streamlined efficiency.

AI is not quite there yet. However, machines already can help businesses with improving the consistency and accuracy of data, as well as finding often deeply-hidden patterns and anomalies within data sets.

AI also excels at integrating or “blending” data. That is, taking data from a variety of different sources and matching it together in a way that preserves the quality and accuracy of the data.

What AI Can’t Do

While automaton can handle complex tasks, some are currently beyond its reach.

For example, AI does not have the sophistication to convert raw data into another easily understood form, also known as “data wrangling.” That takes a human eye.

Also, AI does not yet perform data visualization tasks very well. This also requires a person who can take complex data analysis and present it to business leaders in a way that it is understandable and also actionable.

Perhaps most importantly, AI does not have the curiosity and drive that makes humans conduct data science research in the first place. Without that spark, there would be little need to conduct and validate experiments.

All of this means that the future for data scientists remains bright. While many lower-level data collection and analysis tasks can be taken over by machines, it does not mean that humans aren’t needed to guide the process and interpret it correctly.

Data science AI will likely prove a huge benefit for data scientists who can take more time to develop more complex predicative models and algorithms.

While they will need to be experts in the using machine learning to improve data analysis, they also will find more time to spend on complex tasks rather than mundane data collection tasks. So while data scientists will need to evolve, the job will remain a viable one for decades to come.


Please enter your comment!
Please enter your name here