More than just another Google

Mistakenly compared to the online search giant, Wolfram Research's Alpha is an engine that's about getting the answers you want

Wolfram Research launched its "computational knowledge engine" Alpha to mixed reviews in May. However, founder Conrad Wolfram says the launch of Alpha is only the first step of a multi-decade project.

Alpha already draws on more than two decades of work on the technical computing application Mathematica, the flagship product of Wolfram Research. The application is well-known in academic circles where it is used to perform complex calculations, manipulate data and create graphs and visualisations, but the average internet user is probably not aware of it.

Many reviews compared Alpha unfavourably with general search engines such as Google, but that is like saying that a screwdriver is a bad hammer.

Greg Sterling, contributing editor at the blog Search Engine Land wrote: "By juxtaposing itself with Google or positioning itself as superior to Google in a number of ways, Wolfram created some confusion (and disappointment) in the market."

Alpha and Google are different tools that do different things. Alpha is best thought of as an answers engine rather than a search engine. Alpha doesn't search the web, it queries and performs calculations on some 10tn pieces of mathematical, geometric, financial, chemical, historical and astronomical data. The data sets are curated by the Wolfram Research and have been available in Mathematica since version six, which was launched in May 2007.

It's a system that can perform complex calculations on real-time information such as financial data, but it requires specialist knowledge to unlock its power. With Alpha, Wolfram hopes to unleash this power to the wider world on the web.

The language process

Alpha is built in Mathematica. It takes the computation engine and curated data sets of Mathematica and adds natural language processing. This allows people to enter questions and receive answers from an automated presentation system built largely in Mathematica.

Mathematica users probably instantly understood how to get the most from Alpha because they knew how to create queries and perform calculations using the application. Mathematica users also knew the data that was available to them. Mathematica can draw on so much data that "it is probably more information than people can find," Wolfram said.

By allowing people to ask the system questions, it will allow users unfamiliar with Mathematica to get the answers they want. Mathematica's creator, and Conrad's brother, Stephen Wolfram, described Alpha as the "ultimate 'discovery interface' for Mathematica" allowing people to "sloppily mix all sorts of notations with fragments of natural language". However, the natural language processing engine that interprets the questions needs work.

In a recent blog post, Stephen Wolfram said: "Close to half the time that Wolfram Alpha doesn't give a result, it's not because it doesn't have the necessary knowledge, or can't do the necessary computation. It's because it doesn't understand what's being asked."

They knew the natural language processing engine needed work, but also that they needed more data. 

"We went about as far as we could go without data provided by users' queries," Conrad Wolfram says. And since its release, they have hundreds of thousands of queries to analyse in order to improve the service.

Recently Stephen Wolfram said, "In just a couple of months, we've reduced the 'fall-through rate' of queries we don't understand by 10%."

Alpha is part of a decades long and much larger project of Mathematica started by Stephen Wolfram in the 1980s. It is probably best thought of as an extension of Wolfram Research's effort to bring technical computing and scientific knowledge, if not to the masses, to a much wider audience.

Conrad Wolfram said Alpha is also part of a "business ecosystem of data providers that is starting to take shape", which is democratising knowledge. Governments, scientists, the media and financial businesses are starting to provide data and also easier access to it in the form of machine-readable formats accessed by APIs.

Knowledge is power

The company announced its intention to create an API for Alpha when it launched, and the API launch is "not too far away", according to Wolfram. It will provide access to individual items and to modules, allowing what Wolfram describes as "knowledge-based computing". The modules or pods are the elements automatically generated from an Alpha query.

Wolfram Research has developed an expertise in "holding and organising data", Wolfram says, and the company believes it can provide data management services to other companies and governments. Wolfram Research will also launch a subscription-based professional version of Alpha, allowing users to combine their own private data with the publicly available data sets included in Alpha.

He also sees possibilities for journalism using "charticles". Graphs drawing on updated or even real-time data could be embedded in articles. This is already common on financial news sites, but it could be added to a wider range of articles as the data becomes available.

Wolfram Alpha is not a short-term web play but part of a larger, longer-term and ambitious project. Conrad Wolfram described it as "democratising expertise".

Google says its mission is to "organise the world's information and make it universally accessible and useful". Wolfram Research is trying to make scientific and mathematical knowledge more accessible.

Stephen Wolfram said of Alpha when it launched that: "If the past is a guide, it'll be quite a few years before we understand the full implications of all these possibilities."

Wolfram Research has learned much to improve its answer engine in the months since launch. In using it, I've found I'm learning how to use it and also reminding myself of long ago maths and science lessons. It will be interesting to see how Alpha learns – but also what we learn from it.