Common language

With so many different nationalities working in the Middle East, businesses cannot afford to be held back by a language barrier. Research by IBM in Cairo could help businesses to overcome it, Daniel Stanton hears.

  • E-Mail
By  Daniel Stanton Published  July 30, 2006

|~|DrAhmedTantawyIBM200.jpg|~|Dr Tantawy: Arabic has aspects that need local understanding.|~|At IBM's Middle East development base, the Technology Development Centre (TDC) in Cairo, its team is working on Arabic language solutions. The centre used to be used only to Arabise IBM's products, but it is now the focus of research for IBM in the Middle East. It might also be where a breakthrough in translation technology takes place.

If the TDC developers achieve their goals, an Arabic user could open any English language web page and either see or hear the contents in Arabic, thanks to new translation technologies. IBM's research could also end up improving the way that internet search results are generated.

Dr Ossama Emam, is the human language technologies manager at the TDC in Cairo. "The amount of information is growing very rapidly with the internet, emails, chat rooms," he says. "You have unstructured data and you have structured data."

This makes it very difficult to search for meaning, he explains. "You have the information on the web that can answer a question but you have no way to access this information," he says.

"This is exactly what we would like to do, to do knowledge extraction. What knowledge extraction means is that you go from data to information to knowledge."

Dr Ahmed Tantawy, IBM technical director for the Middle East and North Africa, is head of the centre and knows the difficulties his team is facing. "The search doesn't only depend on the text, it depends on the context," he says.

"There is more complexity in the language that requires understanding of the language. There are more problems with Arabic than with any other language because of the bi-directional nature of it. We often embed some English words."

IBM's technology seeks to place information in context. Its research in Egypt aims to develop technology to deliver Arabic speech recognition, Arabic OCR, machine translation, information extraction, multilingual adaptive semantic association, and Arabic text-to-speech conversion. This means an English document could be converted into Arabic and read to the user, or that spoken Arabic could be recorded and converted into text automatically.

IBM's other international research centres are carrying out similar translation and language recognition projects in Chinese and English, using the same engine, but the Egyptian TDC faces perhaps the most difficult task, due to the complexities of the Arabic language.

"We refuse to have a separate engine to handle Arabic," says Emam. He says that IBM wanted to make sure that the Arabic was not isolated to its own application but could work with the other language applications. However, this has meant addressing certain difficulties.

"Words with three characters like 'ktb' can mean four or five things: 'he wrote', 'he dictated', 'it was written', 'it was dictated', or 'books'. And you can draw the meaning from the context. So this is one challenge," says Emam.
||**|||~|ibmheadset200.jpg|~|IBM: Research in Cairo is trying
to overcome the complexities of translating English into Arabic.|~|"The aim of human translation is perfect translation of sentences so if I have one document to translate it makes a lot of sense to give it to a human to translate because I need perfect translation. But if I have a million documents on the web to translate I cannot by any means have a human being to translate.

"A machine will be available on demand and be fast. The aim is comprehension, so it’s not completely accurate but it will give you an idea. This is the only way to handle millions of documents."

The system's results will be refined each time it is used, since it will start to work out the connections between different words and understand the contexts in which they are commonly used.

"We've developed a new technology to handle machine translation based on a statistical approach," says Emam. "That's what we always do. We solve a problem using a data-driven approach."

Its Arabic OCR system is also being refined. "What we are trying to do is have a magic stick where we convert any degraded text into correct form without going and manually making the correction," explains Emam. "We model for error, we then use this model during our research, so we know for example 'a' will be misrecognised as 'e'." The system was recently put through its paces during a project to digitise one million Arabic books at the nearby Library of Alexandria.

But it is speech that is proving particularly tricky. "When you're a speaker and you pronounce a word, you can have another different pronunciation, so this introduces another element of difficulty for handling Arabic speech," says Emam.

The system will try to accommodate the different forms of Arabic spoken in the Middle East by catering for Modern Arabic, North African, Gulf and Levantine. In addition, it is not always a straight replacement of one word for another. "'Wsyktbh' is one word in Arabic but a phrase in English. It is 'and he will write it'," Emam says. "How can you handle this? You expect this word to go to one word, however it goes to a phrase."

IBM has already released versions of an Arabic dictation product, ViaVoice, but is continually trying to improve on it. "This is the only available Arabic dictation system in the market today, so you can imagine no one else can solve the challenges in the Arabic language. One of the patents we filed is in the heart of this technology, so anyone who wants to do a similar thing has to come to us. We solved it in a very smart way - there's no other way you can handle Arabic in a dictation system because it's a very highly inflected language. You need a lot of vocabulary to cover the tenses that you can cover in English."

The engine also converts Arabic text into speech. "We started to do Arabic text-to-speech in September 2002 with the objective of developing a modern standard of Arabic text-to-speech," Emam says. "The target is to have a high quality voice and to solve Arabic challenges because without vowels you cannot utter Arabic words. At the end of the project we achieved male and female voices. We do automatic vowelisation of Arabic text so you give us a sentence and it will produce the vowelisation and give it to our text-to-speech to pronounce."

If all of these components can work together and continue to adapt to the subtleties of the Arabic language, the centre could come up with technology that takes a significant step towards helping international communication.||**||

Add a Comment

Your display name This field is mandatory


Security code