A review on the cross-lingual information retrieval
---
paper_title: Introduction to Information Retrieval

paper_content:

Class-tested and coherent, this groundbreaking new textbook teaches web-era information retrieval, including web search and the related areas of text classification and text clustering from basic concepts. Written from a computer science perspective by three leading experts in the field, it gives an up-to-date treatment of all aspects of the design and implementation of systems for gathering, indexing, and searching documents; methods for evaluating systems; and an introduction to the use of machine learning methods on text collections. All the important ideas are explained using examples and figures, making it perfect for introductory courses in information retrieval for advanced undergraduates and graduate students in computer science. Based on feedback from extensive classroom experience, the book has been carefully structured in order to make teaching more natural and effective. Although originally designed as the primary text for a graduate or advanced undergraduate course in information retrieval, the book will also create a buzz for researchers and professionals alike.
---
paper_title: Literature review of cross-language information retrieval

paper_content:

Classical Information Retrieval (IR) is the sifting out of the documents most relevant to a user’s information requirement (expressed as a “query”), from a large electronic store of documents. A search engine performs IR by retrieving relevant web pages from the internet. Rather than regarding foreign-language documents simply as unwanted “noise”, Cross Language Information Retrieval allows the user to state their query in one language, and retrieve documents in another. Some CLIR systems use language resources such as bilingual dictionaries to translate the user’s original query, while other systems use machine translation to translate the foreign-language documents beforehand, enabling them to be retrieved by the original query. Problems arise due to ambiguity in language, the use of synonyms to express a single idea, and the lack of context available in translating a short query. This paper will discuss previous work in CLIR, current problems in CLIR, and make recommendations for future work. Keywords-Cross Language Information Retrieval, Lexical Semantics, Disambiguation, Translation.
---
paper_title: Dictionary-Based Cross-Language Information Retrieval: Problems, Methods, and Research Findings

paper_content:

This paper reviews literature on dictionary-based cross-language information retrieval (CLIR) and presents CLIR research done at the University of Tampere (UTA). The main problems associated with dictionary-based CLIR, as well as appropriate methods to deal with the problems are discussed. We will present the structured query model by Pirkola and report findings for four different language pairs concerning the effectiveness of query structuring. The architecture of our automatic query translation and construction system is presented.
---
paper_title: Alignment of bilingual named entities in parallel corpora using statistical models and multiple knowledge sources

paper_content:

Named entity (NE) extraction is one of the fundamental tasks in natural language processing (NLP). Although many studies have focused on identifying NEs within monolingual documents, aligning NEs in bilingual documents has not been investigated extensively due to the complexity of the task. In this article we introduce a new approach to aligning bilingual NEs in parallel corpora by incorporating statistical models with multiple knowledge sources. In our approach, we model the process of translating an English NE phrase into a Chinese equivalent using lexical translation/transliteration probabilities for word translation and alignment probabilities for word reordering. The method involves automatically learning phrase alignment and acquiring word translations from a bilingual phrase dictionary and parallel corpora, and automatically discovering transliteration transformations from a training set of name-transliteration pairs. The method also involves language-specific knowledge functions, including handling abbreviations, recognizing Chinese personal names, and expanding acronyms. At runtime, the proposed models are applied to each source NE in a pair of bilingual sentences to generate and evaluate the target NE candidates; the source and target NEs are then aligned based on the computed probabilities. Experimental results demonstrate that the proposed approach, which integrates statistical models with extra knowledge sources, is highly feasible and offers significant improvement in performance compared to our previous work, as well as the traditional approach of IBM Model 4.
---
paper_title: Overview of MUC-7/MET-2

paper_content:

Abstract : The tasks performed by the systems participating in the seventh Message Understanding Conference and the Second Multilingual Entity Task are described here in general terms with examples. On the level of entity extraction, Named Entities (NE) were defined as proper names and quantities of interest. Person, organization, and location names were marked as well as dates, times, percentages, and monetary amounts. The annotation was SGML within the text stream. An example from MUC-7 (New York Times News Service) in English follows. The ENAMEX TYPE=LOCATIONU.K./ENAMEX satellite television broadcaster said its subscriber base grew NUMEX TYPE=PERCENT17.5 percent/NUMEX during TIMEX TYPE=DATEthe past year/TIMEX to 5.35 million The Named Entity task was carried out in Chinese and Japanese (MET-2) concurrently with English (MUC-7).
---
paper_title: Translation disambiguation in web-based translation extraction for English-Chinese CLIR

paper_content:

Dictionary based translation is a traditional approach in use by cross-language information retrieval systems. However, significant performance degradation is often observed when queries contain words that do not appear in the dictionary. This is called the Out Of Vocabulary (OOV) problem. In recent years, web-based translation extraction was shown to be one of the more effective approaches to the solution of this problem. Previous work focussed on selecting the correct translation from a set of web extracted terms. The common methods for translation selection for web-based translation always rely on word frequency calculation but the results are not always satisfactory. In this paper we present our approach to the selection of terms in a more accurate manner. Our experiments show improvement in translation accuracy over other commonly used approaches.
---
paper_title: Cross-lingual query suggestion using query logs of different languages

paper_content:

Query suggestion aims to suggest relevant queries for a given query, which help users better specify their information needs. Previously, the suggested terms are mostly in the same language of the input query. In this paper, we extend it to cross-lingual query suggestion (CLQS): for a query in one language, we suggest similar or relevant queries in other languages. This is very important to scenarios of cross-language information retrieval (CLIR) and cross-lingual keyword bidding for search engine advertisement. Instead of relying on existing query translation technologies for CLQS, we present an effective means to map the input query of one language to queries of the other language in the query log. Important monolingual and cross-lingual information such as word translation relations and word co-occurrence statistics, etc. are used to estimate the cross-lingual query similarity with a discriminative model. Benchmarks show that the resulting CLQS system significantly out performs a baseline system based on dictionary-based query translation. Besides, the resulting CLQS is tested with French to English CLIR tasks on TREC collections. The results demonstrate higher effectiveness than the traditional query translation methods.
---
paper_title: ORank: An ontology based system for ranking documents

paper_content:

Increasing growth of information volume in the internet causes an increasing need to develop new (semi)automatic methods for retrieval of documents and ranking them according to their relevance to the user query. In this paper, after a brief review on ranking models, a new ontology based approach for ranking HTML documents is proposed and evaluated in various circumstances. Our approach is a combination of conceptual, statistical and linguistic methods. This combination reserves the precision of ranking without loosing the speed. Our approach exploits natural language processing techniques for extracting phrases and stemming words. Then an ontology based conceptual method will be used to annotate documents and expand the query. To expand a query the spread activation algorithm is improved so that the expansion can be done in various aspects. The annotated documents and the expanded query will be processed to compute the relevance degree exploiting statistical methods. The outstanding features of our approach are (1) combining conceptual, statistical and linguistic features of documents, (2) expanding the query with its related concepts before comparing to documents, (3) extracting and using both words and phrases to compute relevance degree, (4) improving the spread activation algorithm to do the expansion based on weighted combination of different conceptual relationships and (5) allowing variable document vector dimensions. A ranking system called ORank is developed to implement and test the proposed model. The test results will be included at the end of the paper.
---
paper_title: Semantic Cross-lingual Information Retrieval

paper_content:

Cross lingual Information Retrieval (CLIR) refers to the information retrieval activities in which the query and/or documents may appear in different languages. Dictionary-based query translation has been a common method in CLIR systems. In these methods we face with the problem of translation ambiguity in which a single word in one language has more than one translation in the other language. In this paper we propose a hybrid approach to retrieve English documents relevant to Persian queries. In this approach we exploit a combination of phrase reorganization, pattern based phrase translation and query expansion before and after translation to improve the dictionary-based query translation. We also propose an improved probabilistic algorithm to choose the best translation of words and phrases. Finally, the documents will be ranked according to statistical language model with some translation steps. Our experimental results show that each of the mentioned methods can bring significant improvement over simple dictionary approaches.
---
paper_title: Combining fields for query expansion and adaptive query expansion

paper_content:

In this paper, we aim to improve query expansion for ad-hoc retrieval, by proposing a more fine-grained term reweighting process. This fine-grained process uses statistics from the representation of documents in various fields, such as their titles, the anchor text of their incoming links, and their body content. The contribution of this paper is twofold: First, we propose a novel query expansion mechanism on fields by combining field evidence available in a corpora. Second, we propose an adaptive query expansion mechanism that selects an appropriate collection resource, either the local collection, or a high-quality external resource, for query expansion on a per-query basis. The two proposed query expansion approaches are thoroughly evaluated using two standard Text Retrieval Conference (TREC) Web collections, namely the WT10G collection and the large-scale .GOV2 collection. From the experimental results, we observe a statistically significant improvement compared with the baselines. Moreover, we conclude that the adaptive query expansion mechanism is very effective when the external collection used is much larger than the local collection.
---
paper_title: GroupLens: an open architecture for collaborative filtering of netnews

paper_content:

Collaborative filters help people make choices based on the opinions of other people. GroupLens is a system for collaborative filtering of netnews, to help people find articles they will like in the huge stream of available articles. News reader clients display predicted scores and make it easy for users to rate articles after they read them. Rating servers, called Better Bit Bureaus, gather and disseminate the ratings. The rating servers predict scores based on the heuristic that people who agreed in the past will probably agree again. Users can protect their privacy by entering ratings under pseudonyms, without reducing the effectiveness of the score prediction. The entire architecture is open: alternative software for news clients and Better Bit Bureaus can be developed independently and can interoperate with the components we have developed.
---
paper_title: HQE: A hybrid method for query expansion

paper_content:

Query expansion methods have been extensively studied in information retrieval. This paper proposes a query expansion method. The HQE method employs a combination of ontology-based collaborative filtering and neural networks to improve query expansion. In the HQE method, ontology-based collaborative filtering is used to analyze semantic relationships in order to find the similar users, and the radial basis function (RBF) networks are used to acquire the most relevant web documents and their corresponding terms from these similar users’ queries. The method can improve the precision and only requires users to provide less query information at the beginning than traditional collaborative filtering methods.
---
paper_title: Stemming Indonesian: A confix-stripping approach

paper_content:

Stemming words to (usually) remove suffixes has applications in text search, machine translation, document summarization, and text classification. For example, English stemming reduces the words "computer," "computing," "computation," and "computability" to their common morphological root, "comput-." In text search, this permits a search for "computers" to find documents containing all words with the stem "comput-." In the Indonesian language, stemming is of crucial importance: words have prefixes, suffixes, infixes, and confixes that make matching related words difficult. This work surveys existing techniques for stemming Indonesian words to their morphological roots, presents our novel and highly accurate CS algorithm, and explores the effectiveness of stemming in the context of general-purpose text information retrieval through ad hoc queries.
---
paper_title: Developing lexicographic sorting: An example for Urdu

paper_content:

Collation or lexicographic sorting is essential to develop multilingual computing. This paper presents the challenges faced in developing collation sequence for a language. The paper discusses both theoretical linguistic and practical standardization and encoding related considerations that need to be addressed for languages for which relevant standards and/or solutions have not been defined. The paper also defines the process, by giving the details of the procedure followed for Urdu language, which is the national language of Pakistan and is spoken by more than 100 million people across the world. The paper is oriented towards organizations involved in developing and using collation standards and the localization industry, and not focused on theoretical issues.
---
paper_title: The next generation web search and the demise of the classic IR model

paper_content:

The classic IR model assumes a human engaged in activity that generates an "information need". This need is verbalized and then expressed as a query to search engine over a defined corpus. In the past decade, Web search engines have evolved from a first generation based on classic IR algorithms scaled to web size and thus supporting only informational queries, to a second generation supporting navigational queries using web specific information (primarily link analysis), to a third generation enabling transactional and other "semantic" queries based on a variety of technologies aimed to directly satisfy the unexpressed "user intent", thus moving further and further away from the classic model. ::: ::: What is coming next? In this talk, we identify two trends, both representing "short-circuits" of the model: The first is the trend towards context driven Information Supply (IS), that is, the goal of Web IR will widen to include the supply of relevant information from multiple sources without requiring the user to make an explicit query. The information supply concept greatly precedes information retrieval; what is new in the web framework, is the ability to supply relevant information specific to a given activity and a given user, while the activity is being performed. Thus the entire verbalization and query-formation phase are eliminated. The second trend is "social search" driven by the fact that the Web has evolved to being simultaneously a huge repository of knowledge and a vast social environment. As such, it is often more effective to ask the members of a given web milieu rather than construct elaborate queries. This short-circuits only the query formulation, but allows information finding activities such as opinion elicitation and discovery of social norms, that are not expressible at all as queries against a fixed corpus.
---

Title: A Review on the Cross-lingual Information Retrieval

Section 1: INTRODUCTION
Description 1: This section provides an overview of the evolution of information retrieval (IR), emphasizing the importance and challenges of cross-lingual IR, where documents retrieved may be in a language different from the query.

Section 2: CROSS-LINGUAL IR EVALUATION CAMPAIGNS
Description 2: This section discusses various evaluation campaigns and workshops that support cross-lingual IR research, such as CLEF, NTCIR, and TREC.

Section 3: Improving Dictionary-based Cross-lingual IR
Description 3: This part covers methods to improve cross-lingual IR using dictionaries, addressing issues like Out-of-Vocabulary (OOV) problems and named entity translation.

Section 4: Translation Resources beyond Human-constructed Dictionaries
Description 4: This segment explores advanced translation resources and methods like query suggestion using query logs to enhance query translation effectiveness.

Section 5: Ontologies
Description 5: In this section, the use of bilingual ontologies for query expansion and translation in cross-lingual IR is discussed, along with related research outcomes.

Section 6: Interaction with the User
Description 6: This section focuses on the importance of user interaction in cross-lingual IR, detailing techniques like Interactive Query Expansion (IQE) and Real-Time Query Expansion (RTQE).

Section 7: Query Expansion
Description 7: This part outlines various query expansion methods, including ontology-based collaborative filtering and fine-grained term reweighting processes to improve IR performance.

Section 8: Available for All Languages
Description 8: Here, the need for cross-lingual IR research to cover a broader range of languages beyond the commonly used ones is emphasized, indicating that it can be extended to all languages.

Section 9: Multi-lingual IR
Description 9: This section addresses the concept of multi-lingual IR, where searches encompass multiple languages, and discusses its implications on users.

Section 10: Information Supply
Description 10: This final section highlights future trends in IR, focusing on context-driven information supply and social search as emerging paradigms beyond traditional query-based IR.

Section 11: CONCLUSION
Description 11: This concluding section summarizes the new paradigms introduced by cross-lingual IR, its challenges, methods to overcome these challenges, and its potential impact on multi-lingual search.