Overview of Stemming Algorithms for Indian and Non-Indian Languages
---
paper_title: A Lightweight Stemmer for Gujarati

paper_content:

Gujarati is a resource poor language with almost no language processing tools being available. In this paper we have shown an implementation of a rule based stemmer of Gujarati. We have shown the creation of rules for stemming and the richness in morphology that Gujarati possesses. We have also evaluated our results by verifying it with a human expert.
---
paper_title: Hybrid Inflectional Stemmer and Rule-based Derivational Stemmer for Gujarati

paper_content:

In this paper we present two stemmers for Gujaratia lightweight inflectional stemmer based on a hybrid approach and a heavyweight derivational stemmer based on a rule-based approach. Besides using a module for unsupervised learning of stems and suffixes for lightweight stemming, we have also included a module performing POS (Part Of Speech) based stemming and a module using a set of substitution rules, in order to improve the quality of these stems and suffixes. The inclusion of these modules boosted the accuracy of the inflectional stemmer by 9.6% and 12.7% respectively, helping us achieve an accuracy of 90.7%. The maximum index compression obtained for the inflectional stemmer is about 95%. On the other hand, the derivational stemmer is completely rule-based, for which, we attained an accuracy of 70.7% with the help of suffix-stripping, substitution and orthographic rules. Both these systems were developed to be useful in applications such as Information Retrieval, corpus compression, dictionary search and as pre-processing modules in other NLP problems such as WSD.
---
paper_title: MAULIK: An Effective Stemmer for Hindi Language

paper_content:

In this paper, a new stemmer has been proposed named as "Maulik" for Hindi Language. This stemmer is purely based on Devanagari script and it uses the Hybrid approach (combination of brute force and suffix removal approach). Stemming can be used to improve the effectiveness of information retrieval. The proposed stemmer is both computationally inexpensive and domain independent. The results are favorable and indicate that the proposed stemmer can be used effectively in Information Retrieval systems. This stemmer also reduces the problem of over-stemming and under-stemming which was found in A Light weight Stemmer for Hindi.
---
paper_title: Analysis and evaluation of stemming algorithms: a case study with Assamese

paper_content:

Stemming is the process of automatically extracting the base form of a given word of a language. Assamese is a morphologically rich, relatively free word order, Indo-Aryan language spoken in North-Eastern part of India that uses Assamese-Bengali script for writing. As it is among the less computationally studied languages, our aim is to extract stem from a given word. We adopt the suffix stripping approach along with a rule engine that generates all the possible suffix sequences. We found 82% accuracy with the suffix stripping approach after adding a root-word list of size 20,000 approximately.
---
paper_title: A Rule-Based Extensible Stemmer for Information Retrieval with Application to Arabic

paper_content:

This paper presents a new and extensible method for information retrieval and content analysis in natural languages (NL). The proposed method is stem-based; stems are extracted based on a set of language dependent rules that are interpreted by a rule engine. The rule engine allows the system to be adapted to any natural language by modifying the NL semantic rules and grammar. The system has been fully tested using Arabic, and partially using English, Hebrew, and Persian. We validate our approach using a database-based prototype.
---

Title: Overview of Stemming Algorithms for Indian and Non-Indian Languages

Section 1: INTRODUCTION
Description 1: Introduce the topic of stemming algorithms and their relevance for Indian and non-Indian languages, highlighting the importance for information retrieval systems.

Section 2: STEMMING ALGORITHMS FOR INDIAN LANGUAGE
Description 2: Discuss various stemming algorithms specifically developed for Indian languages such as Assamese, Gujarati, and Hindi. Include examples and details of specific algorithms and their performance.

Section 3: STEMMING ALGORITHMS FOR NON-INDIAN LANGUAGE
Description 3: Cover stemming algorithms applicable to non-Indian languages like Arabic, English, and Farsi. Provide examples of specific algorithms and their methodologies.

Section 4: CONCLUSION
Description 4: Summarize the findings from the comparative study of stemming algorithms for Indian and non-Indian languages. Discuss the impact of these algorithms on information retrieval and future prospects.