Back to overview

Information Retrieval with Hindi, Bengali, and Marathi Languages

Type of publication Peer-reviewed
Publikationsform Contribution to book (peer-reviewed)
Publication date 2013
Author Savoy Jacques Dolamic Ljiljana Akasereh Mitra,
Project Multilingual and Domain-Specific Information Retrieval
Show all

Contribution to book (peer-reviewed)

Book Multilingual Information Access In South Asian Languages
Editor , P. Majumder M. Mitra P. Bhattacharyya L. Subramaniam D. Contractor & P. Rosso
Publisher Springer-Verlag, Berlin
Page(s) 334 - 352
ISBN 978-3-642-40086-5
Title of proceedings Multilingual Information Access In South Asian Languages


Our first objective in participating in FIRE evaluation campaigns is to analyze the retrieval effectiveness of various indexing and search strategies when dealing with corpora written in Hindi, Bengali and Marathi languages. As a second goal, we have developed new and more aggressive stemming strategies for both Marathi and Hindi languages during this second campaign. We have compared their retrieval effectiveness with both light stemming strategy and n-gram language-independent approach. As another language-independent indexing strategy, we have evaluated the trunc-n method in which the indexing term is formed by considering only the first n letters of each word. To evaluate these solutions we have used various IR models including models derived from Divergence from Randomness (DFR), Language Model (LM) as well as Okapi, or the classical tf idf vector-processing approach. For the three languages studied, our experiments tend to show that IR models derived from Divergence from Randomness (DFR) paradigm tend to produce the best overall results. For these languages, our various experiments demonstrate also that either an aggressive stemming procedure or the trunc-n indexing approach produces better retrieval effectiveness when compared to other word-based or n-gram language-independent approaches. Applying the Z-score as data fusion operator after a blind-query expansion tends also to improve the MAP of the merged run over the best single IR system.