Zurück zur Übersicht

Quality models and resources for the evaluation of machine translation

Titel Englisch Quality models and resources for the evaluation of machine translation
Gesuchsteller/in Popescu-Belis Andrei
Nummer 103318
Förderungsinstrument Projektförderung (Abt. I-III)
Forschungseinrichtung ETI / TIM / ISSCO Université de Genève
Hochschule Universität Genf - GE
Hauptdisziplin Informatik
Beginn/Ende 01.10.2004 - 31.03.2007
Bewilligter Betrag 129'836.00
Alle Daten anzeigen

Keywords (3)

Evaluation of Machine Translation; Quality Models; ISO/IEC Standards

Lay Summary (Englisch)

Lay summary
The main goal of this project is to develop a new method for theevaluation of machine translation (MT) software, that is, for measuringthe qualities of programs that translate texts from one language intoanother.Many metrics have been proposed to measure the qualities of an MT system,but their disparity and diversity make it very difficult to choose theright metric(s), without referring to a framework that compares themqualitatively. Moreover, while some metrics that seem to target adesirable quality are quite costly to apply, cheaper metrics (oftenautomated ones) do not state clearly the quality that they measure. Inbrief, the notions of quality and metric are insufficiently specified inthe domain of MT evaluation. Therefore, systematic evaluation campaignsare infrequent, and give rise to much controversy; outside the campaigns,evaluators are condemned to starting each time from scratch, becausebuilding on previous methodological experience or results is difficult.The solution we propose is the FEMTI framework for MT evaluation, which isbased on recent standards in the field of software evaluation, offering aunified approach to evaluation in a given domain. We are developing afully-functional web-based resource dedicated to MT evaluation, byimplementing a formal correspondence between possible contexts of use ofMT systems, and the quality characteristics required from MT systems usedin the specified contexts. FEMTI also documents and compares the metricsproposed for each quality. FEMTI users (organizations or private persons)start by defining a context of use for their MT system(s), then receive inreturn a list of important qualities and suggestions for measuring them.FEMTI is thus a freely-accessible, scalable framework that embodies alarge amount of expertise, and constitutes a reference for the design ofMT evaluations and their comparison with other evaluations.
Direktlink auf Lay Summary Letzte Aktualisierung: 21.02.2013

Verantw. Gesuchsteller/in und weitere Gesuchstellende


Verbundene Projekte

Nummer Titel Start Förderungsinstrument
113604 Quality requirements and metrics for the evaluation of machine translation: analysis and integration of expertise 01.04.2007 Projektförderung (Abt. I-III)
147653 MODERN: Modeling discourse entities and relations for coherent machine translation 01.08.2013 Sinergia
127510 Improving the coherence of machine translation output by modeling intersentential relations 01.03.2010 Sinergia