Publication

Back to overview

Overview of the 2019 Spoken CALL Shared Task

Type of publication Peer-reviewed
Publikationsform Proceedings (peer-reviewed)
Author BaurClaudia, CainesAndrew, ChuaCathy, GerlachJohanna, QianMengjie, RaynerManny, RussellMartin, StrikHelmer, WeiXizi,
Project A Crowdsourcing Platform for Spoken CALL Content
Show all

Proceedings (peer-reviewed)

Title of proceedings Proceedings of the 8th ISCA Workshop on Speech and Language Technology in Education (SLaTE 2019)
Place Graz, Austria

Open Access

URL https://www.isca-speech.org/archive/SLaTE_2019/pdfs/SLaTE_2019_paper_07.pdf
Type of Open Access Repository (Green Open Access)

Abstract

We present an overview of the third edition of the Spoken CALL Shared Task. Groups competed on a prompt-response task using English-language data collected, through an online CALL game, from Swiss German teens in their second and third years of learning English. Each item consists of a written German prompt and an audio file containing a spoken response. The task is to accept linguistically correct responses and reject linguistically incorrect ones, with “linguistically correct” defined by a gold standard derived from human annotations. Scoring was performed using a metric based on the idea of maximising the ratios correct-accept-rate/false-accept-rate and correct-reject-rate/false-reject-rate. The third edition received sixteen entries, with the best score substantially improving on last year’s edition of the task. We analyse factors which make it difficult to label items correctly, concluding that, as in the previous edition, good speech recognition is most important. Finally, we suggest a strategy for continuing the task.
-