Project

spoken language applications; Computer-Assisted Language Learning; crowdsourcing; web

Lead
L'objectif de ce projet est de développer une plate-forme de crowdsourcing pour créer du contenu pédagogique. La plate-forme logicielle utilisée pour créer et héberger ce contenu sera une version étendue de CALL-SLT Lite (http://callslt.unige.ch/demos-and-resources/), développée dans le cadre d'un projet antérieur financé par le FNS. Comme CALL-SLT Lite a été conçue dès le début dans l’optique du crowdsourcing, l'extension la plus importante sera une interface qui facilite la construction des cours par des utilisateurs non experts.
Lay summary
La tâche principale du projet consistera à mettre en place l'infrastructure nécessaire, en particulier les techniques d’incitation et de récompense pour motiver la communauté, ainsi que les métriques pour suivre les progrès. Sur la base d’autres expériences réussies de ce tyoe (Wikipedia, Amazon, OpenStreetMap, Goodreads etc.), nous pensons qu'il devrait être possible de reprendre des techniques connues comme les votes, la médiation ou la gamification pour donner l'élan nécessaire. Le projet sera étroitement intégré au réseau COST d'enetCollect (http://enetcollect.eurac.edu/), dont l’UNIGE est un partenaire actif. Il visera tout d'abord à recruter les créateurs de contenu et  les utilisateurs auprès des membres d'enetCollect. L'architecture du réseau de crowdsourcing sera conçue de manière à pouvoir intégrer facilement d'autres plates-formes d’apprentissage des langues, notamment celles développées par d'autres groupes d’enetCollect. L'échéancier du projet prévoit la mise en place du réseau au cours de la première année, puis la révision de l'architecture au cours des deuxième et troisième années en réponse aux commentaires des utilisateurs.

Direct link to Lay Summary

Last update: 16.01.2018

Spoken CALL Shared Task, second edition

Author	Baur, Claudia; Caines, Andrew; Chua, Cathy; Gerlach, Johanna; Rayner, Manny
Publication date	07.02.2018
Persistent Identifier (PID)	Spoken CALL Shared Task, second edition
Repository	Geneva University website
Abstract

The Spoken CALL Shared Task is an initiative to create an open challenge dataset for speech-enabled CALL systems, jointly organised by the University of Geneva, the University of Birmingham, Radboud University and the University of Cambridge. The task is based on data collected from a speech-enabled online tool which has been used to help young Swiss German teens practise skills in English conversation. Items are prompt-response pairs, where the prompt is a piece of German text and the response is a recorded English audio file. The task is to label pairs as “accept” or “reject”, accepting responses which are grammatically and linguistically correct to match a set of hidden gold standard answers as closely as possible. Resources are provided so that a scratch system can be constructed with a minimal investment of effort, and in particular without necessarily using a speech recogniser.The first edition of the task was announced at LREC 2016, with training data released in July 2016 and test data in March 2017, and attracted 20 entries from 9 groups. Results, including seven papers, were presented at the SLaTE workshop in August 2017. Full details, including links to resources, results and papers, can be found on the Shared Task home page.Following the success of the original task, we are organising a second edition. We will approximately double the amount of training data, provide new test data, and release improved versions of the accompanying resources. In particular, we will make generally available the open source Kaldi recogniser developed by the University of Birmingham, which achieved the best performance on the original task, together with versions of the training and test data pre-processed through this recogniser. Results will be presented in a special session at Interspeech 2018.

Spoken CALL Shared Task, third edition

Author	Baur, Claudia; Caines, Andrew; Chua, Cathy; Gerlach, Johanna; Rayner, Manny
Persistent Identifier (PID)	Spoken CALL Shared Task, third edition
Repository	Spoken CALL Shared Task, third edition
Abstract

The Spoken CALL Shared Task is an initiative to create an open challenge dataset for speech-enabled CALL systems, jointly organised by the University of Geneva, the University of Birmingham, Radboud University and the University of Cambridge. The task is based on data collected from a speech-enabled online tool which has been used to help young Swiss German teens practise skills in English conversation. Items are prompt-response pairs, where the prompt is a piece of German text and the response is a recorded English audio file. The task is to label pairs as “accept” or “reject”, accepting responses which are grammatically and linguistically correct to match a set of hidden gold standard answers as closely as possible. Resources are provided so that a scratch system can be constructed with a minimal investment of effort, and in particular without necessarily using a speech recogniser.The first edition of the Shared Task was carried out in 2017, with results presented at the SLaTE 2017 workshop in Stockholm. The second edition, with improved training data and improved baseline recogniser resources, was carried out in 2018, results this time being presented at a special session of Interspeech 2018 in Hyderabad. Details, including full results and links to all the papers, are available from the Shared Task 1 site and the Shared Task 2 site.The third edition of the Shared Task will make available the same training data and resources as the second edition. There will be new test data. Given the strong results reported in the second edition, we are also making an important change: THE THIRD EDITION WILL USE A NEW METRIC. This new metric, Dfull, is defined in the instructions tab and motivated in §5 of the Interspeech 2018 overview paper. Unlike the metric used in the two previous editions of the Shared Task, which focused on optimizing performance for correct student responses (i.e. responses which should be accepted), Dfull places equal weight on correct and incorrect utterances. Since incorrect responses are considerably harder to process than correct ones, we expect Dfull to pose interesting new challenges.

Full data for "Assessing the Quality of TTS Audio in the LARA Learning-by-Reading Platform"

Author	Akhlaghi, Elham; Bączkowska, Anna; Berthelsen, Harald; Bédi, Branislav; Chua, Cathy; Cucchiarini, Catia; Habibi, Hanieh; Horváthová, Ivana; Hvalsøe, Pernille; Lotz, Roy; Maizonniaux, Christèle; Ní Chiaráin, Neasa; Rayner, Manny; Tsourakis, Nikos; Yao, Chunlin
Publication date	20.08.2021
Persistent Identifier (PID)	not yet available
Repository	CALLector project website
Abstract

A popular idea in CALL is to use multimodal annotated texts, with annotations typically including embedded audio and translations, to support L2 learning through reading. An important question is how to create the audio, which can be done either through human recording or by a TTS engine. We may reasonably expect TTS to be quicker and easier, but human to be of higher quality. Here, we report a study using the open source LARA platform and ten languages. Samples of LARA audio totalling about 3.5 minutes were provided for each language in both human and TTS form; subjects used a web form to compare different versions of the same item and rate the voices as a whole. Although human voice was more often preferred, TTS achieved higher ratings in some languages and was close in others. Links to the relevant LARA texts, the data collection form and the full results.

Active participation

Self-organised

Software

Number	Title	Start	Funding scheme

We propose a three year project, whose central goal is to develop a crowdsourcing network capable of creating large quantities of content for speech-enabled Computer-Assisted Language Learning (CALL). The software platform used to create and host the content would be an extended version of the CALL-SLT Lite platform, developed under an SNSF-sponsored project which has just ended. CALL-SLT Lite has been designed with crowdsourcing in mind as a possible long-term goal; the modifications required are quite small, mainly consisting of a better ``Wizard''-style web interface.The main thrust of the project is concerned with setting up the required crowdsourcing infrastructure, in particular developing the system of incentives and rewards needed to motivate the content-producers, together with a set of metrics that can be used to track progress. Extrapolating from the histories of successful networks of this kind (Wikipedia, Amazon, Goodreads etc), we guess that a well-designed framework using likes/votes, mediation of comments and gamification will be enough to establish the necessary momentum. The project would be closely integrated with the enetCollect COST network, and where our group plays a leading role; it would initially aim to recruit both content-producers and content-consumers from enetCollect. It is indeed precisely the existence of this highly multinational consortium, which already unites hundreds of people interested in crowdsourcing of CALL content, which makes us optimistic that our goals can be attained. The architecture of the crowdsourcing network will be designed so that other CALL platforms (in particular, platforms developed by other groups in enetCollect) can easily be integrated.The timeline of the project envisages setting up the basic crowdsourcing network during the first year, recruiting producers and consumers from enetCollect, then revising the architecture during the second year in response to user feedback. Work during the last third of the project of the project will depend on how things go during the first two years. If, as we hope, the network grows well, we will naturally aim to keep it moving in a good direction; this will almost certainly involve increased collaboration with other enetCollect groups. If it does not, we will replan appropriately.A successful outcome to this project could have a substantial impact on the development of CALL technology for the whole of Europe.