Project

Back to overview

Designing and evaluating spoken dialogue based CALL systems

English title Designing and evaluating spoken dialogue based CALL systems
Applicant Bouillon Pierrette
Number 153278
Funding scheme Project funding
Research institution ETI / TIM / ISSCO Université de Genève
Institution of higher education University of Geneva - GE
Main discipline Education and learning sciences, subject-specific education
Start/End 01.05.2014 - 30.04.2017
Approved amount 306'740.00
Show all

All Disciplines (4)

Discipline
Education and learning sciences, subject-specific education
Education and learning sciences, subject-specific education
Applied linguistics
Information Technology

Keywords (7)

Translation game; CALL; E-learning; Crowdsourcing; Dialog game; Evaluation; Speech recognition

Lay Summary (French)

Lead
L'idée d'utiliser la reconnaissance de la parole pour l'apprentissage des langues est de plus en plus populaire. Les premiers systèmes focalisaient sur la prononciation ; différentes applications permettent aujourd'hui à l'apprenant de dialoguer avec l'ordinateur et lui donnent ainsi la possibibilité d'exercer ses capacités productives. Il reste à démontrer que ce type d'outil a un impact positif et, si oui, quels facteurs influence l'apprentissage. Ce projet vise à examiner systématiquement ces différents facteurs.
Lay summary

L'idée d'utiliser la reconnaissance de la parole pour l'apprentissage des langues est de plus en plus populaire. Les premiers systèmes focalisaient sur la prononciation ; différentes applications permettent aujourd'hui à l'apprenant de dialoguer avec l'ordinateur et lui donnent ainsi la possibilité d'exercer ses capacités productives. Il reste à démontrer que ce type d'outil a un impact positif et, si oui, quels facteurs influencent l'apprentissage. Ce projet vise à examiner systématiquement ces différents facteurs. 

Notre point de départ sera l'outil CALL-SLT, développé dans le cadre d'un projet SNF antérieur. Durant la première moitié du projet, nous allons étendre le prototype, en particulier améliorer la reconnaissance vocale et les mécanismes de rétroaction vers l'apprenant, développer de nouvelles leçons en français et en anglais et intégrer le système dans un réseau social. Dans la seconde partie, nous allons mener des évaluations systématiques au collège, dans les universités et sur le Web.

Direct link to Lay Summary Last update: 31.03.2014

Lay Summary (English)

Lead
The idea of using speech recognition in Computer-Assisted Language Learning is becoming increasingly popular. Early systems focussed on pronunciation practice; applications are now emerging designed to carry out simple conversations with students and give them the opportunity to practise spontaneous speaking skills. Despite positive anecdotal studies, it still remains to be demonstrated that conversation with a mechanical partner is in fact able to help students acquire language skills, and, if so, which features influence learning outcomes. The current project is aimed at systematic investigation of the issues involved.
Lay summary

The idea of using speech recognition in Computer-Assisted Language Learning is becoming increasingly popular. Early systems focussed on pronunciation practice; applications are now emerging designed to carry out simple conversations with students and give them the opportunity to practise spontaneous speaking skills. Despite positive anecdotal studies, it still remains to be demonstrated that conversation with a mechanical partner is in fact able to help students acquire language skills, and, if so, which features influence learning outcomes. The current project is aimed at systematic investigation of the issues involved.

Our starting point is the CALL-SLT system, built under an earlier SNF-funded project. During the first half of the project, we will extend the existing prototype, in particular improving speech recognition and recognition feedback strategies, developing a substantial quantity of new English and French course content, and embedding the system inside a social network. In the second half, we will use these resources to carry out a series of substantial evaluations in secondary schools, universiities and on the web.


Direct link to Lay Summary Last update: 31.03.2014

Responsible applicant and co-applicants

Employees

Publications

Publication
A Shared Task for Spoken CALL?
Baur Claudia, Gerlach Johanna, Rayner Manny, Russell Martin, Strik Helmer (2016), A Shared Task for Spoken CALL?, in Proc. LREC, Portorož, SloveniaELRA, Portorož, Slovenia.
CALL-SLT: a first experiment in a real FFL training for employees of French companies
M. Dejos, C. Petrovic, P. Bouillon, J. Gerlach, H. Duval, M. Rayner, N. Tsourakis (2016), CALL-SLT: a first experiment in a real FFL training for employees of French companies, in ICT for Language Learning, FlorenceLibreriauniversitaria.it. , Florence.
La reconnaissance vocale pour apprendre une langue étrangère : Expériences avec CALL SLT en collaboration avec Intercountry
Christopher Petrovic (2016), La reconnaissance vocale pour apprendre une langue étrangère : Expériences avec CALL SLT en collaboration avec Intercountry, Université de Genève, Genève.
Translation and technology. The case of translation games for language learning
P. Bouillon, C. Cervini, M. Rayner (2016), Translation and technology. The case of translation games for language learning, in H. Farr (ed.), Routledge, Oxon, New York, 536-549.
The Potential of Interactive Speech-Enabled CALL in the Swiss Education System: A Large-Scale Experiment on the Basis of English CALL-SLT
Baur Claudia (2015), The Potential of Interactive Speech-Enabled CALL in the Swiss Education System: A Large-Scale Experiment on the Basis of English CALL-SLT, Université de Genève, Geneva.
Helping Non-Expert Users Develop Online Spoken CALL Courses
Rayner Manny, Baur Claudia, Bouillon Pierrette, Chua Cathy, Tsourakis Nikos (2015), Helping Non-Expert Users Develop Online Spoken CALL Courses, in Proc SLaTE workshop, Leipzig, GermanySLaTE, Leipzig, Germany.
Supervised Learning of Response Grammars in a Spoken CALL System
Rayner Manny, Baur Claudia, Chua Cathy, Tsourakis Nikos (2015), Supervised Learning of Response Grammars in a Spoken CALL System, in Proc SLaTE workshop, Leipzig, GermanySLaTE, Leipzig, Germany.
Build Your Own Speech-Enabled Online CALL Course, No Experience Required
Rayner Manny, Baur Claudia, Bouillon Pierrette, Chua Cathy, Tsourakis Nikos (2015), Build Your Own Speech-Enabled Online CALL Course, No Experience Required, in Proc 9th INTED, Madrid, SpainIATED, Madrid, Spain.
What Motivates Students to Use Online CALL Systems? A Case Study
Baur Claudia (2015), What Motivates Students to Use Online CALL Systems? A Case Study, in Proc. 9th INTED, Seville, SpainINTED, Seville, Spain.
CALL-SLT: A spoken CALL system based on grammar and speech recognition
M. Rayner, N. Tsourakis, C. Baur, P. Bouillon, J. Gerlach (2014), CALL-SLT: A spoken CALL system based on grammar and speech recognition, in Linguistic Issues in Language Technology, 10(2), 1-23.
CALL-SLT Lite: A Minimal Framework for Building Interactive Speech-Enabled CALL Applications
Rayner Manny, Baur Claudia, Tsourakis Nikos (2014), CALL-SLT Lite: A Minimal Framework for Building Interactive Speech-Enabled CALL Applications, in Proc Workshop on Child Computer Communication, SingaporeELRA, Singapore.
Formative Feedback in an Interactive Spoken CALL System
Tsourakis Nikos, Rayner Manny, Baur Claudia (2014), Formative Feedback in an Interactive Spoken CALL System, in Proc Workshop on Child Computer Communication, SingaporeELRA, Singapore.
Crafting Interesting Dialogues in an Interactive Spoken CALL System
Baur Claudia, Rayner Manny, Tsourakis Nikos (2014), Crafting Interesting Dialogues in an Interactive Spoken CALL System, in Proc 6th Edulearn, Barcelona, SpainIATED, Barcelona, Spain.
Using a Serious Game to Collect a Child Learner Speech Corpus
Baur Claudia, Rayner Manny, Tsourakis Nikos (2014), Using a Serious Game to Collect a Child Learner Speech Corpus, in Proc. 9th LREC, Reykjavik, IcelandELRA, Reykjavik, Iceland.
Overview of the 2017 Spoken CALL Shared Task
Baur Claudia, Chua Cathy, Gerlach Johanna, Rayner Manny, Russell Martin, Strik Helmer, Wei Xizi, Overview of the 2017 Spoken CALL Shared Task, in Proc SLaTE, Stockholm, SwedenSLaTE, Stockholm, Sweden.

Collaboration

Group / person Country
Types of collaboration
Lansad, Université de Grenoble France (Europe)
- in-depth/constructive exchanges on approaches, methods or results
- Research Infrastructure
Intercountry France (Europe)
- in-depth/constructive exchanges on approaches, methods or results
- Publication
- Exchange of personnel
Dipartimento di Lingue, Letterature e Culture Moderne, Università di Bologna Italy (Europe)
- in-depth/constructive exchanges on approaches, methods or results
- Publication
- Research Infrastructure

Scientific events

Active participation

Title Type of contribution Title of article or contribution Date Place Persons involved
séminaire invité Individual talk CALL-SLT, un partenaire de conversation automatique 27.05.2015 Riga, Latvia Bouillon Pierrette;


Knowledge transfer events

Active participation

Title Type of contribution Date Place Persons involved


Associated projects

Number Title Start Funding scheme
124664 A generic platform for CALL based on speech translation 01.05.2009 Project funding

Abstract

The idea of using speech recognition in Computer-Assisted Language Learning is becoming increasingly popular. Early systems focussed exclusively on pronunciation practice; now, however, applications are starting to emerge designed to carry out simple conversations with students and give them the opportunity to practice spontaneous speaking skills. Typically, a system of this kind will be deployed on the web, use small-vocabulary recognition, and structure interaction in some kind of game-like way. The novelty of the field, however, means that few solid results have emerged. Despite positive anecdotal studies, it still remains to be demonstrated that systems with these characteristics are in fact able to help students acquire language skills, and, if so, which features influence learning outcomes.Here, we propose a three-year project aimed at systematic investigation of the issues involved. Our starting point will be the CALL-SLT system which we built under an earlier SNF-funded project. We will develop the current CALL-SLT prototype in several directions; the goal is both to extend the state-of-the-art in speech-enabled CALL, and to enhance the system to the point where it will be feasible to carry out the experiments we have in mind. Specifically, we will implement and compare a range of different speech recognition and recognition feedback strategies, develop a flexible framework for specifying script-based interactive dialogue games, use this framework to develop a substantial quantity of course content, and embed the system inside a social network. Content development will focus on English and French as L2s, with a variety of L1s including at least French, German, Italian and English. During the second half of the project, we will use these resources to carry out a series of evaluations in secondary schools, universities and on the web. The web-based evaluations will be designed to contrast different versions of the system against each other and will be performed using crowd-sourcing methods, making it feasible to recruit large enough sets of subjects to perform controlled experiments. The resulting software base, including lingware and courseware, will be made generally available to the community in fully documented Open Source form. We will also release annotated learner speech corpora collected during the experiments.
-