Project

Back to overview

Theory and methods for accurate and scalable learning machines

Applicant Cevher Volkan
Number 167319
Funding scheme NRP 75 Big Data
Research institution Laboratoire de systèmes d'information et d'inférence EPFL - STI - IEL - LIONS
Institution of higher education EPF Lausanne - EPFL
Main discipline Mathematics
Start/End 01.09.2017 - 31.08.2021
Approved amount 599'456.00
Show all

All Disciplines (2)

Discipline
Mathematics
Information Technology

Keywords (6)

individualized learning system; optimization; active learning ; machine learning ; bayesian optimization; statistical learning theory

Lay Summary (German)

Lead
Unser Projekt beschäftigt sich mit dem maschinellen Lernen - der Fähigkeit von Computern, aus Daten zu lernen - und seiner Rolle für die Gestaltung von Online-Lernsystemen der nächsten Generation. Die Wissensvermittlung soll durch automatische Anpassung dieser Systeme an die Voraussetzungen, Fertigkeiten und Lerngewohnheiten der Studierenden optimiert werden.
Lay summary

Dieses Projekt möchte die Technologie auf die Verarbeitung der von MOOCs generierten riesigen Datensätze vorbereiten und die Online-Lernsysteme der nächsten Generation vollkommen neu gestalten. Zudem überprüfen wir mithilfe statistischer Theorien die Qualität der gewonnen Lernergebnisse. Ausgehend von diesem neuen maschinellen Lernsystem führen wir ein personalisiertes Online-Lernsystem ein, das die Lernerfahrung der Studierenden verbessert, indem es automatisch deren Lerngewohnheiten und Fähigkeiten berücksichtigt. Ausserdem analysieren wir die bei der Gestaltung solcher personalisierter Systeme anfallenden Entscheidungsprozesse und entwickeln ein geeignetes (Bayessches) System für deren optimale Automatisierung.

Durch das Internet und die Entstehung sogenannter MOOCs (Massive Online Open Courses) sind die Ausgaben für den Zugang zu Informationen signifikant gesunken. Der gesellschaftliche Nutzen von MOOCs liegt auf der Hand: Studierende erhalten grenzenlosen, freien Zugang zu den Lehrveranstaltungen weltweit renommierter Universitäten. MOOCs generieren zudem selbst riesige Datenvolumen. Diese können uns mehr über das Lernverhalten der Studierenden verraten und Grundlage massgeschneiderter Lernangebote sein.

Die für die Analyse von MOOCs genutzten herkömmlichen Technologien des Maschinenlernens (etwa neuronale Netzwerke) sind nicht mehr zeitgemäss. Unser Projekt entwickelt daher moderne Technologien der künstlichen Intelligenz, die aus den von MOOCs generierten Datensätzen effizienter lernen können.

Online-Lernsysteme basieren nicht auf statischen Lehrbüchern. Sie lassen sich jederzeit an den rasanten Wandel der heutigen Zeit anpassen. Mit der Entwicklung von Automatismen für die individuelle Anpassung dieser Systeme an die Bedürfnisse der Studierenden ermöglichen wappnen wir den Wissenstransfer in der Lehre für die Zukunft.

Direct link to Lay Summary Last update: 26.07.2017

Lay Summary (French)

Lead
Notre projet se fonde sur l’utilisation de l’apprentissage automatique, soit la capacité des ordinateurs à apprendre à partir de données, pour élaborer la prochaine génération de systèmes d’enseignement en ligne. Notre objectif est d’adapter automatiquement de tels systèmes aux acquis, aux compétences et au style d’apprentissage des étudiants afin d’améliorer la transmission des connaissances qui leur sont dispensées.
Lay summary

Ce projet entend repenser la conception de la prochaine génération des systèmes d’apprentissage en ligne afin de permettre à la technologie de traiter de grands jeux de données, tels que les CLOT. Le projet validera aussi la qualité de nos résultats d’apprentissage en utilisant la théorie statistique. Sur la base de ce nouvel apprentissage automatique, nous allons nous concentrer sur la mise en place d’un système d’apprentissage en ligne individualisé afin d’améliorer les expériences d’apprentissage des étudiants en explorant automatiquement leur style d’apprentissage et leurs compétences. Nous étudierons en outre les problèmes décisionnels liés à la conception de tels systèmes individualisés et développerons un cadre d’optimisation approprié (bayésien) pour leur automatisation.

Internet a sensiblement réduit le coût de l’accès à l’information, préparant la voie aux cours en ligne ouverts à tous (CLOT, ou MOOCs pour massive open online courses). Les bénéfices sociaux des CLOT sont clairs : ils dispensent aux étudiants un accès transfrontalier et libre aux cours des meilleures universités du monde. Les CLOT engendrent en outre d’énormes volumes de données qui peuvent aider à comprendre le comportement des étudiants afin de leur proposer des offres mieux adaptées à leurs besoins.

Les technologies traditionnelles d’apprentissage automatique utilisées pour analyser les données des CLOT, telles que les réseaux neuronaux, sont dépassées. Notre projet va en conséquence développer de nouvelles techniques afin d’apprendre efficacement à partir de jeux de données des CLOT.

Les systèmes d’apprentissage en ligne n’étant pas basés sur des manuels statiques, ils parviennent à s’adapter aux changements du monde en constante évolution d'aujourd'hui. En développant des moyens automatiques pour individualiser ces systèmes de manière à répondre aux besoins des étudiants, nous allons garantir la pérennisation du transfert de connaissances dans le domaine de la formation.

Direct link to Lay Summary Last update: 26.07.2017

Lay Summary (English)

Lead
Our project focuses on applying machine learning - the ability of computers to learn from data - to the design of the next generation of online education systems. Our goal is to automatically adapt such systems to the background, skills and learning style of students to improve the delivery of knowledge to them.
Lay summary

This project will rethink the design of the next generation of online learning systems to enable the technology to deal with Big Data-sized data sets, such as MOOCs. The project will also validate the quality of our learning results using statistical theory. Based on this new machine learning framework, we will focus on implementing an individualised online learning system to enhance students’ learning experience by automatically exploring their learning style and skills. Moreover, we will study the decision problems in the design of such individualised systems and develop an appropriate (Bayesian) optimisation framework for their automation.

The Internet has significantly reduced the cost of access to information, giving rise to so-called massive online open courses (MOOCs). The social benefits of MOOCs are clear: they provide students borderless, free access to lectures from the best universities in the world. Moreover, MOOCs themselves generate huge amounts of data that can help understand student behaviour to better tailor offerings to their needs.

Traditional machine-learning technologies used to analyse MOOCs data, such as neural networks, are behind the times. Consequently, our project will develop modern machine learning techniques for learning efficiently from data sets in MOOCs.  

As online learning systems are not based on static textbooks, they can keep up with today’s fast-paced changes. By developing automatic ways of individualising these systems to the needs of students, we will ensure that knowledge transfer in education is future-proof.

Direct link to Lay Summary Last update: 26.07.2017

Responsible applicant and co-applicants

Employees

Project partner

Publications

Publication
Iterative Classroom Teaching
Yeo Theresa, KamalarubanParameswaran, SinglaAdish, MerchantArpit, AsselbornThibault, FauconLouis, DillenbourgPierre, CevherVolkan (2019), Iterative Classroom Teaching, in 33rd AAAI Conference on Artificial Intelligence, Honolulu, Hawaii, USAAssociation for the Advancement of Artificial Intelligence, AAAI conference.
A Non-Euclidean Gradient Descent Framework for Non-Convex Matrix Factorization
Hsieh Ya-Ping, Kao Yu-Chun, Mahabadi Rabeeh Karimi, Yurtsever Alp, Kyrillidis Anastasios, Cevher Volkan (2018), A Non-Euclidean Gradient Descent Framework for Non-Convex Matrix Factorization, in IEEE Transactions on Signal Processing, 66(22), 5917-5926.
Robust Submodular Maximization: A Non-Uniform Partitioning Approach
Bogunovic Ilija, Mitrovic Slobodan, Scarlett Jonathan, Cevher Volkan (2017), Robust Submodular Maximization: A Non-Uniform Partitioning Approach, in The 34th International Conference on Machine Learning (ICML), SydneyPMLR, Sydney.
Streaming Robust Submodular Maximization: A Partitioned Thresholding Approach
Mitrovic Slobodan, Bogunovic Ilija, Norouzi-Fard Ashkan, Tarnawsi Jakub, Cevher Volkan (2017), Streaming Robust Submodular Maximization: A Partitioned Thresholding Approach, in Conference on Neural Information Processing Systems (NIPS), Long Beach, 2017. These, Long Beach, California, USANIPS, LOng Beach, California, USA.
High Dimensional Bayesian Optimization via Additive Models with Overlapping Groups
Rolland Paul, Scarlett Jonathan, Bogunovic Ilija, Cevher Volkan, High Dimensional Bayesian Optimization via Additive Models with Overlapping Groups, in 21st International Conference on Artificial Intelligence and Statistics (AISTATS), Lanzarotte, Spain AISTATS, Spain.

Collaboration

Group / person Country
Types of collaboration
University of Zurich Switzerland (Europe)
- in-depth/constructive exchanges on approaches, methods or results
University of Fribourg Switzerland (Europe)
- in-depth/constructive exchanges on approaches, methods or results
EPFL Switzerland (Europe)
- in-depth/constructive exchanges on approaches, methods or results
Max Planck Institute Germany (Europe)
- in-depth/constructive exchanges on approaches, methods or results

Abstract

Automated learning from massive datasets promises potentially huge societal and technological benefits. Efficient learning from massive datasets, however, is currently a grand challenge, since the data growth in modern applications is significantly outpacing the growth of computation. Surprisingly, while probabilistic graphical models (PGM) currently form the most successful “learning machines” in practice, the existing theory and methods date back to the previous century. When confronted with the basic computational and storage challenges of the data deluge, this important probabilistic learning framework severely lacks a modern optimization infrastructure. As a result, this project will develop new joint optimization and learning theoretic foundations of PGM’s to not only break the computational barriers for scalability, but also certify the quality of our learning results. Our basic plan of attack is supported by our theoretical preliminary results of trading-off the data-size and computation with guarantees, as well as our recent computational methods that already solve tera-scale, convex optimization problems as relaxations to non-convex problems. To this end, we will investigate three inter-related research thrusts: (Thrust I) An accurate and scalable prediction framework with PGMs: This thrust exploits the underlying geometry of non-convex PGM learning formulations in order to obtain massive speed-ups in learning. We illus- trate how streaming data models, stochastic approximation, primal-dual smoothing, and conver- gence analysis come together to design new, heuristic-free algorithms with theoretical guarantees. (Thrust II) A flexible decision framework via Bayesian optimization: This thrust builds an active learning framework based on Bayesian optimization that adaptively queries an unknown function in order to build an explicit approximation or to optimize the function with theoretical guarantees. For this purpose, we unify key combinatorial structures (e.g., submodularity) with smoothness models (e.g., Gaussian processes) for rigorous guarantees. (Thrust III) Scalable foundations of learning machines with PGMs and BO:This thrust uses neural networks to significantly speed up the task of Bayesian optimization, and Bayesian optimization to optimize the parameters of neural networks. We will explore these connections and interactions in detail, hence allowing the above research directions to complement each other to the full extent possible for scalable automation. One key application that we focus on implementing in the proposed work is ``individualized learning systems'' (ILS). While textbooks and homework assignments effectively addressed the key educational challenges of the 19th and the 20th centuries, they are often insufficient in today's fast-paced developments as they are static, time-consuming to develop, and quickly become out-of-date. Moreover, the classic, monolithic learning style for all students has limited effectiveness in serving to the skill set, background, or goals of individual students. Similarly, while Massive Online Open Courses (MOOCs) in various platforms, such as Coursera and edX, have gained significant popularity on an unprecedented scale, they generally mimic the classical lecture style and its ``non-adaptive'' structure. Recent advances in educational research hints at new ways of enhancing the learning experience given the student's learning style and abilities. Our contention in this proposal is that by leveraging machine learning research along with educational research, we can bring the potentially great benefits of these advances closer to reality. The application of neural networks, Bayesian optimization, and their fusion represents not only a substantial leap over the state-of-the-art in machine learning but has the potential to lead Switzerland on the forefront of ILS and educational research with clear benefits to Swiss society and its economy.
-