Project

Back to overview

Advanced Learning Methods On Dedicated nano-Devices (ALMOND)

Applicant Luisier Mathieu
Number 198612
Funding scheme Sinergia
Research institution Institut für Integrierte Systeme ETH Zürich
Institution of higher education ETH Zurich - ETHZ
Main discipline Interdisciplinary
Start/End 01.03.2021 - 28.02.2025
Approved amount 2'236'277.00
Show all

All Disciplines (5)

Discipline
Interdisciplinary
Material Sciences
Microelectronics. Optoelectronics
Condensed Matter Physics
Information Technology

Keywords (5)

physics-based modeling; spiking neural networks; artificial intelligence; three-factor learning; memristor design

Lay Summary (French)

Lead
L’intelligence artificielle (IA) joue un rôle majeur dans des domaines aussi variés que la reconnaissance vocale, la classification d’images, ou bien les véhicules autonomes. La plupart de ces applications s’appuient sur de réseaux neuronaux profonds. Bien que très puissants, ceux-ci ne permettent pas (encore) à un processeur dédié à l’IA d’atteindre les capacités d’apprentissage d’un être humain, en particulier en terme d’efficacité énergétique. La technologie au cœur des circuits intégrés demande en effet un échange continu de données entre deux entités séparées physiquement, la mémoire et le centre de calcul, qui sont co-localisés dans le cerveau.
Lay summary

Le projet ALMOND a pour objectif de développer une nouvelle génération de processeurs dits neuromorphiques qui évite les problèmes des circuits traditionnels (transfert de données) tout en supportant l’implémentation de méthodes d’apprentissage inspirées par le cerveau humain. Pour cela, des nano-dispositifs appelés memristors seront utilisés, ces derniers pouvant émuler le fonctionnement des synapses qui relient les neurones entre eux. L’originalité d’ALMOND réside dans la prise en compte non seulement de signaux émis par les neurones mais également de ceux provenant de neuromodulateurs tels que la dopamine. Celle-ci sera délivrée par l’intermédiaire d’un contact optique ajouté aux memristors. Une puce électronique basée sur cette technologie sera fabriquée. Elle sera capable de reproduire les actions d’un rat qui apprend à se mouvoir dans un labyrinthe grâce aux récompenses qui lui sont offertes en cas de succès.    

Notre travail contribuera à l’émergences de nouvelles méthodes d’apprentissage pour les réseaux neuronaux. Elles seront programmées sur une toute nouvelle plateforme technologique qui leur sera entièrement consacrée.

Direct link to Lay Summary Last update: 16.03.2021

Responsible applicant and co-applicants

Employees

Project partner

Associated projects

Number Title Start Funding scheme
180604 NCCR SPIN (phase I) 01.08.2020 National Centres of Competence in Research (NCCRs)
182892 NCCR MARVEL: Materials’ Revolution: Computational Design and Discovery of Novel Materials (phase II) 01.05.2018 National Centres of Competence in Research (NCCRs)
175479 Ab-initio modeling of electro-thermal effects in 2-D materials: from single-layer to van der Waals heterostructure (ABIME) 01.03.2018 Project funding (Div. I-III)
184615 Synaptic plasticity in system models 01.04.2019 Project funding (Div. I-III)
169413 ULTIMATE: Upper Limit Technology Investigations Mandatory to Attain Terahertz Electronics 01.02.2017 Project funding (Div. I-III)
175801 Novel Architectures for Photonic Reservoir Computing 01.05.2018 Project funding (Div. I-III)

Abstract

Artificial Intelligence (AI) is nowadays a key technology for workloads as different as speech recognition, pattern classification, medical diagnosis, robotics, or autonomous self-driving vehicles. This revolution has been enabled by ground-breaking techniques for the training of artificial neural networks (ANN), as well as by the availability of large amounts of labelled digital data. Graphics Processing Units (GPUs) and Application Specific Integrated Circuits (ASICs) have brought the required computational power to support this evolution. Yet, despite impressive achievements, even the most efficient computing machines need much more time and energy to learn things than young children can easily and naturally acquire!One reason is that current AI systems are structured very differently than the brain, their memory and processing units - fabricated with complementary metal-oxide-semiconductor (CMOS) - being physically separated. Consequently, data must be constantly transferred between them. This well-known “von Neumann” bottleneck severely limits most brain-inspired computing solutions. In biological neural networks, the memory (synapses) and computing units (neurons) are co-localized, thus allowing for lower power consumptions. Hence, major industrial players - IBM, NVIDIA, Intel, Samsung - are currently developing dedicated hardware where memory and computing are, indeed, co-localized to accelerate neural networks and improve their efficiency. The ALMOND project aims to bring this field to an even higher level by also making the learning processes more biologically-inspired, i.e. by going beyond so-called two-factor Hebbian learning rules. The aforementioned co-localization can be ideally obtained with memristors, a class of nano-devices with two contacts and a tunable, non-volatile conductance. The latter defines the strength (weight) of the synaptic path connecting two neurons, whereas the current flowing through arrays of memristors represents the processed data. To train a neural network the weight of each pseudo-synapse must be adjusted using a learning rule. Recent neuroscience studies have shown that the modification of synaptic weights in the brain does not only depend on signals coming from two contacts corresponding to the firing of pre- and post-synaptic neurons, but also on the presence of a third factor via neuromodulators. Dopamine, for example, globally affects the functionality of synapses and is at the origin of reward-driven learning.Research on memristors is extremely active worldwide, while three-factor learning rules have already been conceptualized by theoretical neuroscientists. Although three-terminal memristors have been fabricated before, none of them has been specifically designed to implement three-factor learning processes, for which very specific requirements must be satisfied. In particular, an independent and global modulation of the memristor conductance must be achieved with its third terminal. Neither a gate contact nor the addition of a transistor in series with a memristor can directly provide these critical features. ALMOND proposes a disruptive and interdisciplinary approach to demonstrate that three-factor learning can be realized with electrically- and optically actuated three-terminal memristors where the optical signal plays the role of neuromodulators. ALMOND is at the convergence of recent developments in Computational Neuroscience (Prof. Wulfram Gerstner, EPFL), Materials Science (Dr. Bert Jan Offrein, IBM Research - Zürich), Computational Nanoelectronics (Prof. Mathieu Luisier, ETHZ), and Device Engineering (Dr. Alexandros Emboras, ETHZ). The technological approach will rely on carefully designed memristors based on the relocation of oxygen vacancies whose switching behavior will be modulated by an additional optical stimulus using integrated plasmonics+photonics. As demonstrators, two chip generations will be produced that can emulate the three-factor reward- or surprise-modulated learning of a mouse navigating through a complex environment.
-