Project

Back to overview

Spiking Memristive Architectures for Learning to Learn

English title Spiking Memristive Architectures for Learning to Learn
Applicant Indiveri Giacomo
Number 186999
Funding scheme CHIST-ERA
Research institution Institut für Neuroinformatik Universität Zürich Irchel und ETH Zürich
Institution of higher education University of Zurich - ZH
Main discipline Microelectronics. Optoelectronics
Start/End 01.01.2020 - 31.12.2022
Approved amount 813'294.00
Show all

All Disciplines (3)

Discipline
Microelectronics. Optoelectronics
Neurophysiology and Brain Research
Information Technology

Keywords (6)

Recurrent Neural Network (RNN); Neuromorphic hardware; Learning; Edge computing; Memristor; Spiking Neural Network (SNN)

Lay Summary (German)

Lead
Giacomo Indiveri
Lay summary
Die künstliche Intelligenz (KI), die von neuronalen Netzwerken und Lernalgorithmen angetrieben wird, hat sich in jüngster Zeit als eine erfolgreiche Technologie zur Lösung einer Vielzahl komplexer Aufgaben wie Mustererkennung, Szenensegmentierung, Verarbeitung natürlicher Sprache oder maschinelle Übersetzung erwiesen. Das Training und die Ausführung solcher Algorithmen erfordert jedoch oft den Einsatz sehr großer Datensätze und hoher Rechenlasten. Mit diesem Ansatz wird diese Technik die steigende Nachfrage nach einer intelligenten Verarbeitung von Daten und Signalen, die "am Rande" gemessen werden (d.h. in der Umwelt oder in Bereichen, die nicht an Rechenzentren zur zentralen Verarbeitung verbunden sind), nicht mehr erfüllen können.
Wir werden dieses Problem angehen, indem wir autonom lernende KI-Systeme entwickeln, die in elektronischen, und von der funktionsweise des Gehirns angeregte/inspirierte Geräten mit geringer elektrischer Leistung lernen können, die "Spiking Neural Networks" (SNN, gepulste neuronale Netzwerke) verwenden. Diese Vorrichtungen werden mit analogen/digitalen Mixed-Signal-Schaltungen realisiert, die mit nanoskaligen, neuen Speichertechnologien verbunden sind.
Dieser Ansatz kombiniert die drei vielversprechendsten Ansätze zur Minimierung des Energieverbrauchs in der KI-Hardware: 1. analoge neuromorphe Berechnung, 2. Puls-basierte Übertragung und 3. memristiver Analogspeicher.
Unser internationales Team besteht aus führenden Experten in jedem dieser Bereiche, die gemeinsam an der Entwicklung eines funktionsfähigen Prototypsystems arbeiten werden, das für reale KI-Anwendungen mit komplexen zeitlich-varierenden Eingaben und Kurzzeitspeicheranforderungen geeignet ist.
Die Lernmethode, die wir in diesem Zusammenhang anwenden werden, basiert auf "Learning to Learn" (L2L) in neuromorpher Hardware. Zusammenfassend lässt sich sagen, dass das Ziel dieses Projekts darin besteht, vielseitige und adaptive kleine neuromorphe KI-Maschinen mit geringer elektrischer Leistung zu bauen, die auf SNNs mit memristiven Synapsen basieren und L2L verwenden. Die Projektergebnisse werden mit einem Versuchsaufbau in einer realen Robotikanwendung nachgewiesen, welche die Vorteile neuer theoretischer Entwicklungen als auch neuromorpher Hardware-Realisierung demonstrieren kann.
Direct link to Lay Summary Last update: 06.09.2019

Lay Summary (English)

Lead
Giacomo Indiveri
Lay summary
Artificial intelligence (AI), fueled by neural networks and learning algorithms, has recently emerged as a successful technology for solving a wide range of complex tasks, such as pattern recognition, scene segmentation, natural language processing, or automatic language translation. However, training and running such algorithms often requires the use of very large data-sets and heavy computational loads. Following this approach, this technology will not be able to sustain the increasing demand for intelligent processing of data and signals measured "at the edge", i.e., in the environment or in areas that are not connected to data centers for central processing. In this project we will tackle this problem by developing autonomously learning AI systems able to learn in low-power, brain-inspired electronic devices that employ spiking neural networks (SNNs). These devices will be implemented using mixed-signal analog/digital circuits interfaced to nano-scale emerging memories. This approach will combine the three most promising avenues for minimizing energy consumption in AI hardware: (1) analog neuromorphic computation, (2) spike-based communication, and (3) memristive analog memory. Our international team comprises leading experts in each of these fields, who will collaborate on the development of a functional prototype system suitable for real-world AI applications that have complex temporal inputs and demand short-term memory. The learning method we will use in this context is based on the application of "learning to learn" (L2L) in neuromorphic hardware. In summary, the goal of this project is to build versatile and adaptive low-power small size neuromorphic AI machinery based on SNNs with memristive synapses using L2L. The project results will be validated with an experimental setup deployed in a real-world robotics application, as a proof of concept that can demonstrate the benefits of both the new theoretical developments and its neuromorphic hardware realization.
 
Direct link to Lay Summary Last update: 06.09.2019

Responsible applicant and co-applicants

Employees

Project partner

Associated projects

Number Title Start Funding scheme
180316 Neural Processing of Distinct Prediction Errors: Theory, Mechanisms & Interventions 01.09.2018 Sinergia
204651 A neuromorphic device to monitor epileptogenicity in the intracranial EEG 01.01.2022 Project funding
160756 Hybrid CMOS/Memristive Neuromorphic Systems for Data Analytics 01.09.2015 Sinergia

Abstract

Contemporary AI applications often rely on deep learning, which implies heavy computational loads with current technology. However, there is a growing demand for low-power autonomously learning AI systems that are employed “in the field”. We will investigate in this project options for learning in low-power unconventional hardware that is based on spiking neural networks (SNNs) implemented in analog neuromorphic hardware combined with nano-scale memristive synaptic devices. Hence, the envisioned computational paradigm combines the three most promising avenues for minimizing energy consumption in hardware: (1) analog neuromorphic computation, (2) spike-based communication, and (3) memristive analog memory. Experts in each of these fields will collaborate on the development of a functional prototype system. We will in particular consider recurrent SNNs (RSNNs) as theirinternal recurrent dynamics render them more suitable for real-world AI applications that have temporal input and demand some form of short-term memory. We will adapt a recently developed training algorithm such that it can be used to optimize SNNs in neuromorphic hardware with memristive synapses. “In the field” applications often demand online adaptation of such systems, which often necessitates hardware-averse training procedure. To overcome this problem, we will investigate the applicability of “learning to learn” (L2L) to spiking memristive neuromorphic hardware. In an initial optimization, the hardware is trained to become a good learner for the target application. Here, arbitrarily complex learningalgorithms can be used on a host system with the hardware “in the loop”. In the application itself, simpler algorithms - that can be easily implemented in neuromorphic hardware - provide adaptation of the hardware RSNNs. In summary, the goal of this project is to build versatile and adaptive low-power small size neuromorphic AI machinery based on SNNs with memristive synapses using L2L. We will deliver an experimental system in a real-world robotics environment to provide a proof of concept.
-