Project

Back to overview

Real-time sound recognition using neuromorphic VLSI

English title Real-time sound recognition using neuromorphic VLSI
Applicant Indiveri Giacomo
Number 119973
Funding scheme Project funding
Research institution Institut für Neuroinformatik Universität Zürich Irchel und ETH Zürich
Institution of higher education University of Zurich - ZH
Main discipline Electrical Engineering
Start/End 01.04.2009 - 31.03.2013
Approved amount 322'639.00
Show all

Keywords (9)

sound-recognition; neuromorphic; analog-digital VLSI; plasticity and learning; automated speech recognition (ASR); event-based processing; silicon cochlea; neuromorphic VLSI; spike-based learning

Lay Summary (English)

Lead
Lay summary
The ability of machines in parsing and recognizing elements of natural acoustic scenes still falls far short of abilities of biological systems. Parts of the reason for this might be the fundamental differences in handling sounds by biological systems and by machines. In machines acoustic signals are typically chopped into short segments equally spaced in time, each segment described by a set of features that are subsequently classified by machine learning techniques; in biological systems auditory processing makes use of continuous time, stimulus-driven, asynchronous, distributed, collective, and self-organizing principles. The goal of this project is to improve sound recognition in machines by closer emulation of these biological principles, and to do so by using real-time low-power neuromorphic VLSI technology. Specifically, we aim to develop an autonomous VLSI sound recognition system that implements a real-time biologically realistic model of auditory processing applicable to recognition of relevant classes of sounds in natural environments, including human speech. We have already fabricated spiking silicon cochlea devices and spike-based neural network learning chips. The project consist in developing cortical-like models for auditory processing, in collaboration with speech-recognition experts at Johns Hopkins University, integrating them into embedded systems using the existing neuromorphic hardware, and achieving real-time context-dependent sound-recognition.
Direct link to Lay Summary Last update: 21.02.2013

Responsible applicant and co-applicants

Employees

Publications

Publication
A VLSI neuromorphic device for implementing spike-based neural networks
Indiveri Giacomo, Chicca Elisabetta (2011), A VLSI neuromorphic device for implementing spike-based neural networks, in Neural Nets WIRN11 - Proceedings of the 21st Italian Workshop on Neural Nets, ItalyIOS Press Ebooks, Netherlands.
Neuromorphic silicon neuron circuits
Indiveri G, Linares-Barranco B, Hamilton TJ, van Schaik A, Etienne-Cummings R, Delbruck T, Liu S-C, Dudek P, Häfliger P, Renaud S, Schemmel J, Cauwenberghs G, Arthur J, Hynna K, Folowosele F, Saighi S, Serrano-Gotarredona T, Wijekoon J, Wang Y, Boahen K (2011), Neuromorphic silicon neuron circuits, in Frontiers in Neuroscience, 5(MAY), 1-23.
Speaker-independent isolated digit recognition using an AER silicon cochlea
Abdollahi M, Liu S-C (2011), Speaker-independent isolated digit recognition using an AER silicon cochlea, in Biomedical Circuits and Systems Conference, San Diego, USAIEEE, San Diego, USA.
Systematic configuration and automatic tuning of neuromorphic systems
Sheik Sadique, Stefanini Fabio, Neftci Emre, Chicca Elisabetta, Indiveri Giacomo (2011), Systematic configuration and automatic tuning of neuromorphic systems, in International Symposium on Circuits and Systems, BrasilIEEE, Brasil.
Exploring Olfactory Sensory Networks: Simulations and Hardware Emulation
Beyeler Michael, Stefanini Fabio, Proske Henning, Chicca Elisabetta (2010), Exploring Olfactory Sensory Networks: Simulations and Hardware Emulation, in Biomedical Circuits and Systems Conference BIOCAS, CyprusIEEE, Paphos, Cyprus.
Spike-based learning with a generalized integrate and fire silicon neuron
Indiveri Giacomo, Stefanini Fabio, Chicca Elisabetta (2010), Spike-based learning with a generalized integrate and fire silicon neuron, in International Symposium on Circuits and Systems, Paris, FranceIEEE, Paris, France.

Collaboration

Group / person Country
Types of collaboration
Johns Hopkins University United States of America (North America)
- in-depth/constructive exchanges on approaches, methods or results
Centre for Theoretical and Computational Neuroscience/Plymuth University Great Britain and Northern Ireland (Europe)
- in-depth/constructive exchanges on approaches, methods or results

Scientific events

Active participation

Title Type of contribution Title of article or contribution Date Place Persons involved
International Workshop on Neuromorphic and Brain-Based Computing Systems Poster A VLSI chip with spike-based synaptic plasticity for online learning in real-time 22.03.2013 Grenoble, France, France Indiveri Giacomo;
Annual Meeting of the Swiss Society of Neuroscience Poster Spike-based auditory scene analysis 02.02.2013 Geneva, Switzerland Abdollahi Mohammad;
Neuroscience Center Zurich ZNZ Symposium 2012 Poster Spike based sensing and processing using neuromorphic systems 14.06.2012 Zurich, Switzerland Abdollahi Mohammad; Indiveri Giacomo; Stefanini Fabio; Liu Shih-Chii;
Neuroscience Center Zurich ZNZ Symposium 2011 Poster Toward spike based sensing and processing using neuromorphic systems 16.09.2011 Zurich, Switzerland Indiveri Giacomo; Stefanini Fabio; Abdollahi Mohammad; Liu Shih-Chii;
Neuroscience Center Zurich ZNZ Symposium 2010 Poster Analysis of spike-based sensing and processing systems 17.09.2010 Zurich, Switzerland Liu Shih-Chii; Stefanini Fabio; Indiveri Giacomo; Abdollahi Mohammad;
PheroSys Workshop on Neuromorphic Olfaction Talk given at a conference Olfactory processing with an event-based neuromorphic VLSI device 24.04.2010 Cyprus, Cyprus Stefanini Fabio;


Knowledge transfer events

Active participation

Title Type of contribution Date Place Persons involved
BrainFair Zürich 2012 Performances, exhibitions (e.g. for education institutions) 12.03.2012 Zurich, Switzerland Indiveri Giacomo; Stefanini Fabio; Abdollahi Mohammad; Liu Shih-Chii;
BrainFair Zürich 2011 Performances, exhibitions (e.g. for education institutions) 14.03.2011 Zurich, Switzerland Liu Shih-Chii; Stefanini Fabio; Abdollahi Mohammad; Indiveri Giacomo;


Associated projects

Number Title Start Funding scheme
121713 Neuromorphic Attention 01.01.2009 Project funding

Abstract

We propose to develop an autonomous embedded sound recognition system that implements a real-time biologically realistic model of auditory processing applicable to recognition of relevant classes of sounds in natural environments, including human speech. The ability of machines in parsing and recognizing elements of natural acoustic scenes still falls far short of abilities of biological systems. Parts of the reason for this might be the fundamental differences in handling sounds by biological systems and by machines. In machines acoustic signals are typically chopped into short segments equally spaced in time, each segment described by a set of features that are subsequently classified by machine learning techniques; in biological systems auditory processing makes use of continuou time, stimulus-driven, asynchronous, distributed, collective, and self-organizing principles. Our goal is to improve sound recognition in machines by closer emulation of these biological principles, and to do so by using real-time low-power neuromorphic VLSI technology. There is a unique opportunity to attack this problem: we can take advantage of recent advances in understanding auditory perception up to cortical levels of cognitive processing, and at the same time exploit the progress made in neuromorphic engineering that offers the possibility of buidling robust, massively parallel, data-driven implementions of biologically realistic processing, in real-time. The sound recognition system we propose to develop will be based on neural architectures implemented on multiple neuromorphic VLSI chips. It will be an embedded multi-chip system, comprising both analog and digital components interfaced via data-driven asynchronous communication schemes based on the Address-Event Representation (AER) protocol. Input auditory signals will be processed by silicon AER cochleas. The resulting spikes will be fed into a cortical-like module for auditory processing. This module will combine state-of-the art techniques derived from classical speech-recognition research with neuroscience models of auditory cortex to extract relevant auditory features. These features, represented as streams of spikes, will be then transmitted to an AER classification chip, comprising an array of silicon neurons and synapses with biologically plausible temporal dynamics, and a novel robust spike-driven learning mechanisms. Feedback signals from the classifier chip will be sent back to the silicon cochlea, actively changing its response properties, and implementing active sensing mechanisms analogous to the ones observed in real cochleas. The on-line (continuosly active) learning mechanisms on the classification chip will support the formation of activity-dependent representations, during perceptual interaction with auditory stimuli. In this way, task-specific context-dependent sound-recognition will emerge. This project will achieve two important objectives: on one side it will advance the state of the art of machine sound recognition by building a hardware sound recognition system, based on hybrid analog/digital asynchronous VLSI devices, that operates continuously in real-time and adapts to the statistics of its input signals; on the other side it will drive the development of the AER hardware infrastructure for interfacing neuromorphic devices to standard computing architectures. This will allow the exploration of hybrid biologically inspired classical machine learning computing paradigms, also by other research groups that do not have access to the neuromorphic VLSI technology. Both potential achievements could have a huge impact on the Swiss and international research community: the first would further strengthen the leading position of Swiss research in auditory and speech processing, and the second would open new opportunities for interdisciplinary collaborations among different research groups and institutions.
-