Project

Discipline

Stochastic inference processor; Graphical models; Electronics inference system; Dendritic processing

Lead
Lay summary
Discovering hidden causes from incomplete sensory data is an ubiquitous problem for biological nervous systems. They achieve this inference with a performance that is as yet unrivaled by technology. Graphical Models (GM) such as Belief Networks and Factor Graphs are powerful models for processing uncertain data, and are being successfully applied to abstract questions of human sensory processing and cognition. However, these abstract Graphical Models and the algorithms that support them do not map easily onto the observed structure and operation of biological neural networks. How can we extend these models and algorithms so that they can be implemented on a neural network where the precision of neurons is limited to only a few bits, communication between them is by asynchronous events, and the inference process must play out in real physical time? How does biology achieve this magic of reliable, scalable inference on its peculiar style of ’hardware’? We propose to address this interesting open question by developing novel spike-based algorithms for inference and learning in probabilistic Graphical Models, and by implementing these concepts in various hardware substrates including custom neuromorphic VLSI circuits. In particular, we will focus on Graphical Models in conjunction with a new version of the Belief-Propagation algorithm which renders it computationally feasible in the case of analog variables. Within this conceptual framework, discrete events in time (spikes) are naturally interpreted as samples of temporal values and are thus rendered capable of carrying analog messages. This new, analog representation of Belief-Propagation forms a clear-cut example of how biologically inspired coding of information leads to computations hardly to be performed in classical, software-based systems. In addition, we will investigate implementations of this algorithm on hardware systems because of the superior scaling in speed that can be achieved by massively parallel hardware versions of locally interacting processes in larger problems solved by message-passing schemes. The potential achievements in this proposal will contribute knowledge towards state-of-the-art event-based processing and future distributed computing technology. Furthermore, the results will bring new insights to neuroscience, especially in the context of understanding information processing in nervous systems.

Direct link to Lay Summary

Last update: 21.02.2013

Active participation

Title	Year

Discovering hidden causes from incomplete sensory data is an ubiquitous problem for biological nervous systems. They achieve this inference with a performance that is as yet unrivaled by technology. Graphical Models (GM) such as Belief Networks and Factor Graphs are powerful models for processing uncertain data, and are being successfully applied to abstract questions of human sensory processing and cognition. However, these abstract Graphical Models and the algorithms that support them do not map easily onto the observed structure and operation of biological neural networks. How can we extend these models and algorithms so that they can be implemented on a neural network where the precision of neurons is limited to only a few bits, communication between them is by asynchronous events, and the inference process must play out in real physical time? How does biology achieve this magic of reliable, scalable inference on its peculiar style of 'hardware'? We propose to address this interesting open question by developing novel spike-based algorithms for inference and learning in probabilistic Graphical Models, and by implementing these concepts in various hardware substrates including custom neuromorphic VLSI circuits.In particular, we will focus on Graphical Models in conjunction with a new version of the Belief-Propagation algorithm which renders it computationally feasible in the case of analog variables. Within this conceptual framework, discrete events in time (spikes) are naturally interpreted as samples of temporal values and are thus rendered capable of carrying analog messages. This new, analog representation of Belief-Propagation forms a clear-cut example of how biologically inspired coding of information leads to computations hardly to be performed in classical, software-based systems.In addition, we will investigate implementations of this algorithm on hardware systems because of the superior scaling in speed that can be achieved by massively parallel hardware versions of locally interacting processes in larger problems solved by message-passing schemes. We will explore computational advantages of hardware implementations of this algorithm in various applications especially those where real-time information is provided by the spatio-temporal asynchronous spike outputs of novel biologically-inspired artificial sensors such as the retina and the cochlea. Spike events from these sensors are naturally correlated in time and neuronal space, thus they already carry some meaningful information about observed processes in the external world.In summary, this project intends to achieve two important goals: First, it will increase fundamental understanding of how inference can be performed using a neural substrate. Second, it will advance the state-of-the-art of real-time distributed event-driven technology by providing new hardware system architectures that can implement inference algorithms. The potential achievements in this proposal will strengthen the top position of Swiss research in state-of-the-art event-based sensors and future distributed computing technology. Furthermore, the results will bring new insights to neuroscience especially in the context of understanding information processing in nervous systems.