computational neuroscience, spiking neurons, learning, reinforcement learning, decision making, perceptual learning, psychophysics, memory, behavior, neurons, synaptic plasticity

Lead
Lay summary
Human and animals learn by changing the strength of connections between neurons. Suppose we have to learn to navigate through a complex maze - a labyrinth often found in the gardens of old castles. In this case we need to learn that at the first bifurcation we need to turn left, at the second bifurcation right, at the crossing a sharp left-turn and so on. Suppose now that different locations correspond to different `place' neurons and turning left and right to two other populations of neurons. If we strengthen the connection between the neurons coding for the first bifurcation, and those cells coding for left-turn, then the combination `turn left at first bifurcation' becomes more likely. We study in this project in models whether there is an optimal way of changing the connections upon a succesful action - and how this could be implemented in biologically plausible neuronal models.

Direct link to Lay Summary

Number	Title	Start	Funding scheme

Reward-based learning encompasses a broad class of algorithms in the field of machine learning that allow to optimize the behavior of an agent (e.g. of a real or simulated robot) so as to maximize the total expected reward. These algorithms describe learning in machines that is reminiscent of learning in animals or humans as studied in animal behavior (e.g. conditioning) or human psychophysics. Learning in humans or animals in turn is thought to be related to changes in synaptic connections between neurons in the brain. Hence the question arises whether models of synaptic plasticity on the level of spiking neurons can be connected to formal `reinforcement' learning models in machine learning and to human psychophysics and animal behavior. This project combines the expertise from two laboratories in computational neuroscience (EPFL-LCN/Wulfram Gerstner and Univ. Berne/Walter Senn) who have both previously worked on spike-based models of synaptic plasticity, with the machine learning expertise of the Schmidhuber group at IDSIA (Lugano) who have a long-standing track record in formal models of reinforcement learning, with the psychophysics laboratory of Michael Herzog (EPFL-LPSY) who has a long tradition in human vision and perceptual learning, and with the rodent behavior expertise of Carmen Sandi (EPFL-BMI).