QLS Seminar Series - Guillaume Lajoie
Top-down optimization recovers biological coding principles of single-neuron adaptation in RNNs
Guillaume Lajoie, University of Montreal
Tuesday December 14, 12-1pm
Zoom Link:聽
Abstract:聽Spike frequency adaptation (SFA) is a well studied physiological mechanism with established computational properties at the single neuron level, including noise mitigating effects based on efficient coding principles. Network models with adaptive neurons have revealed advantages including modulation of total activity, supporting Bayesian inference, and allowing computations over distributed timescales. Such efforts are bottom-up, modeling adaptive mechanisms from physiology and analysing their effects. How top-down environmental and functional pressures influence the specificity of adaptation remains largely unexplored.
In this talk, I will discuss work where we use deep learning to uncover optimal adaptation strategies from scratch, in recurrent neural networks (RNNs) performing perceptual tasks. In our RNN model, each neuron's activation function (AF) is taken from a parametrized family to allow modulation mimicking SFA, and an adaptation controller is trained end-to-end to control an AF in real time, based on pre-activation inputs to a neuron. Remarkably, we find emergent adaptation strategies that implement SFA mechanisms from biological neurons, including fractional input differentiation. This suggests that even in simplified models, environmental pressures and objective-based optimization are enough for sophisticated biological mechanisms to emerge.