Temporal information processing with spiking neural networks
In contrast to the widely used and successful artificial neural networks (ANNs), biological neurons as found in the mammalian brain use discrete temporal events, so-called action potentials (spikes), to transmit and process information. Encoding and processing information using spike patterns (Fig. 1) represents a novel computing paradigm that harbours attractive benefits such as energy efficiency (due to sparse activations) and low time-to-solutions, e.g., by using the time-to-first-spike coding scheme where the result of a spiking neural network (SNN) is given by its earliest output spike [1-6].
However, despite their potential benefits, SNNs that utilise temporal coding are notoriously hard to train, and for a long time the state of the art has been to map ANNs to SNN architectures using a rate-based coding scheme by averaging over the number of spikes a neuron emits in a certain time interval, losing the attractive properties of purely temporal coding in the process [7,8]. Recently though, several methods have been proposed that allow us to tap into the potential of temporal coding with spikes for machine learning applications. The most prominent one are "Surrogate Gradients" [9,10], which enable the training of SNNs using the error backpropagation algorithm. This progress opens up new possibilities in studying the properties of spike-based computing systems trained on specific tasks, e.g., for deployment on board a spacecraft.
Project overview
In general, as part of this project we investigate how complex information can be represented using temporal spike patterns, e.g., by exploring adaptive neuron models for efficiently transforming visual inputs to spikes [11], or by exploring novel ways of representing structured information like graphs using spike trains [12]. However, the main focus of this project lies in evaluating the applicability and potential benefits of SNNs for space-relevant tasks like scene classification on board a Cubesat [13] and event-based navigation and control of spacecraft [14]. This research is done together with colleagues of the Politecnico di Milano and the University of Heidelberg as part of two ongoing Ariadna studies.
References:
[1] Thorpe, S., Delorme, A., & Van Rullen, R. (2001). Spike-based strategies for rapid processing. Neural networks, 14(6-7), 715-725.
[2] Thorpe, S., Fize, D., & Marlot, C. (1996). Speed of processing in the human visual system. nature, 381(6582), 520-522.
[3] Mostafa, H. (2017). Supervised learning based on temporal coding in spiking neural networks. IEEE transactions on neural networks and learning systems, 29(7), 3227-3235.
[4] Göltz, J., Kriener, L., Baumbach, A., Billaudelle, S., Breitwieser, O., Cramer, B., ... & Petrovici, M. A. (2021). Fast and energy-efficient neuromorphic deep learning with first-spike times. Nature machine intelligence, 3(9), 823-835.
[5] Comsa, I. M., Potempa, K., Versari, L., Fischbacher, T., Gesmundo, A., & Alakuijala, J. (2020). Temporal coding in spiking neural networks with alpha synaptic function. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 8529-8533). IEEE.
[6] Kheradpisheh, S. R., & Masquelier, T. (2020). Temporal backpropagation for spiking neural networks with one spike per neuron. International Journal of Neural Systems, 30(06), 2050027.
[7] Pfeiffer, M., & Pfeil, T. (2018). Deep learning with spiking neurons: opportunities and challenges. Frontiers in neuroscience, 774.
[8] Kucik, A. S., & Meoni, G. (2021). Investigating spiking neural networks for energy-efficient on-board ai applications. a case study in land cover and land use classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 2020-2030).
[9] Zenke, F., & Ganguli, S. (2018). Superspike: Supervised learning in multilayer spiking neural networks. Neural computation, 30(6), 1514-1541.
[10] Neftci, E. O., Mostafa, H., & Zenke, F. (2019). Surrogate gradient learning in spiking neural networks: Bringing the power of gradient-based optimization to spiking neural networks. IEEE Signal Processing Magazine, 36(6), 51-63.
[11] Hadjiivanov, A. (2021). Continuous Learning and Adaptation with Membrane Potential and Activation Threshold Homeostasis. arXiv preprint arXiv:2104.10851.
[12] Dold, D., & Garrido, J. S. (2021). Spike: Spike-based embeddings for multi-relational graph data. In 2021 International Joint Conference on Neural Networks (IJCNN) (pp. 1-8). IEEE.
[13] Helber, P., Bischke, B., Dengel, A., & Borth, D. (2019). Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(7), 2217-2226.
[14] Sikorski, O., Izzo, D., & Meoni, G. (2021). Event-based spacecraft landing using time-to-contact. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 1941-1950).