Spiking Neural Networks for Object Detection

Description: Spiking Neural Networks (SNNs) are considered the third wave of AI [7], holding significant potential for more biologically plausible and energy efficient models. Specifically, in computer vision has seen major improvements in SNN performance in recent years, with a focus on object detection from images and event data. SNNs leverage asynchronous and event-driven computation (spikes), which makes them more efficient than their artificial counterparts (ANNs). Early works in SNNs focused on converting ANNs to SNNs after regular training with backpropagation [1, 3], however, these converted SNNs suffer from high latency and performance degradation. Surrogate gradient learning [2] was proposed to mitigate these issues by directly training SNNs, sparking many advances in object detection with SNNs [4, 5, 6] . Specifically, Li et al. [6] introduce the bio-inspired multi-scale spiking detector (MSD), which achieves state-of-the-art performance among SNNs on the common object-detection dataset COCO and the event-based dataset Gen1. MSD is based on a novel spiking convolutional neuron (SCN) and the optic nerve nucleus block (ONNB), which is constructed from SCNs and models the optic nerve found in the brain.

The goals of this project are to write a comprehensive literature review on SNNs for object detection, re-implement the MSD, analyze its components, and apply MSD to a new dataset.

Supervisor: Anna Penzkofer

Distribution: 40% literature review, 40% implementation, 20% analysis

Requirements: : Programming proficiency in Python, experience with deep learning (PyTorch), ideally some experience with spiking neural networks but not mandatory.

Literature: [1] Zhang et al., Spiking neural networks for object detection and semantic segmentation across event-driven and frame-based modalities: A review, Intelligent Opto-Electronics (2025)

[2] Neftci et al., Surrogate Gradient Learning in Spiking Neural Networks: Bringing the Power of Gradient-Based Optimization to Spiking Neural Networks, IEEE Signal Processing Magazine (2019)

[3] Kim et al. Spiking-YOLO: Spiking Neural Network for Energy-Efficient Object Detection, AAAI (2020)

[4] EMS-YOLO: Su et al., Deep Directly-Trained Spiking Neural Networks for Object Detection. ICCV (2023)

[5] SpikeYOLO: Luo et al., Integer-Valued Training and Spike-Driven Inference Spiking Neural Network for High-Performance and Energy-Efficient Object Detection. ECCV (2024)

[6] Multi-scale Spiking Detector (MSD): Li et al., Brain-Inspired Spiking Neural Networks for Energy-Efficient Object Detection. CVPR (2025)