Towards more efficient few-shot learning based human gesture recognition via dynamic vision sensors

For human gesture recognition task, recent fully supervised deep learning models have achieved impressive performance when sufficient samples of predefined gesture classes are provided. However, these model do not generalise well for new classes, thus limiting the model accuracy on unforeseen gesture categories. Few-shot learning based human gesture recognition (FSL-HGR) addresses this problem by supporting faster learning using only a few samples from new classes of gestures. In this paper, we aim to develop a novel FSL-HGR method that is suitable for deployment on affordable edge devices which enable energy-efficient inference across large number of classes. Specifically, we adapt a surrogate gradient-based spiking neural network model to efficiently process video sequences collected via dynamic vision sensors. With a focus on energy-efficiency, we design two novel strategies, spiking noise suppression and emission sparsity learning which significantly reduce the spike emission rate in all layers of the net work. In addition, we introduce a dual-speed stream contrastive learning algorithm to achieve higher performance without increasing the additional computational burden associated with inference using dual stream processing. Our experimental results on model performance and accuracy demonstrate the effectiveness of our approach. We achieved state-of-ate-art 84.75%, and 92.82% accuracy on 5way-1shot and 5way-5shot learning task with 60.02% and 58.21% reduced spike emission number respectively compared to a standard SNN architecture without the spiking noise suppression and emission sparsity learning strategies when processing the DVS128 Gesture dataset.