Quark: implementing convolutional neural networks entirely on programmable data plane

The rapid development of programmable network devices and the widespread use of machine learning (ML) in
networking have facilitated efficient research into intelligent data plane (IDP). Offloading ML to programmable data plane (PDP) enables quick analysis and responses to network traffic dynamics, and efficient management of network links. However, PDP hardware pipeline has significant resource limitations. For instance, Intel Tofino ASIC has only 10Mb SRAM in each stage, and lacks support for multiplication, division and floating-point operations. These constraints significantly hinder the development of IDP. This paper presents Quark, a framework that fully offloads convolutional neural network (CNN) inference onto PDP. Quark employs model pruning to simplify the CNN model, and uses quantization to support floating-point operations. Additionally, Quark divides the CNN into smaller units to improve resource utilization on the PDP. We have implemented a testbed prototype of Quark on both P4 hardware switch (Intel Tofino ASIC) and software switch (i.e., BMv2). Extensive evaluation results demonstrate that Quark achieves 97.3% accuracy in anomaly detection task while using only 22.7% of the SRAM resources on the Intel Tofino ASIC switch, completing inference tasks at line rate with an average latency of 42.66μs.