tinyML Summit 2021 tiny Talks: Performing Inference on Binarized Neural Networks with xcore.ai

tinyML Summit 2021 https://www.tinyml.org/event/summit-2021
tinyTalks Hardware Optimization
Performing Inference on Binarized Neural Networks with xcore.ai
Laszlo KINDRAT, Senior Technologist, XMOS

Ultra-low bitwidth neural networks have been a hot topic in tinyML community, both in terms of novel hardware accelerators, as well as software solutions for training and deployment. In particular, binarized neural networks (BNNs) show large potential due to their simple hardware requirements. xcore.ai (a fast, economical crossover processor from XMOS), has a vector unit with specialized instructions for performing inference on BNNs, which to the best of our knowledge makes it the first MCU class chip with a BNN accelerator in mass production. In this talk we describe these instructions in detail, and how they enable a theoretical maximum of 286GOps/s when executing binarized neural networks. Secondly, we give an overview of our machine learning model deployment toolchain that seamlessly integrates with Larq, a popular open-source framework for training binarized neural networks. Finally, we present performance benchmarks on image classification models with various combinations of binarized and 8bit quantized layers.

source