tinyML Summit 2021 https://www.tinyml.org/event/summit-2021
tinyTalks Algorithms and Tools
“An Introduction to an Open-Source Fixed-Point Inference Framework – NNoM”
Jianjia MA, Research Fellow, University of Southampton
Recent years, the optimization on Neural Networks (NN) structures (such as Inception, ResNet) has effectively reduce the overall computational complexity of an NN model, which brings more potential tinyML applications. However, the complexity of deploying NNs also increased due to more arguments in deeper models, complex structure management and memory management. Neural Network on Microcontroller (NNoM) is a high-level inference framework which aims for providing an easy to use interface for developers to deploy complex NN model while the framework can manage the layer structure, content related arguments and memory. The use of NNoM minimizes the effort of deployment thus developers can focus on optimizing the structure to improve model efficiency.
NNoM is written in C (ISO/IEC 9899:1999) for compatibility to different tinyML development environments. We provided a set of Python scripts which calibrate and quantize parses Keras’ model and write into a single C header for inference. NNoM has a unique compiling process to minimize the memory cost and inter-layer switching time. There are two backends supported, a local C backend and the optimized CMSIS-NN provided by ARM.
A typical footprint for a VGG type model NNoM is 8.9k ROM excluding weights. NNoM supports 30+ different layers, including convolutional layer, fully connected layer, recurrent layers, activations, and others. Evaluation of the quantized model is necessary, NNoM provides many evaluation APIs for evaluating the performance and accuracy on the targeted platform. Besides, NNoM comes with many examples including Speech Keyword Spotting and Speech Noise Suppression. Source code available at https://github.com/majianjia/nnom.