tinyML Summit 2021 tiny Talks: Neutrino: A BlackBox Framework for Constrained Deep Learning Model…

tinyML Summit 2021 https://www.tinyml.org/event/summit-2021
“Neutrino: A BlackBox Framework for Constrained Deep Learning Model Optimization”
Davis SAWYER, Co-Founder & Chief Product Officer, Deeplite

Designing modern deep learning-based solutions requires deeper models with a greater number of layers. While a larger, deeper model can provide competitive accuracy, it creates several logistical challenges and unreasonable resource requirements during development and deployment. This has been one of the key reasons for deep learning models not being excessively used in various production environments, especially in tinyML devices. There is an immediate requirement for optimizing and compressing these deep learning models to enable on-device intelligence. In this research, we introduce a black-box framework, Neutrino- for production-ready optimization of deep learning models. The framework provides an easy mechanism for users to provide constraints such as a tolerable drop in accuracy or target size of the optimized models to guide the optimization process. The framework is easy to include in an existing production pipeline and is available as a Python Package or Docker image, supporting PyTorch and Tensorflow libraries. The optimization performance of the framework is shown across multiple benchmark datasets and popular deep learning models, providing a 3-30x reduction in model size (pre-quantization). Furthermore, we will share how the framework is currently used in production and results from several tinyML applications like visual wake words are summarized.

source