EMEA 2021 Lightning Talks: Location specific TinyML model calibration

EMEA 2021 https://www.tinyml.org/event/emea-2021
Lightning Talks
Location specific TinyML model calibration
Sam LEROUX, PhD Student, Ghent University

TinyML refers to the use of machine learning models on resource constrained edge devices. Crucially, TinyML allows for intelligent data analysis close to the sensor. We argue that there is a lot of potential to exploit this locality principle to train smaller, more accurate models. SInce the ML model will only process data from a specific sensor instance, it only needs to perform well for this specific sensor and does not need to generalize to data from other locations, making it possible to use models that are smaller than a general purpose model. Intelligent surveillance cameras for example will always monitor the same scene. It is wasteful to use a general object detection model that can recognize a wide variety of objects if the camera is overlooking a highway where it only needs to detect cars and trucks. In addition, it will always observe the objects from a fixed angle so it should only detect features that describe a car from the front for example as the model will never see a car from the back. Another model that observes objects from the side will need to detect other features to make accurate predictions but crucially, no model needs to be able to detect both types of features. Anomaly detection is another use case that could benefit greatly from location specific models. In a factory for example, we could use microphones located near machines to detect failures or accidents. Different locations in the factory have different soundscapes and what is an anomaly at one location might be normal at another location.
A disadvantage of these location specific models is that we now have to train and keep track of a large number of independent models. In this talk we show how we can use Rune and Hammer, two tools developed by Hotg.ai to easily train, manage and deploy location specific models. Rune is our container technology that wraps the ML model and all code for pre- or post-processing of the data into a portable webassembly format. Similar to a docker image, a Rune is configured using a configuration file that describes what input sensor is needed, what preprocessing steps are required, what ML model is used and how the predictions should be used. Because all this functionality is compiled into webassembly, Rune images can be deployed on servers, desktops, smartphones and other edge devices. Hammer is our service which can manage a large number of Runes. Hammer can upload Runes to different devices and can monitor the state and performance of the models. Hammer can collect data from different sources and use these to update the models which are then packaged as a Rune and sent over to the device. The Runes can be swapped on the fly without any interruption in operation. To keep track of the different versions of the model, Rune images can be tagged with a name and version number.
We will show this system in action using a toy example where we trained a speech recognition model to classify voice commands (up, down, left, right) as input for a game of 2048. The model is first trained for english commands. We then show how Hammer can be used to manage different models to deal with different dialects or even different languages.

source