EMEA 2021 Student Forum: TinyML Platform for On-Device Continual Learning with Quantized Latent…

EMEA 2021 Student Forum
TinyML Platform for On-Device Continual Learning with Quantized Latent Replays
Leonardo RAVAGLIA, PhD Student, University of Bologna, Italy

In the last few years, research and development on Deep Learning models & techniques for ultra-low-power devices – in a word, TinyML – has mainly focused on a train-thendeploy
assumption, with static models that cannot be adapted to newly collected data without cloud-based data collection and finetuning. Latent Replay-based Continual Learning (CL) techniques [1] enable online, serverless adaptation in principle, but so far
they have still been too computation- and memory-hungry for ultra-low-power TinyML devices, which are typically based on microcontrollers. In this work, we introduce a HW/SW platform for end-to-end CL based on a 10-core FP32-enabled parallel
ultra-low-power (PULP) processor. We rethink the baseline Latent Replay CL algorithm, leveraging quantization of frontend and Latent Replays (LRs) to reduce their memory cost with minimal impact on accuracy. In particular, 8-bit compression
of the LR memory proves almost lossless compared to the full-precision baseline implementation, but requires 4_ less memory, while 7 bit can also be used with minimal accuracy degradation. We also introduce optimized primitives for forward
and backward propagation on the PULP processor, together with data tiling strategies to fully exploit its memory hierarchy, while maximizing efficiency. Our results show that by combining these techniques, continual learning can be achieved in practice
using less than 64MB of memory – an amount compatible with embedding in TinyML devices. On an advanced 22nm prototype of our platform, called VEGA, the proposed solution performs on average 42_ faster than a low-power STM32 L4 microcontroller, being 22_ more energy efficient – enough for a lifetime of 317h when learning a new mini-batch of data once every minute.

source