tinyML Summit 2021 tiny Talks: TinyML Software Runtime for Hybrid Multicore Architecture

tinyM Summit 2021 https://www.tinyml.org/event/summit-2021/
tiny Talks
“TinyML Software Runtime for Hybrid Multicore Architecture”
Nilanjan ROYCHOWDHURY, Principal Software Architect, Eta Compute

A lot of emphasis in tinyML has been in designing the best neural network and optimizing it to reduce the number of operations and memory needs. Yet, training a very efficient neural network is only one piece of the equation for TinyML. The other piece is how to run it on actual embedded hardware.
Indeed, the tinyML hardware is very often complex, including many cores for the sake of efficiency. Moreover, because sensor processing requires a combination of signal processing, procedural computing and neural network acceleration, the hybrid multicore architecture is becoming popular for edge AI hardware with a combination of heterogenous cores: CPU, DSP and NPU.
To run efficiently on these hybrid multicore systems, there must be a runtime that allocates resources, core and memory, in the most optimized way, while minimizing processing overhead and memory transfers.
In this presentation we will review the various ways the industry is addressing this challenge and how Eta Compute solved it with the TENSAI Flow runtime and executors.

source