tinyML Summit 2022: Tiny models with big appetites: Cultivating the perfect data diet

tinyML Summit 2022
tinyML Vision Session
Tiny models with big appetites: Cultivating the perfect data diet
Jelmer NEEVEN, Deep learning scientist and software engineer, Plumerai

Although lots of research effort goes into developing small model architectures for computer vision, real gains cannot be made without focusing on the data pipeline. Production-worthy computer vision models need large quantities of training data, even when the models themselves are tiny. But since tiny models are eager to take shortcuts that don’t generalize in practice, we can’t tolerate low-quality data. In this talk, we cover the wide variety of techniques we use for curating optimal training datasets and designing better data sampling strategies. We also show the importance of measuring model robustness in diverse real-world environments. All of this is made possible by Plumerai’s in-house data infrastructure and tooling, built specifically for producing tiny computer vision models.

source