tinyML Talks Pakistan: SuperSlash: Unifying Design Space Exploration and Model Compression…

SuperSlash: Unifying Design Space Exploration and Model Compression methodology for the design of deep learning accelerators for TinyML
Rehan Hafiz
Professor, Faculty of Engineering
Information Technology University (ITU)

Deploying Deep Learning (DL) models on resource-constrained embedded devices is a challenging task. The limited on-chip memory on such devices results in increased off-chip memory access volume, thus limiting the size of DL models that can be efficiently realized in such systems. Sophisticated DSE (Design Space Exploration) schemes have been developed in the past to reduce the off-chip memory access volume. However, DSE alone cannot reduce the amount of off-chip memory accesses beyond a certain point due to the fixed model size. Model compression via pruning can be employed to reduce the size of the model and the associated off-chip memory accesses. However, we found that pruned models with even the same accuracy and model size may require a different number of off-chip memory accesses depending upon the pruning strategy adopted. Furthermore, the classical pruning schemes are not guided by the goals of DSE. In this talk, we discuss SuperSlash, a unified solution for DSE and Model Compression. SuperSlash estimates off-chip memory access volume overhead of each layer of a deep learning model by exploring multiple design candidates. In particular, it evaluates multiple data reuse strategies for each layer, along with the possibility of layer fusion. Layer fusion aims at reducing the off-chip memory access volume by avoiding the intermediate off-chip storage of a layer’s output and directly using it for processing of the subsequent layer. SuperSlash then guides the pruning process via a ranking function, which ranks each layer according to its explored off-chip memory access cost. The talk shall thus present a technique to jointly perform the pruning and DSE to fit in large DNN models on accelerators with low computational resources.

source