tinyML On Device Learning Forum – Warren Gross: On-Device Learning For Natural Language Processing..

On-Device Learning For Natural Language Processing with BERT
Warren J. GROSS, Professor, McGill University

Resource-constrained devices are increasingly the deployment targets of machine learning applications. Static models, however, do not always suffice for dynamic environments. On-device training of models allows for quick adaptability to new scenarios. With the increasing size of deep neural networks, as noted with the likes of BERT and other natural language processing models, comes increased resource requirements, namely memory, computation, energy, and time. Furthermore, training is far more resource intensive than inference. Resource-constrained on-device learning is thus doubly difficult, especially with large BERT-like models. By reducing the memory usage of fine-tuning, pre-trained BERT models can become efficient enough to fine-tune on resource-constrained devices. In this talk we discuss techniques for fine-tuning BERT models that reduces fine-tuning time and optimizes memory accesses on mobile GPUs, while maintaining accuracy of the deep neural networks.

source