tinyML Asia 2023 – Sungjoo Yoo: Towards Efficient Neural Rendering

Towards Efficient Neural Rendering
Sungjoo YOO, Director, Neural Processing Research Center (NPRC)

Neural rending, aka NeRF (neural radiance field), is expected to play a crucial role on AR/VR devices like Vision Pro.

The original NeRF method demonstrated the feasibility of novel view synthesis only using multi-view images and their camera poses. However, it also presented new challenges, especially, a computational inefficiency problem that needs to be resolved in order to offer users real-time photorealistic experiences.

In only a few years since the introduction of NeRF, significant (1000x) improvements in computational efficiency have been achieved thanks to tremendous efforts by numerous researchers.

In this talk, we will explain how such a drastic change was possible, mostly via realizing pre-computation on voxel grids or exploiting existing optimized graphics pipeline, and what new challenges are ahead towards true realism needed to make AR/VR devices everyday necessities.

source