Machine Learning

GenAI on the Edge Forum: Toward a Foundation Model for Efficient Damage Assessment Following Natural



Toward a Foundation Model for Efficient Damage Assessment Following Natural Disasters
Maryam RAHNEMOONFAR, Associate Professor of Computer Science and Engineering, Lehigh University

Natural disasters caused by climate change are becoming more frequent and severe. These disasters pose a threat to human health, infrastructure, and natural systems. In order to respond and recover quickly and effectively after a natural disaster such as a hurricane, wildfire, or flooding, access to aerial images is crucial for the response team. Small Unmanned Aerial Vehicles (UAVs) with cost–effective sensors are a great solution for collecting thousands of images with high flexibility and easy maneuverability for rapid response and recovery. Furthermore, UAVs can access hard–to–reach areas and perform data–gathering tasks that are unsafe or impossible for humans. Combining multiple data modalities such as vision, language, and radar data is a promising technique for damage assessment. However, applying deep learning methods to radar time series images is challenging due to the lack of enough training data compared to optical images. Moreover, optical data can be difficult to use due to their limitations in all weather conditions. Traditional analyses provide some insights into the data, but the complexity, scale, and multimodality nature of the data require advanced, intelligent solutions. In this presentation, I will discuss some of our current innovative solutions such as generative models for multimodal imagery and explainable and interactive models for multimodal vision and language perception. Our goal is to provide an accurate damage assessment with multi–modal data after a natural disaster and facilitate rapid response and recovery. I will also discuss our current efforts toward developing a foundation model that can be transferred to any robot–based multi–modal downstream tasks with very few labeled data.

source

Authorization
*
*
Password generation