Why human forecasting matters: Dissecting sources of TC predictability via multi-modal learning

Abstract

Accurate tropical cyclone (TC) intensity prediction is crucial for disaster preparedness and risk mitigation. While existing approaches primarily rely on single modalities such as satellite imagery or atmospheric measurements, we present MultiCyclone, the first multi-modal deep learning framework that integrates tabular meteorological data, reanalysis wind fields, and textual weather discussions for TC intensity prediction. Our approach leverages 197 tropical cyclones from the Atlantic basin (2005-2020) comprising 3,871 observations to make several key contributions: (1) the first use of textual weather reports for TC intensity prediction through fine-tuned transformer models, (2) multi-modal fusion strategies combining structured and unstructured data, and (3) uncertainty quantification for multi-modal predictions.