Cloud Detection Challenge -Methods and Results

Accurate cloud detection is critical for advancing atmospheric monitoring and meteorological forecasting. This paper presents the Cloud Detection Challenge, an initiative aimed at enhancing cloud detection through innovative solutions using lidar-based ceilometer data. This initiative was hosted by IEEE MetroXRAINE 2024, and 11 teams participated in this initiative. Participants were provided with a novel dataset of backscatter profiles converted into time-height plots, offering unique insights into atmospheric conditions beyond conventional imagery. Data collection employed a Lufft CHM 15k ceilometer, capturing cloud dynamics every 15 seconds located near Mt. Etna, an active volcano in Italy. The dataset includes 1568 hourly labeled backscatter profiles, serving as a benchmark for state-of-the-art deep learning models. The challenge sets a baseline performance of 89.57% accuracy, 92.73% F1-score, 89.82% precision, and 95.84% recall, inviting participants to develop models to exceed these results. Submissions proposed a wide-range of AI-based approaches, including Transformer and Convolutional Neural Network architectures, showcasing the potential of advanced image analysis techniques in lidar-based cloud detection. This paper details the challenge framework, as well as the methodologies proposed by top-performing teams, offering a comparative evaluation of their effectiveness. Our initiative advances cloud detection technologies and underscores their broader implications for environmental monitoring, agriculture, and satellite imaging. The insights and dataset presented herein lay the groundwork for future advancements in leveraging lidar data for atmospheric analysis.