A multimodal deep learning approach for very short-term solar forecasts using sky images and historical numerical data

TY - JOUR

T1 - A multimodal deep learning approach for very short-term solar forecasts using sky images and historical numerical data

AU - Jonathan, Anto Leoba

AU - Bamisile, Olusola

AU - Cai, Dongsheng

AU - Ejiyi, Chukwuebuka Joseph

AU - Nkou, Joseph Junior Nkou

AU - Victor, Kombou

AU - Ukwuoma, Chiagoziem C.

AU - Wei, Liu

AU - Huang, Qi

N1 - Publisher Copyright: © 2025 Elsevier Ltd

PY - 2025/6/15

Y1 - 2025/6/15

N2 - The increased solar energy integration into the power system introduces various issues due to its intermittent nature. As solar power penetration increases, grid management becomes increasingly complex, highlighting the importance of precise solar power forecasts. This paper addresses a critical challenge in solar power integration, highlighting the significance of very short-term solar forecasting (VSTSF) for grid operators. We propose a novel MULSKIN hybrid framework that combines a custom ResNet and a novel targeted feature attention mechanism (TFAM), which dynamically focuses on the most relevant regions in sky images, improving the model's ability to capture essential cloud dynamics and spatiotemporal patterns. Unlike existing methods in literature that either use image (sky images) or numerical (meteorological) data, the proposed MULSKIN approach integrates both data types, allowing for a more holistic understanding of atmospheric conditions. This paper contributes to advancing the field of solar forecasting by presenting a new, effective, and scalable solution suitable for operational deployment in modern energy systems. With six years of high-resolution data, the model outperforms traditional machine learning (ML), deep learning (DL) models, and baseline reference models, achieving a forecast skill score of 32.84 % and a root mean square error (RMSE) of 54.36W/m2. Our Results indicate that incorporating sky images alongside numerical meteorological data significantly enhances forecasting accuracy, with the MULSKIN model demonstrating its superiority in capturing dynamic cloud movements and weather patterns over short time horizons. The findings underscore the potential of combining multimodal data sources to address the challenges of VSTSF, offering promising improvements in solar irradiance prediction for real-time grid operations.

AB - The increased solar energy integration into the power system introduces various issues due to its intermittent nature. As solar power penetration increases, grid management becomes increasingly complex, highlighting the importance of precise solar power forecasts. This paper addresses a critical challenge in solar power integration, highlighting the significance of very short-term solar forecasting (VSTSF) for grid operators. We propose a novel MULSKIN hybrid framework that combines a custom ResNet and a novel targeted feature attention mechanism (TFAM), which dynamically focuses on the most relevant regions in sky images, improving the model's ability to capture essential cloud dynamics and spatiotemporal patterns. Unlike existing methods in literature that either use image (sky images) or numerical (meteorological) data, the proposed MULSKIN approach integrates both data types, allowing for a more holistic understanding of atmospheric conditions. This paper contributes to advancing the field of solar forecasting by presenting a new, effective, and scalable solution suitable for operational deployment in modern energy systems. With six years of high-resolution data, the model outperforms traditional machine learning (ML), deep learning (DL) models, and baseline reference models, achieving a forecast skill score of 32.84 % and a root mean square error (RMSE) of 54.36W/m2. Our Results indicate that incorporating sky images alongside numerical meteorological data significantly enhances forecasting accuracy, with the MULSKIN model demonstrating its superiority in capturing dynamic cloud movements and weather patterns over short time horizons. The findings underscore the potential of combining multimodal data sources to address the challenges of VSTSF, offering promising improvements in solar irradiance prediction for real-time grid operations.

KW - MULSKIN hybrid framework

KW - Numerical data

KW - Sky images

KW - Targeted feature attention mechanism

KW - Very short-Term solar forecasting

UR - https://www.scopus.com/pages/publications/105008570605

U2 - 10.1016/j.renene.2025.123774

DO - 10.1016/j.renene.2025.123774

M3 - Article

AN - SCOPUS:105008570605

SN - 0960-1481

VL - 255

JO - Renewable Energy

JF - Renewable Energy

M1 - 123774

ER -