A preliminary study on forecast verification and evaluation of the Pangu-Weather model in Shandong
WANG Liang
YANG Chengfang
ZHOU Cheng
GUO Junjian
WEI Haiwen
Abstract：This study conducts a comparative analysis between the forecast products of ECMWF(denoted as EC)and three initial field-driven forecast products,which are based on the ECMWF global model,CMA-GFS and NCEP-GFS of the Pangu-Weather Model(denoted as EC_PG,CMA_PG and NCEP_PG,collectively known as the three Pangu models).The forecast performance of the Pangu-Weather model is evaluated preliminarily against EC for surface and upper-air meteorological elements in Shandong from August to December 2024.The main results are as follows.(1)In the 2-m temperature forecast,EC_PG has the best root mean square error(RMSE)and accuracy(ACC)indexes within the 168-h forecast period.Compared with EC,the three Pangu models can effectively reduce the nighttime high forecast errors and improve the underestimation in the Shandong plains at 09:00 UTC.(2)In the 10-m wind forecast,EC_PG takes the lead in the wind speed and direction scores within the 168-h forecast period.Compared with EC,the three Pangu models can effectively reduce the nighttime high forecast errors of wind speed.The three Pangu models and EC show an overall trend of overestimation of wind speed at night and underestimation during the day.(3)In the forecast of upper-air(500 hPa,700 hPa and 850 hPa)elements,the three Pangu models outperform EC in the specific humidity forecast at each layer within the 72-h forecast period,and CMA_PG performs the best.Among the three Pangu models,EC_PG has the best performance in the temperature and wind speed forecast at each layer.Compared with EC,EC_PG has significantly improved the temperature forecast at each layer at Zhangqiu station,while its wind speed forecast at 850 hPa at Qingdao station is not as good as EC.Overall,among the three Pangu models evaluated,EC_PG has the optimal integrated forecast performance,and its forecast ability is significantly better than that of EC for key meteorological elements such as 2-m temperature,10-m wind speed and specific humidity at key layers(500 hPa,700 hPa and 850 hPa).This study provides initial ideas for systematic and comprehensive verification of artificial intelligence(AI)meteorological models in the future.
Keywords：the Pangu-Weather modelweather forecastverificationShandong
Publication Date：2025-08-30
Online Publishing Date：2026-05-22(First online date of this platform, not the publication date of the document)
Pages：12( 131-142 )