Snow plays a crucial role for the surface energy budget of the hydrological and climate systems, making it an important component in climate simulations and long-range weather forecasts. The amount of water stored in a snowpack, known as snow water equivalent (SWE), is a key parameter for flood risk assessment, water resource management, and, more broadly, land-atmosphere coupling and climate modeling. In the warming world, snow conditions are undergoing substantial changes. The global snow cover extent has been shown to decline over the last decades in both models and observations and the length of the snow season has been shown to diminish in mountain regions . Snow droughts, either driven by deficits in snowy precipitation or by premature snowmelts, have been linked to hydrological extremes . On local scales, a reduction of the fraction of solid to liquid precipitation in mountain regions has also been linked to rainfall extremes and associated hazards .

Given the changes in snow accumulation, melt, and associated hydrological hazards, reliable seasonal predictions of snow conditions are becoming increasingly important. Seasonal snow forecasts are routinely provided by operational meteorological prediction centres, using state-of-the-art ensemble dynamical prediction systems, based on coupled atmosphere-ocean models. In such forecasts, snow variables are initialized as realistically as possible through data assimilation (DA) methods of varying complexity, which ingest in-situ and space-borne observations. There are currently considerable efforts to improve the fidelity and reliability of such seasonal forecasts, as well as the initialization of land variables, including snow.

Realistic initial snow conditions can improve forecast skill not only for snow itself but also for other surface and atmospheric variables through its high albedo and low thermal conductivity, as well as its delayed influence on soil moisture . Improved forecasts of snow variables may also enhance the representation of large-scale atmospheric circulation in regions with strong snow–atmosphere coupling, such as East Asia . However, seasonal forecasting models have limitations, such as various parametrizations or their relatively coarse horizontal resolution which does not allow resolving snow heterogeneities. Similarly, data ingested through DA have limitations, such as the sparsity and non-representativeness of in-situ station data, or the observational constraints and errors of remotely sensed data. In addition, DA approaches used in prediction centres have also limitations e.g., assimilation might not be performed in mountain regions.

While seasonal forecast systems are often evaluated in terms of skill e.g. and their ability to capture large-scale modes of variability e.g., an equally important aspect of forecast quality is their statistical reliability, i.e. the consistency between forecast probabilities and observed frequencies. Reliability diagrams, used to illustrate this consistency, are among the diagnostic measures incorporated in the World Meteorological Organization Standardized Verification System for Long-Range Forecasts . Reliability of seasonal-mean near-surface temperature and precipitation forecasted by the European Centre for Medium-Range Weather Forecasts (ECMWF) Integrated Forecasting System (IFS) 4 was examined in . Building on the same methodology, quantified changes in near-surface temperature and precipitation in the current operational ECMWF seasonal forecasting system, SEAS5, over System 4. Reliability assessment has also been used as a validation approach for model bias correction and downscaling methods . However, such assessment has never been performed for probabilistic snow forecasts.

The aim of this study is to fill this gap and to address the following question: How reliable are seasonal forecasts of snow? To that end, the reliability assessment is applied to seasonal snow hindcasts produced by the ECMWF in the framework of the CopERnIcus climate change Service Evolution (CERISE) project (see acknowledgments and https://cerise-project.eu/, last access: 19 August 2026), which is aimed at enhancing the quality of the Copernicus Climate Change Service (C3S) reanalysis and seasonal forecast products by means of improved land-atmosphere data assimilation and land surface initialization. The estimated reliability of probabilistic snow forecasts will, naturally, depend on the snow product they are assessed against. Numerous snow products are now available to meet the growing demand for historical snow datasets in climate, hydrology and cryosphere research. Recent study by compared twenty three historical gridded snow products against a combination of in-situ snow course and airborne gamma-ray based measurements. Based on the overall assessment, the best performing product was shown to be the ERA5-Land reanalysis albeit it is an off-line surface model run which does not assimilate snow measurements. However, for non-mountainous Eurasia, the European Space Agency (ESA) Climate Change Initiative (CCI) Snow project (hereafter ESA Snow-CCI) product, combining space-born observations of passive microwave brightness temperatures and in-situ measurements of snow depth, scored second-best. Therefore, in this study the winter-mean ECMWF hindcasts of snow water equivalent are evaluated against two products, namely, the ERA5-Land reanalysis and ESA Snow-CCI version 4. The reliability assessment is performed over the years 1993–2022 in eight land regions in the Northern Hemisphere excluding mountainous regions.

The paper is organized as following. An introduction into the basics of the reliability assessment is given in Sect. . All datasets used for the analysis are described in Sect. –. The results are presented in Sect.  and discussed in Sect. . The conclusions are drawn in Sect. .

This section summarizes the reliability of ECMWF snow hindcasts in DJF 1993–2022 for low- and high snow accumulation winters as shown in Fig. . Figure a shows that assessment against ERA5-Land for low snow accumulation winters yields one region with perfect, five regions with useful, and two regions with marginally useful improved reliability categories. The results are substantially different when verified against ESA Snow-CCI (Fig. b). In this case, snow hindcasts are assigned marginally useful and marginally useful improved categories in all eight land regions. When verified against ERA5-Land in high snow accumulation winters, snow hindcasts are assigned perfect reliability category in two land regions, useful category in five land regions, and marginally useful improved category in one region (Fig. c). Verification against ESA Snow-CCI yields a different picture – only one region is assigned useful reliability category while other seven regions are assigned either marginally useful or marginally useful improved category, see Fig. d. While some regional variability in reliability categories is present, the overall hindcast performance is considered good, as only marginally useful to perfect categories (Categories 3–5) are obtained for both verification datasets and tercile events. This is not the case for seasonal forecasts of near surface temperature and precipitation as shown in previous studies .

Individual reliability diagrams behind the color-coded categories in Fig.  are shown in Figs.  and for low- and high snow accumulation winters correspondingly. An important aspect of the forecast system seen in these figures is sharpness (see Sect. ). While the ECMWF hindcasts cover a wide range of probabilities, their sharpness is rather limited. ALA, CNAS and ENAS regions are characterized by a strong concentration of forecasts in the lower probability bin (around 38 %–47 % of all predictions in a region), while the occurrence of high-probability forecasts remains modest (around 5 %–13 %). A more balanced distribution between low (29 %–34 %) and near-climatological (34 %–38 %) probabilities is typical for CND and WNAS regions, suggesting weaker forecast confidence and a tendency toward less decisive probability assignments. Finally, three regions are dominated by forecasts close to the climatological mean, indicating limited sharpness. Those are WNA, NEU, and TIB, where the majority of predictions (44 %–58 %) cluster around the climatological mean. The above described patterns show that while the forecast system is capable of issuing a range of probabilities, it frequently favors low or near-climatological probabilities, with comparatively few predictions with high probabilities.

Another important feature seen in Figs.  and is the tendency of the ECMWF hindcasts to be over-confident i.e., to issue probabilities that are too extreme (too low for low probability bins and too high for high probability bins). This is reflected by reliability lines with slopes flatter than 45°, although the exact slope depends on the region and verification dataset. Nevertheless, the reliability categories obtained are always marginally useful to prefect, independent of the verification dataset. While reliability categories are generally one category lower when assessed against ESA Snow-CCI, two regions stand out. In CND region, the hindcasts show excellent performance when assessed against ERA5-Land while only marginally useful when compared with ESA Snow-CCI. In NEU region, reliability assessment always yields marginally useful improved category independent of the verification data and binary event.

In this study, the binary event definition is based on tercile thresholds that corrects biases and implies that the best-fit reliability line always passes through the climatological intersection (fk=o‾, o‾k=o‾). However, a clear off-set of the best-fit reliability line is present in TIB and WNA regions when compared with ESA Snow-CCI (Figs. b and b), and in TIB region when compared with ERA5-Land (Figs. a and a), indicating a tendency toward over-forecasting. This behavior is driven by grid points where SWE in the verification data remains zero throughout the 30-year period (snow-free land), while the hindcasts predict a variable amount of snow. As a result, these points contribute only to “no-event” outcomes across multiple forecast probability bins, leading to the observed offset in the reliability line. While this offset is evident in both TIB and WNA regions, it is considerably more pronounced in TIB due to a larger number of grid points with snow free conditions in the observations but not in the hindcasts (see Fig. ). On that figure, a snow-free area south-west of the Himalayan arc can be seen in both ERA5-Land and ESA Snow-CCI. Incidentally, it is also present in regional snow re-analyses, such as the High Mountain Asia Snow Reanalysis . However, the snow-free conditions appear farther south in the hindcasts, indicating a bias in the snow transition line in the ECMWF model. The biases between ERA5, ERA5-Land, in-situ and satellite observations over the Tibetan Plateau (encompassed by the region labeled TIB here) are documented by .

It is important to note that the results shown in panels (a) and (b) of Figs.  and are based on the same set of hindcasts. The differences between the reliability categories therefore reflect differences between the verification datasets. To better understand these differences, Fig. a shows probability density distributions of SWE anomalies in the ECMWF hindcasts, ERA5-Land and ESA Snow-CCI for different land regions. Despite similarities in the probability density distributions, ERA5-Land SWE anomalies are closer to those in the hindcasts than ESA Snow-CCI anomalies. One can also see that anomalies in all datasets and regions are right-skewed. The skewness is similar among datasets in WNA, NEU, WNAS and TIB regions, while differences are more pronounced in other regions. Differences in the skewness of the verification datasets imply that the lower and upper terciles represent different parts of the respective distributions. Time series of snow anomalies in Fig. b also show discrepancies in the temporal evolution of SWE anomalies and in the distribution of years with strong positive and negative anomalies. The latter is important for the definition of probabilistic tercile thresholds used in the reliability assessment, as explained in Sect. . As a result, the same forecast probabilities may correspond to different extreme events and therefore different observed frequencies in the verification data, leading to systematic differences in the assessed reliability.

To support this interpretation, we compute a Continuous Ranked Probability Score (CRPS) which is a continuous probabilistic verification metric often used for the evaluation of forecast systems . It represents the accumulated Brier score (see Sect. ) across all possible threshold values, with values closer to zero indicating better forecast performance. The CRPS is computed for the ECMWF hindcasts with respect to the ERA5-Land and ESA Snow-CCI verification datasets. In addition, we compute Spearman's rank correlation coefficient and the agreement in binary event classification between the two verification datasets. The former is calculated at each grid point and measures the consistency in the ranking of years with SWE anomalies between the two verification datasets. The latter is defined as the percentage of data points for which the tercile-based binary event is classified consistently in both verification datasets at the same grid point and time. Since the binary event classification is based on tercile thresholds derived from the SWE anomaly distributions, differences in the ordering of years between the verification datasets can lead to different binary event classifications for the same location and time. Therefore, regions with higher Spearman's rank correlation are expected to show higher agreement in the binary event classification, reflecting a greater consistency between the verification datasets.

Figure  shows values of the CRPS and Spearman's rank correlation coefficient averaged within the eight regions from Table  together with the agreement metric computed by pooling all grid points and years within each region. Although the reliability categories shown in Fig.  differ for the two verification datasets, their CRPS values are similar (Fig. a). This can be explained by the fact that CRPS integrates over the full predictive distribution and is relatively robust to moderate differences between verification datasets. At the same time, relatively low rank correlation and only 70 %–80 % agreement in the tercile-based binary event classification (Fig. b) indicate that the two verification datasets do not classify the same years consistently as low and high snow accumulation winters. Since reliability in this study is evaluated for tercile-based binary events, the forecast system is verified against different realizations of the binary event depending on the observational reference. Consequently, forecast probabilities may appear more reliable with respect to one verification dataset than the other. The highest values of the Spearman's correlation and the agreement metric are found in regions WNA, NEU and WNAS, where differences in reliability categories are smallest. The lowest values are found in CND region which is also associated with the largest difference in CRPS values (Fig. a) and reliability categories (Fig. ). We therefore conclude that the differences in reliability categories shown in Figs. – are primarily due to differences in binary event classification between the verification datasets used in this study.

No detrending was applied to SWE anomalies prior to the reliability assessment, as this study aims to evaluate the forecast system in a real-world context, including any long-term systematic changes present in both the hindcasts and the verification datasets. However, to ensure the robustness of the results obtained, a sensitivity test was performed in which SWE anomalies were detrended at each grid point prior to the reliability assessment. For the reliability assessment against ERA5-Land, only TIB region changed category from useful to marginally useful. However, this region has already been identified as particularly challenging. All other regions retained the same categories as in Fig. a and c. For ESA Snow-CCI, detrending led to some regional changes in reliability categories. However, the categories remain within marginally useful to marginally useful improved, except for WNAS region during high-snow-accumulation winters same as in Fig. b and d. This indicates that the reported reliability of the ECMWF seasonal snow hindcasts is robust with respect to the presence or absence of long-term trends in the data.

Figure  demonstrates that reliability assessment and categorization are somewhat dependent on the choice of verification dataset. However, such behavior is not limited to snow variables only. A similar discrepancy can be seen in the reliability assessment of near surface temperature and precipitation in works by and , where ERA-Interim reanalysis for temperature and GPCP analysis for precipitation in the former versus EWEMBI dataset for both variables in the latter were used.

Although it may seem intuitive to place greater trust in Earth observation products, these datasets have notable limitations. It is known that ESA Snow-CCI has a systematic bias in estimates of large SWE values (>15 cm) due to the reduced sensitivity of passive-microwave retrievals to deep snow and liquid water in the snowpack . In addition, the mismatch between point-scale in-situ measurements being assimilated and the SWE grid-scale estimates adds uncertainty into the final product, with retrieval performance declining farther from assimilated snow depth observations . A recent study by suggests bias correction for ESA Snow-CCI SWE retrievals that could be used in the future assessments. At the same time, ERA5-Land, while generally robust , is known to overestimate SWE, especially in mountainous areas, based on global and regional studies .

Regional variations in reliability are also evident and could be linked to, for example, regional differences in snow distribution as well as technical limitations in both the forecast model (e.g., dependence on the empirical snow cover-to-snow depth conversion rule) and the verification datasets (e.g., the aforementioned ESA Snow-CCI bias for large SWE values). Reliability categories obtained when verified against ESA Snow-CCI are slightly higher over Eurasia than over North America. This may reflect the well-documented better performance of this product in Eurasia compared to North America , likely linked to the wider network of observation stations used for SWE retrievals.

As noted in Sect. , numerous gridded snow products are currently available, e.g., outputs from coupled reanalysis systems, standalone land model simulations driven by meteorological forcings, satellite-derived snow estimates or actual snow observations. However, those products often show substantial discrepancies in both magnitude and spatial–temporal variability of snow, as well as in long-term snow trends see e.g.,. When reanalysis is used for verification, an issue might be that it has the same origin as the one used to initialize the forecast which may lead to higher reliability categories. On the other hand, the use of satellite-derived snow products for verification may lead to an underestimation of reliability categories due to higher uncertainty in observations and retrieval methods. The inconsistencies between existing snow datasets therefore emphasize the need for continued efforts to develop a high-quality, internally consistent snow dataset specifically suited for the forecast verification purposes. Such dataset should, ideally, (i) be independent of the model and data assimilation system used to initialize the hindcasts, (ii) provide consistent observations over a long period, (iii) have good spatial and temporal coverage, and (iv) have known errors, biases and limitations.

Data assimilation plays an important role in constraining the forecast initial snow to observations. The type of assimilated data and the assimilation method are, therefore, of a key importance. In the hindcasts used here, both land and atmosphere initial conditions are derived from ERA5 reanalysis. Since the off-line ERA5-Land reanalysis uses ERA5 as atmospheric forcing, higher reliability categories obtained when verified against ERA5-Land may partly reflect the shared dependence on ERA5. Consistent with this interpretation, similarly high categories were found when the hindcasts were verified against ERA5 itself (not shown). The results should, therefore, be interpreted as conditional on this shared ERA5 forcing dependence, rather than as a standalone measure of forecast reliability.

Avenues for future work include, but are not limited to, a more detailed assessment of seasonal SWE forecast reliability as a function of lead time, for example, through monthly assessments throughout the winter season. In addition, exploring prediction of extreme snow events, like the snow droughts mentioned in Sect. , would be of a great interest as they have been linked to hydrological extremes. It is also important to investigate the reliability of SWE forecasts in mountainous areas in spring, when snowmelt occurs with potential implications for hydropower management and flood-risk assessment. Finally, a multi-model comparison of seasonal SWE forecasts produced by operational meteorological prediction centers would provide additional insight into model performance and reliability.

In this study, we assess reliability of winter-mean snow hindcasts in 1993–2022 produced by the ECMWF within the CERISE project. In probabilistic forecasting, reliability for a binary event is defined as the consistency between forecast probabilities and observed frequencies. The assessment is performed against two SWE datasets: ERA5-Land reanalysis and ESA Snow-CCI (version 4) in eight land regions in the Northern Hemisphere excluding mountainous areas. To evaluate the forecast performance in low and high snow accumulation winters, we consider two binary events based on terciles of the long-term distribution: (i) winter-mean SWE anomaly lies below the lower tercile, and (ii) winter-mean SWE anomaly lies above the upper tercile.

A simple categorization is used to quantify the reliability based on the weighted linear regression to the data in reliability diagrams. Unlike seasonal forecasts of near surface temperature and precipitation evaluated in previous studies which occasionally found low reliability categories, SWE hindcasts yield only marginally useful to perfect reliability categories. Although the reliability assessment performed is sensitive to the choice of verification dataset and yields slightly different reliability categories when verified against ERA5-Land and ESA Snow-CCI. This sensitivity highlights the need for continued efforts to develop a high-quality, internally consistent snow dataset suited for the forecast verification purposes.