Attached

Review of “Decadal re-forecasts of glacier climatic mass balance” by van der Laan et al.

(https://doi.org/10.5194/egusphere-2024-387)

This paper presents an analysis of glacier mass balance forecasts on multi-year to decade timeframes, using the mass-balance module of the Open Global Glacier Model. The authors compare forecasts made with climate forcing from Global climate model historical simulations, observation-initialized reforecasts from decadal prediction models, and a simple persistence forecast. Comparing their simulated mass balance re-forecasts to both in-situ and geodetic glacier mass balance observations, they conclude that using the initialized climate reforecasts provides an improvement in skill over GCM-based forcings. The predictability of glacier mass balance on short timeframes is an important problem, and the overall approach of comparing these forcing strategies using the mass balance model and assessing them against observations seems sound. However, there are some significant issues with the clarity of methods and results. It is not clear to me that the authors have demonstrated a meaningful difference in skill between reforecasts and GCM-based forcings. My main comments are detailed below, with some additional minor comments later on.

Major comments:

1) Significance of skill improvements

The principal finding is that reforecasts provide an improvement in skill over using GCM output as forcing, and they emphasize the improvements for decadal mean and cumulative balance (since the skill for yearly forecasts is low; e.g., Fig 1 and line 244). However, there appears to be a huge spread in the Mean Absolute Error (MAE) metrics, such that the 1-sigma ranges substantially overlap. For example, in table 2 for decadal means, the authors report MAE of 0.29 +/- 0.32 mw.e. using reforecasts, and 0.27 +/- 0.31 mw.e. using GCMs. Overlaps are even greater for cumulative balance: 1.33 +/- 3.21 and 1.58 +/- 2.96. Presumably the standard deviations correspond to the distribution of errors across the individual WGMS glaciers. It’s hard to see how this decrease in MAE is significant, given such wide distributions. The authors do not really comment on the wide spread of these error statistics. At the very least, they need to be discussed and the overall conclusions put in the context of these wide distributions.

At a more technical level, there are some other issues with reported statistics that I find puzzling and not well explained.

• To return to the MAE metric in table 2, the +/- range in many cases exceeds the central value reported, implying negative values, which don’t make sense for Mean Absolute Error which should be positive definite. If the standard deviation of a positive-definite distribution is so large, does that imply a very long tail and some very large errors?
• Also in Table 2, what is the Model Error statistic? As far as I can tell this is never explained.
• At line 250 it is stated that the period 2000-2020 gives 11 full decades. This is technically true in a moving-window sense but these 11, 1-decade windows are not statistically independent samples. Why are these reported as different decades? There is a lot of potentially independent information across different individual glaciers, so why is Fig. 2 plotted in terms of these 11 heavily overlapping windows?

Together, these make it hard to interpret the significance of the overall conclusions.

2) ReForecast drift correction

I found the explanation of the reforecast bias correction to be confusing. The authors note that the bias correction is lead-time dependent, but do not really explain why (lines 215-16). This would seem to be an important point to explain thoroughly in order to compare GCM to reforecast-based glacier simulations. In particular, I can’t tell how to interpret the increased skill from reforecasts, in light of the differences in bias correction when using reforecast vs. GCM data to force the model. The reforecast data are bias corrected using different lead-time-based climatologies over 1971--2000. Different lead times aren’t considered for the GCM-driven forecasts, so the GCM data have a single bias correction step using CRU TS data from 1961—1990. Are differences in prediction skill (i.e., simulated vs. observed mass balance) related to different bias correction methods, or the fact that reforecasts start from an observed climate field?  Either would be useful to know about, but the authors don’t address whether the bias correction has an effect. (also – why correct reforecasts over 1971—2000 and GCM data using 1961—1990 means? No explanation is given)

3) Background on reforecasts and sources of skill

I think more background on initialized reforecasts is needed, to help the reader understand (i) the product being used to force the MB model, and (ii) where prediction skill might be coming from (if at all). I completely agree that decade timescales are of applied/operational importance, and this is an area worthy of focus. However, I found it puzzling that there is essentially no discussion of internal climate variability which is the main reason that forecasts on multi-annual to decade timeframes are challenging. The initialized climate models used for decadal forecasts are not summarized to much degree, or differentiated from GCMs, except for the fact that they are “initialized”, but the authors do not really explain what is meant by “initialized”. Again, this is key context when the main result is the relative skill of reforecast vs. GCM-driven mass balance predictions. Some physical reasoning for why an initialized forecast introduces more skill would be important for making sense of the results.

4) Visual examples of reforecasts

Finally, I think a figure showing some examples of the mass balance reforecasts would be helpful for understanding the method and results. The figures are largely aggregated statistics. Picking a glacier as a case study, and showing timeseries (perhaps individual members and ensemble means) of reforecasts under GCM vs. initialized forcing would help the reader immensely in understanding what the errors stats reported later would actually look like in terms of a forecast. It is great to draw on the wealth of WGMS data for validation, but I found myself wondering what these results actually look like for a given glacier. How quickly do the initialized reforecasts decorrelate due to internal variability? How do noise and trends compare? One or two examples would go a long way.

Minor comments

114: What explains the pre-defined range of 50-600 for the melt factor? If outside of this is deemed not “physically realistic”, what is assumed with an order-of-magnitude variation here? 50 kg m^-2 K^-1 seems very low – that’s 0.05 m w.e. K^-1? At least a citation would be useful. Also note typo: K^-1 not K^1.

123: “which represent forecasting from very simple to complex”. Some word is missing. Using simple to complex methods?

135: (and in general) If all of this is in terms of ensemble means, won’t much of the absolute error in comparing observations to reforecasts come from the observation (a single timeseries) having more interannual variability than the ensemble mean? I think it can be valid to focus on ensemble means, but might need to alert the reader to this.

136: For persistence forecasts with multi-year lead times, are the X years just repeated? Or is the mean used for forcing? I was unclear on how this works.

166: Drift correction is mentioned here but hasn’t been described yet, which may confuse a first-time reader.

180: default correction factor – citation?

235: errors in m w.e. – is that per year, or cumulative?

244-45: “inability of a simple mass balance model to reliably simulate individual years”. I don’t think this has to do with the mass balance model… this is the inherent challenge of internal climate variability on these timescales.

280-2: Isn’t lower skill from single model ensembles here partly because the ensemble is smaller? Or are you comparing N=3 ensembles of either one of each, or 3 of the same? Please clarify.

 Fig 4 caption: What is decadal forcing? decadal reforecast forcing?

359: What is “mean cumulative mass balance”? Seems contradictory. Or “mean” as in net annual balance?

365: SSP’s drifting apart over a few to 10 years strikes me as a tiny effect comparted to internal variability and other factors on these timescales. See e.g., Hawkins and Sutton (2009).

370: Climate change increasing the amplitude of natural variability is rather contentious, I would be hesitant to assert it as “likely” here. And I don’t think this is the conclusion of Nijsse et al., 2019 – they are looking at the magnitude of variability across different models with different equilibrium climate sensitivities - which is different than the variability increasing as warming progresses.

Reference

Hawkins, E., & Sutton, R. (2011). The potential to narrow uncertainty in projections of regional precipitation change. Climate dynamics, 37, 407-418.