ACCESS-S2 is the BoM's seasonal forecast system which became operational in October 2021, replacing the ACCESS-S1 system . The model components of both ACCESS-S2 and ACCESS-S1 are identical with the same numbers of levels and resolution. They consist of the Global Atmosphere 6.0 (GA6) , the Unified Model's Global Land 6.0 , NEMO Global Ocean 5.0 and Global Sea Ice 6.0 (CICE; ). The atmosphere has a N216 horizontal resolution (∼60 km in the mid-latitudes) with 85 vertical levels. The land model uses the same horizontal grid as the atmosphere with four soil levels. The ocean component has a nominal horizontal resolution of 1/4° with 75 vertical levels. The sea-ice component, based on CICE version 4.1, has the same resolution as the ocean component and five sea-ice thickness categories as well as an open water category.

Both systems take atmospheric initial conditions derived from ERA-interim for their hindcasts. The only difference between the hindcasts of the two systems are the ocean and sea-ice initial conditions and the ensemble generation procedure.

ACCESS-S1's ocean and sea-ice initial conditions come from the Met Office FOAM system, which uses a multivariate, incremental three-dimensional variational (3D-Var), first-guess-at-appropriate-time (FGAT) data assimilation scheme and assimilates sea surface temperature (SST), sea surface height (SSH), in situ temperature and salinity profiles, and satellite observations of sea-ice concentration using the EUMETSAT OSISAF product described in the next section. If sea-ice concentration innovations are positive, these are added to the first ice category with a fixed thickness of 50 cm. If they are negative, they are removed from the thinnest category first and from thicker categories if needed. Therefore, although this system only assimilates sea-ice concentration data, sea-ice thickness is modified.

ACCESS-S2, on the other hand, is initialised from ocean conditions generated by the BoM weakly coupled ensemble data assimilation scheme described in . This scheme uses an optimal interpolation method and assimilates temperature and salinity profiles from EN4 . SSTs are nudged to Reynolds OISSTv2.1 in areas where SSTs are over 0 °C and sea surface salinity is weakly nudged to the World Ocean Atlas 2013 climatology .

Of most relevance for this work, sea-ice concentrations are not assimilated in ACCESS-S2. Assimilation cycles are performed daily. The coupled model runs for 24 h initialised from the previous cycle. Then the restart file fields of the ocean component are used as first guess in the data assimilation cycle and the innovations are used to build the next ocean initial conditions for the following cycle. The atmosphere fields from that daily integration are not used and instead the model atmosphere is initialised using ERA-Interim. The sea-ice initial conditions for the next cycle are the unaltered output of the previous daily integration. Then the cycle starts again and the coupled model runs for another 24 h. During this integration the sea-ice component is affected by the ocean innovations and the new atmosphere initial conditions via the coupler.

The ACCESS-S1 hindcast set is made up of nine members created by perturbing the atmospheric fields only with a random field perturbation and runs for 217 d for the period 1990–2012 initialised at the first of every month. The ACCESS-S2 hindcast set used in this study runs for the period 1981–2018. Ensemble members are created in the same manner as ACCESS-S1 members; however, due to computing cost limitations, only three members per forecast initialisation date were run for 279 d. Larger ensembles were generated by aggregating three-member ensembles initialised on successive days . Here, we build a nine-member time-lagged ensemble from three consecutive three-member forecasts initialised at the first of every month and the two previous days and run for 279 d. We analyse the ensemble mean hindcasts unless otherwise specified and, although the hindcast periods differ for each forecasting system, we only use data from the overlapping period (1990–2012) to compute error measures, this excludes the recent period of low sea-ice extent for our analysis.

Anomalies for each hindcast set are taken with respect to their own climatology specific to each initialisation date and forecast lead time, for the period 1990–2012. This serves as a first-order correction of model bias and drift. For monthly means, we define “0 lead time months” as the monthly mean forecasts immediately after the initialisation.

Besides sea-ice concentration, we also analyse mean sea-ice thickness, which we compute as total sea-ice volume divided by total sea-ice area.

For verification we use satellite-derived sea-ice concentration, which estimates the proportion of each grid area that is covered with ice. Different datasets are derived using different algorithms and satellite platforms, and each have their own biases and uncertainties. Estimates of inter-product uncertainty of sea-ice extent (defined as the total region of the Southern Ocean with at least 15 % sea-ice cover, ) are of the order of 0.5×106 km2 . As will be shown below, this spread is minimal compared with the typical errors in the ACCESS-S2 and ACCESS-S1 forecasts, so the overall conclusions of this study are independent of the verification dataset used.

We use NOAA/NSIDC's Climate Data Record V4 (CDR; ) as the primary sea-ice verification dataset. It takes the maximum value of the NASA Team and NASA Bootstrap sea-ice concentration products to reduce their low concentration bias . Both source algorithms use data from the Scanning Multichannel Microwave Radiometer (SMMR) on the Nimbus-7 satellite and from the Special Sensor Microwave/Imager (SSM/I) sensors on the Defense Meteorological Satellite Program's (DMSP) -F8, -F11, and -F13 satellites. The data have a spatial resolution of 25 by 25 km and daily from November 1978 onwards.

The European Organisation for the Exploitation of Meteorological Satellites (EUMETSAT) Ocean and Sea Ice Satellite Application Facility (OSI; ) based on the SSMIS sensor is another satellite-derived sea-ice concentration product. It is based on mostly the same sensors as the NOAA CDR but computed independently using different algorithms. Figures prepared with this dataset are provided in the appendix and do not differ significantly from the ones prepared using CDR.

For evaluation purposes, we use a series of measures. We evaluate sea-ice extent anomaly hindcasts by their Root Mean Squared Error (RMSE) and correlation coefficient. The hindcast ensemble mean sea-ice extent is defined as the mean sea-ice extent of individual ensemble members.

Pan-Antarctic (net) sea-ice extent serves as a hemispheric measure of the amount of sea ice, but it does not take into account the spatial distribution. A model could have a relatively accurate extent of the net ice but with different regional distributions. To account for location errors, we computed the RMSE of grid-point sea-ice concentration anomalies.

We compute RMSE as the square root of the area-averaged squared differences between grid-point forecasted and observed sea-ice concentration anomalies. We compute a pan-Antarctic RMSE by averaging over the whole NOAA/NSIDC CDRV4 Southern Hemisphere domain, and also a zonally-varying RMSE computed over 15 longitude slices 24° wide around Antarctica.

All error measures were computed on the NOAA/NSIDC CDRV4 domain grid, to which model output was bilinearly interpolated. Note that the ACCESS CICE model grid has resolution between two and three times higher than NOAA/NSIDC CDRV4.

Forecast errors are also compared with hypothetical forecasts based on the persistence of anomalies and on climatology. The persistence forecast is generated by extending the observed sea-ice concentration anomalies from the day of the forecast initialisation and comparing it with the actual anomalies observed. The climatological forecast error is computed as the standard deviation of daily anomalies.

As a measure of forecast improvement over the hypothetical forecast, we use the skill score , defined as: 1S=1-RMSEfRMSEr where RMSE_f is the RMSE of the forecast, RMSE_r is the RMSE of the reference forecast. Negative skill score indicates that the forecast is worse than the reference forecast while positive values indicate an improvement. A perfect forecast would have zero RMSE and thus a skill score of 1. Furthermore, we also compute sea-ice concentration bias, defined as the mean difference between the forecast and observations.

We performed all analyses in this paper using the R programming language , using data.table and metR packages. Significant processing was performed using the CDO command line operators . All graphics are made using ggplot2 . The paper was rendered using knitr and Quarto .

Figure  (Fig.  for OSI) shows mean sea-ice extent of the ACCESS-S1 and ACCESS-S2 hindcasts (row a) and their differences from mean sea-ice extent of NOAA/NSIDC CDRV4 (row b). Mean extent at the first of every month is indicated with circles for the initial conditions and with triangles for the leadtime corresponding to the longest lead time possible for ACCESS-S1 (between 213 and 216). At this long lead time, information of the initial conditions is essentially lost and the forecast reverts close to each model's preferred equilibrium state.

ACCESS-S2 initial conditions (circles in Fig.  column 2) show an overall negative bias, especially between late summer and early winter, while ACCESS-S1 initial conditions (circles in Fig.  column 1) are very close to observations, as expected from the assimilation of sea-ice observations to produce the initial conditions of ACCESS-S1. Both systems' equilibrium states (triangles) show negative biases of sea-ice extent, particularly in the growth phase of late-autumn and winter months. This is due primarily to the melt season being longer in the model than in observations and with faster melt between January and March and the growing seasons being shorter with slower growth during March and April. This is then followed by faster growth between May and July (Figs.  and ). Many sea-ice models exhibit this systematic underestimation during the sea-ice minimum and early freezing season , which could indicate problems in the representation of thermodynamics in the model . It is also not surprising that both forecasting systems converge to a similar equilibrium state because they share the same model formulation.

The difference between the initial conditions (circles) and the model equilibrium state (triangles) can be mostly attributed to the effect of data assimilation, which in ACCESS-S2 is due solely to the coupling of sea ice with the atmosphere and the ocean. From April to September, in ACCESS-S2, circles are closer to observations than the triangles are, indicating that the information from the ocean and atmosphere data assimilation is affecting sea ice and improving the initial conditions. During these months, ACCESS-S1 can overestimate the sea-ice extent at short lead time. For the rest of the year circles are overlaid with triangles in ACCESS-S2, indicating that the indirect effect of the ocean and atmosphere data assimilation on sea ice is minimal and that this component of the model is virtually free-running.

To further understand the bias in ACCESS-S2, Fig.  (Fig. ) shows spatial patterns of the differences of monthly mean sea-ice concentrations between NOAA/NSIDC CDRV4 and ACCESS-S2 hindcasts at the shortest monthly lead time. From October to May, the model underestimates sea-ice concentrations in most regions except for the inner Weddell Sea in April and May, where sea-ice concentrations saturate to 1 both in the observations and forecasts. In winter, the differences are limited to a narrow band around the sea-ice edge with slight positive biases in the African sector of East Antarctica and negative biases around the Indian Ocean sector which partially compensate, resulting in the near-zero extent bias seen in those months (Fig. ).

ACCESS-S1 has a comparatively smaller overall bias (Figs.  and ). The largest values are found between April and June, when the faster growth results in large positive bias along the sea-ice edge, and in January, when the faster melt leads to large negative bias in the Weddell and Amundsen-Bellingshausen Seas.

Figure  (Fig. ) shows monthly sea-ice extent anomalies forecasted at selected lead times. Compared with ACCESS-S1, ACCESS-S2 anomaly forecasts are relatively poor (large RMSE) even for the first month (lead time 0), which is worse than ACCESS-S1 forecasts even at a lead time of six months. ACCESS-S2 shows much larger interannual variability than observations, with dramatic lows between 1995 and 2007, and highs between 2007 and 2015.

Unexpectedly, for ACCESS-S2, RMSE improves with lead time, even though the correlation degrades with lead time. This effect is seen in all months except from July to September (Fig. ). This is puzzling behaviour that goes contrary to what is usually seen in prediction models. The explanation seems to be the aforementioned increase in interannual variability. Figure  (Fig. ) shows the interannual standard deviation of monthly sea-ice extent of the forecasts as a function of lead time compared with observations. ACCESS-S1 standard deviation lies within the observed standard deviation regardless of lead time, while ACCESS-S2 standard deviation is more than twice that of observations at zero lead time and only approaches the observed value at nine months lead time for most months.

The reason for the increased variability in ACCESS-S2 is not clear. One possible explanation is that ACCESS-S2 simulates thinner sea ice than ACCESS-S1 (Fig.  bottom row) that is more sensitive to atmospheric and oceanic forcing. To explore this possibility, Fig.  shows the correlation between sea-ice thickness and the magnitude of sea-ice concentration anomalies in the ACCESS-S2 reanalysis. It shows that in most regions and months, thinner ice is correlated with stronger sea-ice concentration anomalies.

Figure  shows that indeed ACCESS-S2 simulates thinner sea ice compared to ACCESS-S1 overall at almost all lead times and in all months except for summer at short lead times (December–January, 0–1 months; February–March, 0–2 months). However, Fig.  also shows that sea-ice thickness tends to decrease at longer lead times, when ACCESS-S2's variability converges to observed variability. Furthermore, the thickness difference between ACCESS-S1 and ACCESS-S2 is large also in months in which ACCESS-S2 sea-ice extent variability is comparable to ACCESS-S1 and where ACCESS-S2's RMSE does not decrease with lead time, such as August and September.

Even if ACCESS-S2's sea ice is more sensitive to atmospheric and oceanic forcings than ACCESS-S1 by being overall thinner, this is not a sufficient explanation for the pattern of increased variability at shorter lead times and its absence in particular months. Instead, the strengths of the atmospheric and oceanic forcings might also be playing a role. The fact that ACCESS-S1 and ACCESS-S2 share the same model configuration and that the increased variability is more extreme at short lead times (Fig. ) suggests that the data assimilation procedure might also be partly responsible. We speculate that it is possible that sea ice in the ACCESS-S2 system is left in an unbalanced state by assimilating atmospheric and oceanic data but not sea-ice data, leading to large responses that are amplified by the thinner ice. Teasing this out would require an in-depth analysis of the data assimilation innovations, which is out of scope for this work.

To assess ACCESS-S2 forecasts in more detail, we compute error measures for all hindcasts started on the first of every month. Figure  (Fig. ) shows the mean RMSE of sea-ice concentration anomalies for ACCESS-S1 and ACCESS-S2 hindcasts compared against persistence and climatological forecasts used as a benchmark. Due to errors in the initial conditions, it is expected that persistence forecasts would be better than the model forecasts at very short lead times, but that the persistence forecast errors would grow faster and may eventually surpass the model forecast errors. The black line shows that the persistence forecast error indeed grows rapidly and reaches its maximum in about 30 d for most months except for February, when it grows much slower. The ACCESS-S1 forecast errors grow slower than persistence forecast errors and remain lower after less than 10 d on average. The ACCESS-S2 forecast error starts high in all months and is lower than the persistence forecast error after more than 15 d in most months except for forecast initialised in February, when it takes 80 d.

At longer lead times, it is more appropriate to compare errors with the climatological forecast error. The lead time at which ACCESS-S1 forecast error is higher than the climatological forecast error varies between more than 60 d and less than 20 d depending on forecast initialisation month with the minimum length in June. ACCESS-S2 forecasts never have lower error than climatology, on the other hand, except marginally in October forecasts.

In summary, ACCESS-S1 forecasts have a longer lead time in which it is skillful in the summer than the other seasons. Forecasts initialised between May and July are particularly poor, and June cannot be forecasted better than the benchmarks, suggesting a winter barrier in predictive skill (Fig. ). This is similar to the mid-winter loss of predictability observed by for the Weddell Sea, who attributed it to deep warm water entraining into the mixed layer.

We quantify the skill difference between ACCESS-S1 and ACCESS-S2 in Fig.  by computing the shortest lead time in which the mean RMSE of each system's forecast are not statistically different. The median difference is around 35 d but the biggest difference is between February and April, where ACCESS-S1 forecasts have lower RMSE than ACCESS-S2's for more between 65 and 125 d. The smallest difference, on the other hand, is in winter, when ACCESS-S1's forecast are statistically indistinguishable from ACCESS-S2's after less than 20 d. Assimilation of sea-ice concentration data seems to have a much greater effect in the late summer than in the winter.

To analyse the spatial distribution of the model error, we computed the RMSE of zonal mean sea-ice concentration anomalies on 15 slices of 24° longitude span for each forecasting system. We control for some areas being naturally easier to forecast than others by computing the RMSE skill score with the climatological forecast RMSE as reference.

For ACCESS-S1 forecasts (Figs.  and ), skill tends to be lower off the coast of East Antarctica even at short lead times. This is particularly important for forecasts initialised between May and July, where we see that the low skill in early winter (Figs.  and ) is only a feature in East Antarctica where skill is even negative at almost zero lead time. ACCESS-S1 forecasts have relatively high predictive skill in the Weddell Sea in winter, consistent with other seasonal prediction systems . July forecasts have positive (albeit not statistically significant) skill with hindcasts initialised as early as February. The winter predictive skill barrier thus seems to come in great part from large errors in East Antarctica.

In West Antarctica there is a hint of easterly-propagating skill in forecasts initialised in February and March. This is consistent with , who found that the memory of sea-ice anomalies is stored in ocean heat content anomalies that are transported eastwards by the Antarctic Circumpolar Current.

ACCESS-S2 forecasts (Figs.  and ) also have lower skill over East Antarctica. From July to December, even though the pan-Antarctic average skill is negative at all lead times (Fig. ), it is positive (but not statistically significant) for up to a month in West Antarctic. Since oceanic and atmospheric forcings provide the only source of observational information, this suggests that sea ice in this region is particularly sensitive to oceanic and atmospheric forcing. The fact that this is evident in the months in which El Niño–Southern Oscillation teleconnections via the Pacific-South American mode and the Amundsen Sea Low are more important for atmospheric circulation also suggests the influence of tropical Pacific variability. February and March are the only two months that can be forecasted with marginally positive skill in large regions.

Finally, Fig.  (Fig. ) shows the difference in skill between ACCESS-S1 and ACCESS-S2. Large differences in skill indicate areas and months that are most affected by the data assimilation present in ACCESS-S1. Between January and March, when ACCESS-S1 is the most skillful (Fig. ), most of the higher skill compared with ACCESS-S2 is present in the Ross and Weddell Seas. In April and May, the higher skill seems more homogeneous. In June and July, ACCESS-S1 has higher skill only at very short lead times and in limited regions; in particular we see that in June there is no skill difference in the King Haakon Sea – between 0 and 90° E.

This analysis rests on the assumption that the main difference between ACCESS-S1 and ACCESS-S2 is the lack of data assimilation of sea-ice concentrations. However, although both systems use the same model configuration and atmospheric initial conditions, they also differ in the ensemble generation scheme and their ocean initial conditions.

As mentioned in Sect. , ACCESS-S1 ensemble members are generated by adding random field perturbations to the atmosphere only, which then are transferred to the other components via the coupled simulation while ACCESS-S2 ensemble members are generated by time-lagged ensemble.

We believe this difference would only alter the ensemble spread and should have minimal impact in the dramatic differences in ensemble mean error shown in Fig. . In ACCESS-S1's scheme, ensemble members are all but guaranteed to be underdispersed in the ocean and sea-ice components. To test the effect of the ensemble generation scheme on ensemble spread, we compute the mean variance of RMSE of individual forecasts as a function of lag for each forecasting system. Figure  shows that indeed ACCESS-S1's initial spread is much lower than ACCESS-S2's in every month. However, ACCESS-S1's ensemble spread grows quickly, and after less than a week, both systems' spread is comparable. For forecasts initialised in February, March and April, ACCESS-S1 even has larger spread than ACCESS-S2.

Regarding ocean initial conditions, unfortunately we cannot directly compare ACCESS-S1 and ACCESS-S2 initial conditions because those data are currently not available. However, we believe there are reasons to consider that the effect would be of second-order importance. While the ocean data assimilation process is different, the underlying observations are similar, especially in the high latitudes, where there are few direct observations in total.

As an indirect comparison of ocean initial conditions, in their Fig. 9 show the correlation of ocean heat content in the upper 300 m of the ocean between EN4 and ACCESS-S2 and ACCESS-S1 reanalysis. In both systems, correlations are relatively high around 60° S and very low in the Antarctic coast; importantly, their correlation patterns are very similar, suggesting that the ocean initial conditions are not too dissimilar between the two systems. A limitation of this comparison is that the low correlation seen in both systems in the ice-covered regions only indicates that both systems have poor ocean initial conditions in that region but does not necessarily mean that their initial conditions are similar. Even though the ocean initial conditions are similarly correlated with the reference dataset in both systems, SST forecasts were shown by to be highly degraded in the Southern Ocean even at zero months lead time in ACCESS-S2 compared with ACCESS-S1. Correlation of SST anomalies in the ice-covered regions is much lower for ACCESS-S2 (, Fig. 13) and with a higher positive bias (their Fig. 12), which is consistent with the greater negative sea-ice concentration bias in ACCESS-S2 (Fig. ). This raises the possibility that it is the lack of assimilation of sea-ice data that is degrading SST forecasts and not the other way around.