The historical simulations of sea ice during 1979 to 2005 by the Coupled
Model Intercomparison Project Phase 5 (CMIP5) are compared with satellite
observations, Global Ice-Ocean Modeling and Assimilation System (GIOMAS)
output data and Pan-Arctic Ice Ocean Modeling and Assimilation System
(PIOMAS) output data in this study. Forty-nine models, almost all of the
CMIP5 climate models and earth system models with historical simulation, are
used. For the Antarctic, multi-model ensemble mean (MME) results can give
good climatology of sea ice extent (SIE), but the linear trend is incorrect.
The linear trend of satellite-observed Antarctic SIE is 1.29 (
The Coupled Model Intercomparison Project Phase 5 (CMIP5) provides a very useful platform for studying climate change. Simulations and projections by more than 60 state-of-the-art climate models and earth system models are archived under CMIP5. Assessment of the performance of CMIP5 outputs is necessary for scientists to decide which model outputs to use in their research and for model-developers to improve their models. Here, we focus on the assessment of sea ice simulations under the CMIP5 historical experiment. The CMIP5 data portal contains sea ice outputs from 49 coupled models. Many of these CMIP5 sea ice simulations have been evaluated and several valuable studies have been published.
For the Antarctic, the main problem of the CMIP5 models is their inability to reproduce the observed slight increase of sea ice extent (SIE). Turner et al. (2013) first assessed CMIP5 Antarctic SIE simulations using 18 models, and summarized that the majority of these models have too little SIE at the minimum sea ice period of February, and the mean of these 18 models' SIE shows a decreasing trend over 1979–2005, opposite to the satellite observation that exhibits a slight increasing trend. Polvani et al. (2013) used four CMIP5 models to study the cause of observed Antarctic SIE increasing trend under the conditions of increasing greenhouse gases and stratospheric ozone depletion. They concluded that it is difficult to attribute the observed trend in total Antarctic sea ice to anthropogenic forcing. Zunz et al. (2013) suggested that the model Antarctic sea ice internal variability is an important metric to evaluate the observed positive SIE trend. Using simulations from 25 CMIP5 models, Mahlstein et al. (2013) pointed out that internal sea ice variability is large in the Antarctic region and that both the observed and simulated trends may represent natural variation along with external forcing.
For the Arctic, CMIP5 models offer much better simulations. Stroeve et al. (2012) evaluated CMIP5 Arctic SIE trends using 20 CMIP5 models. They found that the seasonal cycle of SIE was well represented, and that the simulated SIE decreasing trend was more consistent with the observations over the satellite era than that of CMIP3 models but still smaller than the observed trend. They also noted that the spread in projected SIE through the 21st century from CMIP5 models is similar to that from CMIP3 models. Massonnet et al. (2012) examined 29 CMIP5 models and provided several important metrics to constrain the projections of summer Arctic sea ice projection. Liu et al. (2013) also pointed out that CMIP5 projections have large inter-model spread, but they also found that they could reproduce consistent Arctic ice-free time by reducing the large spread using two different approaches with 30 CMIP5 models.
Most evaluations of CMIP5 sea ice simulation in these studies are based only on some of CMIP5 models' outputs with some metrics, because other CMIP5 model outputs were not yet submitted. By now, all the CMIP5 participants have finished their model runs and submitted their model outputs. So, here we will evaluate all CMIP5 sea ice simulations with more metrics in both the Antarctic and the Arctic in an attempt to provide the community a useful reference. Generally speaking, our study shows the following: that the performance of Arctic sea ice simulation is better than that of Antarctic sea ice simulation, that sea ice extent simulation is better than sea ice volume simulation, and that mean state simulation is better than long-term trend simulation. If we want to get a similar result with all CMIP5 sea ice simulations, the number of models during analysis should be more than 22.
The rest of the paper is structured as follows. Section 2 presents sea ice data and analysis methodology used in this study. Model assessment is given in Sect. 3. Conclusions and discussion are provided in Sect. 4.
Sea ice simulations of CMIP5 historical runs from 49 CMIP5 coupled models
are now available. Monthly sea ice concentration (SIC) and sea ice thickness
from these models are used in this study. These outputs are published by the
Earth System Grid Federation (ESGF) (
Antarctic sea ice metrics in CMIP5 models, satellite observations
and GIOMAS data set. Column (a) is mean annual SIE in million km
Climatology
We select several metrics to assess the sea ice simulations in CMIP5 models. Mean state, seasonal cycle, the model internal variability, linear trends and simulation errors are used. For the Arctic sea ice, model mean state and seasonal cycle are important to Arctic sea ice projection (Massonnet et al., 2012). For the Antarctic sea ice, the model internal variability is an important metric to evaluate the observed positive SIE trend (Zunz et al., 2013). Annual mean SIE, SIE amplitude, standard deviation of detrended SIE anomaly (SIE variability), SIE linear trend and CMIP5-simulated SIE root mean square (RMS) error are shown in Tables 1 and 2. The same metrics for SIV are also shown in Tables 1 and 2. Each CMIP5 model-simulated SIC and sea ice thickness are given in the Supplement. Detailed analyses for Antarctic and Arctic are as follows.
CMIP5 multi-model ensemble mean (MME) Antarctic climatological SIE compares
well with the satellite-observed SIE, but the inter-model spread is large
(Fig. 1a and Table 1). Satellite observations show that the Antarctic SIE
has the minimum value of 3.0 million km
Arctic sea ice metrics in CMIP5 models, satellite observations and
PIOMAS data set. Column (a) is mean annual SIE in million km
CMIP5 model-simulated and satellite-observed SICs in February and September during 1979–2005 are shown in Figs. S3 and S4. In February most models have an overly small SIC compared with satellite observations, especially in the Bellingshausen Sea and the Amundsen Sea. More than half of CMIP5 models have no sea ice in the Bellingshausen Sea or in the Amundsen Sea. CNRM-CM5, GFDL-CM2p1, GFDL-CM3, GFDL-ESM2G, GFDL-ESM2M, IPSL-CM5B-LR and MIROC5 almost have no sea ice in February in the Antarctic. But ACCESS1.3, BNU-ESM, CCSM4, CESM1-BGC, CESM1-FASTCHEM, CSIRO-Mk3.6, FGOALS-g2, FIO-ESM and NorESM1-ME have more sea ice than satellite observations. Although CMIP5 simulated MME SIE fits the observations well, the MME spatial pattern of SIC does not fit the observations so well. MME SICs in the Weddell Sea, the Bellingshausen Sea and the Amundsen Sea are smaller than the observations. In September, most CMIP5 models have better performance than that in February, and MME SIC also has a better spatial pattern.
Monthly
Linear trends (unit: % per decade) of satellite-observed
Antarctic sea ice concentration during 1979 to 2005.
Figures 1b and 2 show that linear trends of CMIP5 MME Antarctic SIE do not
agree with the satellite observations. Many studies showed that Antarctic
SIE has an increasing trend since the end of 1970s (Cavalieri et al., 1997, 2003;
Zwally et al., 2002; Turner et al., 2009).
Satellite-observed Antarctic SIE has a small increasing linear trend with
the rate of 1.29 (
The trends of observed Antarctic SIC have large spatial differences (Fig. 3), but the simulated Antarctic SIC trends are almost decreasing everywhere (Fig. 4). Figure 3 shows that decreasing SIC is mainly in the Antarctic Peninsula, which is one of the three high-latitude areas showing rapid regional warming over the last 50 years (Vaughan et al., 2003). SIC also decreases in the Bellingshausen Sea and the Amundsen Sea in summer and autumn. The increasing SIC is mainly in the Ross Sea all year round and in the Weddell Sea in summer and autumn. Figure 4 clearly shows that CMIP5 MME SIC has decreasing trend everywhere except in the coast of the Amundsen Sea and in part of the Ross Sea in spring and winter.
Linear trends (units: % per decade) of CMIP5-simulated
Antarctic sea ice concentration during 1979–2005.
Climatology
SIV depends on both sea ice coverage and sea ice thickness. SIV is more
directly tied to climate forcing than SIE. So, SIV is an important climate
indicator in climate study. The observed sea ice thickness records are
mainly from submarine, aircraft and satellite. But the observations are not
continuous spatially or temporally over a long period (Stroeve et al.,
2014). For the Antarctic, the observed sea ice thickness data are quite
limited. A climatological 2.5
Climatology
Monthly
Linear trends (units: % per decade) of satellite-observed
Arctic sea ice concentration during 1979–2005.
CMIP5 SIV simulations have more problems than the SIE simulations. The main
problems of CMIP5 Antarctic SIV simulations include overly big SIV in summer,
overly small SIV in winter, overly large model spread, and an incorrect linear trend
compared with the GIOMAS data (Fig. 5). The annual mean SIV from GIOMAS is
11.02
CMIP5 shows a quite good annual cycle of Arctic SIE, but the model error in
winter is larger than that in summer and model spread is large (Fig. 6a).
Arctic SIE reaches the maximum value of 15.7 million km
CMIP5 MME SIE shows a decreasing trend that is consistent with the satellite
observation, though the decreasing rate is a little smaller than that of the
observation (Figs. 6b and 7). The satellite-observed SIE linear trend over
the period of 1979–2005 is
Figures 8 and 9 show that the spatial patterns of CMIP5-simulated SIC reduction rate are consistent with the observations from 1979 to 2005, but the decreasing rates are smaller than the observations. In spring and winter, the observed decreasing SIC is mainly in the Okhotsk Sea, Baffin Bay, Greenland Sea and Barents Sea; CMIP5-simulated decreasing SIC is also in these regions. In summer and autumn, the main decreasing SIC is in the Chukchi Sea, the Barents Sea, and the Kara Sea (Figs. 8 and 9), and CMIP5 MME SIC has similar characteristics. However, CMIP5 simulations have larger trends in the central Arctic Ocean.
Linear trends (units: % per decade) of CMIP5-simulated Arctic
sea ice concentration during 1979–2005.
Climatology
Compared with PIOMAS sea ice thickness, the main problems of CMIP5
simulations are smaller Arctic SIV all year round and an overly large model
spread (Fig. 10). In spring, the Arctic has the largest SIV. Long-term mean
PIOMAS SIV is at its maximum in April at 29.5
The first ensemble realizations of the 49 CMIP5 historical simulations are evaluated in terms of the performance of sea ice. Our results show that the Arctic sea ice simulations are better than the Antarctic sea ice simulations, and SIE simulations are better than SIV simulations. CMIP5 MME SIV is too little in winter and spring, because the sea ice thickness in CMIP5 models is too thin in winter and spring compared with the GIOMAS and PIOMAS data. In the Antarctic, MME can reproduce good mean state and monthly amplitude for SIE, but for SIV MME mean state and amplitude are smaller. In the Arctic, MME can reproduce good mean state and monthly amplitude for both SIE and SIV. CMIP5 simulations have very different variability (indicated by standard deviation of detrended monthly SIE and SIV) for different models. From Tables 1 and 2 we can conclude that the performance of each model is different. For the Antarctic, ACCESS1.0, BCC-CSM1.1, CESM1-CAM5-1-FV2, CMCC-CM, EC-EARTH, GISS-E2-H-CC, MIROC-ESM, MIROC-ESM-CHEM, MRI-CGCM3, MRI-ESM1 and NorESM1-M can give better SIE and SIV mean state. For the Arctic, ACCESS1.3, CCSM4, CESM1-BGC, CESM1-CAM5, CESM1-CAM5-1-FV2, CESM1-FASTCHEM, EC-EARTH, MIROC5, NorESM1-M and NorESM1-ME can give better mean state of SIE and SIV. The Arctic SIE linear trends of BNU-ESM, CanCM4, CESM1-FASTCHEM, EC-EARTH, GFDL-CM2p1, HadCM3, HadGEM2-AO, MIROC-ESM-CHEM, MPI-ESM-MR and MRI-ESM1 are closed to the observations.
Both satellite-observed Antarctic SIE and GIOMAS Antarctic SIV show increasing trends over the period of 1979–2005, but CMIP5 MME Antarctic SIE and SIV have decreasing trends. Only eight models' SIE and eight models' SIV show increasing trends. Can these few CMIP5 models reproduce the correct Antarctic sea ice trend? If we use these eight CMIP5 models to plot Antarctic SIC trends (not shown) as in Fig. 4, we will find that these eight CMIP5 model mean SIC trends have different spatial patterns with the observations (Fig. 3), although their model mean SIE and SIV have increasing trends. Satellite-observed Antarctic SIE has increased trends, but when we use the satellite-observed sea ice record, we should also keep in mind that it may also have a large degree of uncertainty. Eisenman et al. (2014) point out that sensor transition may cause a substantial change in the long-term trend.
The ratio of SIE and SIV RMS errors between the errors calculated using different number of CMIP5 models and the error calculated using all 49 CMIP5 models.
We can see that the CMIP5 MME does a good job in terms of climatological mean, but their inter-model spread is large. The number of models used in published studies is usually less than the total CMIP5 models. How many models can give as similarly good simulations as all the available CMIP5 models do? We first choose the CMIP5 models randomly. The model number changes from 1 to 49. We then calculate the SIE and SIV RMS errors between MME and observations or GIOMAS and PIOMAS data sets. For each fixed model number, we choose these models randomly at many different times, and then calculate the mean of the RMS errors. Figure 11 shows the ratio of SIE and SIV RMS errors between the errors calculated using different number of CMIP5 models and the errors calculated using all 49 CMIP5 models. We can see that the model errors decrease quickly as the model number increases; and the more models we use, the smaller error we have. For a fixed model number, the ratios of SIE are larger than the ratios of SIV, and Antarctic SIE has the largest ratio. When the model number is greater than 30, the model errors do not change much anymore. If we choose a criterion of RMS error larger than 15 % of all the model RMS error, the model number of 22 is the critical number for Arctic SIE. It means that more than 22 CMIP5 models should give similar MME as all 49 CMIP5 models.
In this study, satellite observations, PIOMAS and GIOMAS data during the period of 1979–2005 are used to assess the sea ice simulations from CMIP5 models. We always expect the models can capture the observed trends during this period. But we should note that simulations without data assimilation are always out of phase with the natural variability seen in the observations. So the differences between simulations and observations can either be due to model biases or natural climate variability (Stroeve et al., 2014).
Satellite-observed sea ice concentration data are provided by