Inter-comparison and evaluation of sea ice type concentration algorithms

Sea ice has been monitored in terms of concentration and types with microwave satellite observations since the late 1970s. However, it remains an open question as to which sea ice type concentration (SITC) method is most appropriate for ice type distribution and hence climate monitoring. This paper presents key results of inter-comparison and evaluation for eight SITC methods. The SITC methods were inter-compared with two sea ice age (SIA) and three sea ice type (SIT) products using microwave radiometer and scatterometer data from 2000 to 2015. Their performances were evaluated quantitatively with 5 samples that are used for generating tie points, and qualitatively with the RADARSAT imagery. The methods that combined scatterometer and radiometer data have overall better performances on ice type discrimination. The best methods are ECICEQSCAT-2 for the years 2000-2009 and ECICE-ASCAT for 2009-2015, both using scatterometer data along with radiometer data. Although the SIA and SIT products are fairly good datasets for delineating ice type distributions, the SITC methods are better on preserving details like varied concentration of different ice types and work better under specific sea ice conditions, 10 for instance, homogeneous sea ice regions with little artifact for SIA algorithms to track.

The SICCI-SIA product was generated on a polar stereographic grid with 20 km spacing, available from October 2012 to September 2017. The product is based on an Eulerian advection scheme, which accounts for observed divergence/convergence and freezing/melting of sea ice (Korosov et al., 2018). The algorithm uses daily sea ice drift and sea ice concentration products from the Ocean and Sea Ice Satellite Application Facility (OSISAF) (Lavergne, 2016a, b;Tonboe et al., 2017). Here the grid 90 cell is not assigned single ice age but fractions of different ice age. The main advantage of this product is the ability to generate the fraction of individual ice age in each pixel, which makes it more comparable to SITC results.

Sea ice type products
The sea ice type (SIT) products used in this study include one from the Copernicus Climate Change Service (referred to as C3S-SIT) and another one developed by Rivas et al. (2018) from the Royal Netherlands Meteorological Institute (KNMI) (referred 95 to as KNMI-SIT). The C3S-SIT product uses microwave radiometer data only, whereas KNMI-SIT is a purely scatterometerbased product.
In the C3S-SIT product, the sea ice types (MYI or FYI, both having ice concentration >30%) are assigned from atmospherically corrected brightness temperatures using a Bayesian approach  that computes the probability of occurrence of the most likely ice type. For the period from 2000 to 2015, the radiometer data consists of the FCDR of 100 Microwave Imager Radiances, described in Section 2.1. The C3S-SIT dataset is provided for the winter months (October to April) on a 25 km EASE grid, available through the C3S Climate Data Store (https://cds.climate.copernicus.eu).
The KNMI-SIT product uses a maximum likelihood class discrimination approach (Rivas et al., 2018) based on probabilistic distance to ocean wind and sea ice geophysical model functions (GMFs). GMFs describe the behaviour of backscatter as a function of observational geometry (i.e., incidence and azimuth angles) and geophysical variables such as wind speed and Figure 1. Geographic locations of the SAR images and outline of the Arctic Basin (red contour, provided by (Rivas et al., 2018)).

Three sea ice type concentration algorithms
In this study, we investigated the following three algorithms for sea ice type concentration retrievals: the NASA Team (NT) 120 algorithm (Cavalieri et al., 1984;Steffen and Schweiger, 1991), the Bootstrap (BT) algorithm (Comiso, 2012) and the ECICE algorithm (Shokr et al., 2008). NT and BT are purely radiometer-based algorithms, while ECICE allows combination of radiometer and scatterometer observations to estimate the concentration of any number of given ice types as long as it is less than the number of observations. The BT algorithm was first presented in Comiso (1990) for sea ice type concentrations using vertically polarized brightness 125 temperatures at 19 GHz and 37 GHz (T B19V and T B37V ). In Comiso (2012), the authors made an improvement to the algorithm by using dynamic tie points. Instead of using brightness temperatures directly, the NT algorithm uses polarization ratio (PR) and gradient ratio (GR) to calculate the concentration of FYI and MYI. These two independent ratios are: where T B19H is the horizontally polarized brightness temperature from the 19 GHz channel and other parameters are defined 130 in the same manner. The Environment Canada's Ice Concentration Extractor (ECICE) algorithm starts with a linear mixing model that decomposes each observation into contributions from each surface type (in our case FYI, MYI and OW) weighted by their concentrations. This algorithm requires a priori probability distribution function for each observation (input parameter) from each of the given surface types. The number of input parameters must be equal to or larger than the number of surface types. In this 135 study, six sets of input parameters were used in ECICE: two sets of purely radiometer data, and four sets of combined data (scatterometer and radiometer). Each set is comprised of four input parameters. The six sets of input parameters used in ECICE along with those in the NT and BT algorithms are presented in Table 1. Table 1. Input parameters and the resolution of the sea ice type concentration methods.

Method
Input parameters Resolution (km)   Averages of these samples were regarded as tie points in the BT and NT algorithms, whereas their probability distribution 150 functions (PDF) were used in ECICE for fair comparison. The PDFs and tie points for the brightness temperatures of all samples are shown in Figure 3. This section starts with a comparison between the two SIA products, NSDIC-SIA and SICCI-SIA. It then proceeds with comparison between the results from the eight SITC methods and each SIA product, and afterwards against the three SIT 155 products. While the comparison provides clues for the best SITC algorithm, the follow-up evaluation against tie points and the SAR images provides more concrete proof. All the comparisons of this study were performed on the data of a pre-defined area within the Arctic Basin and limited by the polar hole of 87.8 • N. Outline of the pre-defined Arctic Basin is as delineated by the red contour in Figure 1, provided by Rivas et al. (2018). In this area, MYI is the dominant ice type and ambiguities between MYI and deformed FYI are negligible compared to marginal ice zone. The area of a given ice age from the NSIDC-SIA product is estimated as the integral of the area of all pixels that contain the corresponding age. On the other hand, the area from the SICCI-SIA product is estimated as the integral of the fractions for the given ice age. This feature of SICCI-SIA product (the ability to generate the fraction of individual ice age in each pixel) makes 165 it more suitable for comparison against results from the SITC methods. MYI area from the NSIDC-SIA product is overall higher than the SICCI-SIA product, which is expected from the way of calculating the area. For the winters of 2012-2017, the average MYI area from NSIDC-SIA and SICCI-SIA is 2.53 ×10 9 km 2 170 and 1.89 ×10 9 km 2 , respectively. For the winter of 2012-2013, the MYI is only represented by second-year ice (SYI, no older ice) in the SICCI product since calculations were initiated in that winter and all MYI was assigned to be SYI. As calculations proceed, the percentage of SYI decreases in following winters and reaches 53.5% in the winter of 2016-2017, which is very close to that of NSIDC-SIA (57.6%). Within each winter, the MYI area has similar declining patterns from both products. The average reduction between October and April is 916.81 ×10 3 km 2 and 713.54×10 3 km 2 from the SICCI and NSIDC product,

SITC results versus SIA products
Comparison between the daily MYI area from the SITC methods against SIA products from NSDIC and SICCI is depicted in Figure 4. In the ideal case, the SITC data should match the upper green boundary. It can be seen that MYI area from all the SITC algorithms are in better agreement with the SICCI-SIA than NSIDC-SIA product. The SITC methods produce smaller 180 MYI area compared to the NSDIC product, especially in the beginning of the freezing season (October and November).

Method
Oct  Table 3. Mean differences (MD) and mean absolute differences (MAD) of MYI area between 6 SITC products and the adjusted value from NSIDC-SIA for January, October, and April of the years 2007-2015.

Method
Oct As mentioned in Section 2.2, the backtracking technique used in the SICCI-SIA products makes it a better source for SITC methods comparison. However, due to the better availability, the NSIDC-SIA product was used for long-term comparison instead of the SICCI-SIA product. Comparison of MYI area between SITC and SIA product is presented in the form of monthly averages as shown in Figure 5. As indicated in Section 4.1.1, MYI area from the NSIDC-SIA product is consistently 185 larger than that from the SICCI-SIA product, by 641.36 ×10 3 km 2 on average. This value is subtracted from the MYI area from the NSIDC-SIA product and this new parameter is referred to as "adjusted MYI area from NSIDC-SIA". It is represented by the lower boundary of the shaded area in Figure 5. Mean differences (MD) and mean absolute differences (MAD) between the SITC MYI area and the NSIDC-SIA adjusted MYI area are listed in Tables 2 and 3 for the QSCAT and ASCAT period, respectively. The MD represents any potential bias of the SITC MYI area relative to the NSIDC-SIA adjusted MYI area,190 whereas the MAD represents the magnitude of the mean spreading from the NSIDC-SIA adjusted MYI area. The MAD can be large even when there is no bias.
For October, in the top panel of Figure 5, all the SITC methods show less MYI than the NSIDC-SIA product, while the MYI area from the NT method exhibits a good agreement with the SICCI-SIA product. MYI area from the ECICE-QSCAT-1, ECICE-QSCAT-2 and NT method fall either completely or mostly into the shaded area of the NSIDC-SIA time series, which 195 indicates that these three methods agree well with the SIA products in October. For October throughout the years of 2000-2015, the ECICE-QSCAT-1 product has the smallest MAD of MYI area among all the SITC methods and on average gives approximately 1 ×10 5 km 2 higher estimate of MYI area than the NSIDC-SIA adjusted MYI (Tables 2 and 3). On the other hand, the NT method gives the smallest bias in both periods (MD of -42.31 ×10 3 km 2 and -122.03 ×10 3 km 2 in Tables 2 and   3, respectively) but slightly larger MAD than the ECICE-QSCAT-1 product ( For January, in the middle panel of Figure 5, the spread of the MYI area from the eight SITC methods is the smallest compared to October and April. This is expected from the cold temperatures and stable physical properties of sea ice in January, which leads to smaller uncertainties than beginning and end of winter. Most of the SITC MYI area fall between the values from NSIDC-SIA and SICCI-SIA. In the QSCAT period (2000)(2001)(2002)(2003)(2004)(2005)(2006)(2007)(2008)(2009), the BT method yields the smallest MYI area 205 difference, and the ECICE-QSCAT-2 has the second lowest (see Table 2). In the ASCAT period (2007)(2008)(2009)(2010)(2011)(2012)(2013)(2014)(2015), the three smallest MYI area differences are those from the ECICE-ASCAT-1, BT and ECICE-ASCAT-2 methods (Tabel 3).
In April, in the bottom panel of Figure 5, the NT, ECICE-QSCAT-1 and ECICE-SSMI-2 methods produce more MYI than the NSIDC-SIA product in some years, which could be a potential overestimation of MYI and can be confirmed in Section 4.4 (the case in the East Siberian Sea). Although the BT method yields the lowest MD in April throughout the years of 2000-2015

210
(see Table 2 and 3), it produces less MYI than any other SITC and SIA product in other months, which is regarded as a sign of underestimation for MYI. Except for BT, the smallest MYI area difference is that from ECICE-QSCAT-2 for the QSCAT period and ECICE-ASCAT-1 for the ASCAT period.

Comparison with sea ice type products
Sea ice type (SIT) and SITC products are widely used in climate studies. However there has been no study focusing on 215 the comparison of them. This subsection provides an overall comparison of the monthly MYI concentrations from the SITC methods for the three sets of ice pixels that are identified in each SIT product. For the C3S-SIT product, the three sets of pixels are those of FYI, MYI and MIX (pixels of either FYI or MYI). For the KNMI-SIT product, the three sets are FYI, SYI and MYI, where SYI represents the yonger multiyear ice and MYI here represents multiyear ice older than two years. Results from the SITC methods are considered to match the SIT product if the MYI concentration is high for MYI (and SYI) pixels from 220 the latter and low for FYI pixels.  The MIX pixels in the C3S-SIT product are those where the C3S-SIT algorithm has problems to distinguish between FYI and MYI. Therefore a large spread of the MYI concentrations from different SITC methods for these pixels are expected (see middle panel in Figure 6). The SYI pixels of the KNMI-SIT products correspond to the youngest multiyear ice, which might still capture some of the typical features that are used to identify FYI. Within each winter, MYI concentrations for the MIX (in C3S-SIT product) and SYI pixels (in KNMI-SIT products) have large variations and similar changing trends as those for 240 the MYI pixels (see middle panel in Figures 6, 7 and 8). During the winters from 2000 to 2015, there is no clear pattern for the MYI concentrations of the C3S-SIT MIX pixels, whereas those for the SYI pixels of the KNMI-SIT products mostly have clear increasing trend from October to January and stabilize to a value of 50%-70% until April.
The above discussions highlight two observations. Firstly, ECICE-QSCAT appears to be the most suitable approach to reproduce the information in the SIT products. However, the use of QSCAT data in ECICE may lead to exaggerated MYI 245 concentrations in the beginning of winter (October-November), especially for FYI pixels of the C3S-SIT product. Secondly, the convex shape of the average MYI concentration for the MYI pixels of SIT products in each winter indicates that all the SITC methods agree more with the SIT products during cold dry winter conditions and less during transition seasons.

Validation with samples/tie points
This section presents a quantitative validation using samples for tie points that were used in the SITC methods, whereas the next 250 section gives a qualitative validation using SAR images. The validation against tie points is to describe the overall performance of the SITC methods, while the latter validation is to show their performances under specific conditions.  Figure 9.   and BT) have non-uniformly distributed MYI concentrations between 50%-100% in area B, below 30% in area A and around 60% in area C. Discrepancies in area C indicate that all the purely radiometer-based methods and ECICE-QSCAT-1 tend to misidentify deformed FYI as MYI under high wind conditions.  (NSIDC-SIA is a weekly product). In this case, although the use of scatterometer data helps to improve the estimation of MYI concentration, it tends to misidentify young ice as MYI when backscatter from the former is high. (3)   highly dynamic regions. Moreover, the performance is better with QSCAT (Ku-band) than ASCAT (C-band). As it is shown 330 in Rivas et al. (2018), the separability between deformed FYI and MYI is better at Ku-band than C-band. Explanations can be found in a study by Ezraty and Cavanié (1999). Ku-band and C-band are similarly responsive to surface roughness, e.g., over deformed FYI, but Ku-band, with its wavelength around 2.2 cm, matches the characteristic dimension of air bubbles in MYI better. (5)  In area A, the backscatter is not as high as that in areas B and C, however its texture and floe-like structure make it appear to be partly covered with MYI. In this case, MYI concentrations from all the SITC methods have similar distribution pattern, with values between 50% and 90% in areas B and C, and below 40% in area A. In comparison, 340 the two SIA products give slightly different estimates than the SITC methods and SAR image. NSIDC-SIA misidentifies MYI as FYI in the southwest part of the SAR image (west of area C), whereas SICCI-SIA underestimates MYI concentrations in area C. Mismatches between the NSIDC-SIA product and SAR image can be partly explained by the low temporal resolution (weekly) of NSIDC-SIA, while the main reason is probably the incorrect representation of ice drift, which is the basis for all SIA products and is difficult to calculate when there is no distinct characteristics to track. As reported in Korosov et al. (2018), 345 the overall homogeneous distribution of MYI contration in SICCI-SIA is due to the void of potential artifacts in the ice drift product that is used for sea ice age tracking. On the other hand, the sea ice concentration product used in SICCI-SIA leads to an overall underestimation of old ice fraction. (6): ice in the Fram Strait A case from the Fram Strait is shown in Figure 15. pack MYI and open ocean, and the high wind speed, it is quite likely to be brash ice in area B, which is typical in the highly dynamic marginal ice zone. In this area, all the SITC methods manage to give near-zero MYI concentrations, while the two SIA products both misidentify FYI as MYI. Another discrepancy between the SITC and SIA results is for area C, where the SAR image reveals bright backscatter signature yet smooth texture, which appears to be deformed FYI. In contrary to the situation in area B, the SIA products are able to identify FYI but not the SITC methods. This case, along with the previous one, proves 360 that sometimes the SITC methods can work better than the SIA products on ice type discriminations, and in other cases the SIA products capture better the ice type distributions than most of the SITC methods. It is important to have both products for sea ice type distribution monitoring.

Conclusions
Results from the eight SITC methods are inter-compared with two SIA products and three SIT products, using microwave 365 radiometer and scatterometer data spanning the winters (October -April) from 2000 to 2015. Performances of the SITC methods are evaluated quantitatively and qualitatively with samples that are used to generate tie points as well as SAR images.
Overall, the results can be summarized as follows: -Using scatterometer data along with radiometer data helps the discrimination of FYI and MYI. The most appropriate approach to reproduce sea ice type distributions in the Arctic is the combined method: ECICE-QSCAT-2 for the years -In the SITC methods, Ku-band scatterometer works better than C-band scatterometer on identifying MYI due to the higher sensitivity to volume scattering in MYI. On the other hand, performance of the QSCAT-based methods depends on the input parameters (combination of observations from scatterometer and radiometer), whether they contribute to the distinction of ice types under different conditions.

375
-Among the pure radiometer-based SITC methods, the ECICE-SSMI methods work slightly better than the NT method on identifying MYI, while the BT method has an overall underestimation of the MYI concentration.
-The NSIDC-SIA and SICCI-SIA products work fairly well on identifying the ice types that can be distinguished from the eight SITC methods, however they highly rely on the ice drift product used in the algorithm. The weekly NSIDC-SIA product does not provide ice age information as precisely as the SICCI-SIA product, due to the lower temporal resolution 380 and its monitoring of oldest ice. The SICCI-SIA product, on the other hand, seems to underestimate MYI concentration in homogeneous regions with little artifacts to track.
-Among the three SIT products, the eight SITC methods have the best agreement with the KNMI-QSCAT-SIT product.
Besides, all the SITC methods agree better with the SIT products in mid-winter than transition seasons.
-Although the SIA and SIT products are fairly good datasets for delineating ice type distributions, the SITC methods 385 are better on preserving details like varied concentration of different ice types and work better under specific sea ice conditions.