the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 03 Jul 2023
SAR Deep Learning Sea Ice Retrieval Trained with Airborne Laser Scanner Measurements from the MOSAiC Expedition
Abstract. Automated sea ice charting from Synthetic Aperture Radar (SAR) has been researched for more than a decade and still, we are not close to unlocking the full potential of automated solutions in terms of resolution and accuracy. The central complications arise from ground truth data not being readily available in the polar regions. In this paper, we build a dataset from 20 near coincident X-Band SAR acquisitions and as many Airborne Laser Scanner (ALS) measurements from the Multidisciplinary drifting Observatory for the Study of Arctic Climate (MOSAiC), between October and May. This dataset is then used to assess the accuracy and robustness of five machine learning based approaches, by deriving classes from the freeboard, surface roughness (standard deviation at 0.5 m correlation length) and reflectance. It is shown that there is only a weak correllation of the radar backscatter and the sea ice topography. Accuracies between 40 % and 69 % percent and robustnesses between 68 % and 85 % give a realistic insight into modern classifiers' performance across a range of ice conditions over 8 months. It also marks the first time algorithms are trained entirely with labels from coincident measurements, allowing for a probabilistic class retrieval. The results show that segmentation models able to learn from the class distribution significantly perform pixel-wise classification approaches.
- Preprint
(18229 KB) - Metadata XML
- BibTeX
- EndNote
Status: closed
-
RC1: 'Comment on tc-2023-72', Anonymous Referee #1, 16 Aug 2023
Summary:
The authors are addressing the lack of a standard deep learning approach for sea ice classification in SAR imagery by assessing existing deep learning approaches with an improved data set. By utilizing airborne laser scanner (ALS) measurements from the MOSAiC expedition that are closely timed with TerraSAR-X collects, they derive their own near-coincident training label data set. They then evaluate a number of existing approaches on this improved data set to determine which approaches are more informative.Broad Comments:
The manuscript covers an exciting topic that it is an open problem in the sea ice and SAR community. However, the text itself is currently lacking some much-needed details that would help the reader identify the full contributions of this work. The authors state in the Introduction that they are assessing existing deep learning approaches for sea ice classification in SAR imagery by testing them on a more reliable data set. The goal seems to be to advance our understanding of which deep learning approaches are actually advantageous for sea ice classification, leading to a standard for the community. However, there wasn’t enough detail provided about the previous studies nor this study’s methods to know if enough was kept consistent when repeating the analysis to have comparable results (e.g., did the previous studies also use X-band data? Were model parameters and frameworks kept consistent? etc.). The authors provide some discussion towards their differing results, especially in terms of resolution, but more information would be useful.Specific Comments:
Section 1. Introduction- Page 2, Figure 1: The caption mentions a red circle, but it was not visible to me. Perhaps check printed color (and consider using a different color for those who are colorblind).
- Page 3, lines 47-48: Traditionally, these types of datasets tend to be too sparse to provide robust training sets for image-based ML models. How does this dataset differ?
- Page 3, line 57: How close is near-coincident? An example in parentheses would be helpful here.
Section 2. Methodology
- Page 3, first paragraph: More information is needed here on how the TSX data is processed (what corrections were performed, filtering, etc.) and what the effective resolution of the data is after such processing. Were you considering backscatter as your data? If so, what was it normalized to (e.g., , etc.)?
- Page 3, first paragraph: It is stated that only co-polarized channels are used. How do you mitigate effects from wind-roughened waters and other confounding factors which are more prevalent in co-polarized data?
- Page 3, line 68: The ALS dataset seems to focus on the winter seasons. Is any comparison done for seasonality transfer?
- Page 3, second paragraph: What was the regional coverage of the ALS dataset? Did it cover all representative portions of the Arctic, or was it constrained to a particular region?
- Page 3, second paragraph: A verbal comparison between ALS dataset footprint size versus the footprint of a TSX image would be useful for context.
- Page 4, line 87: Was a distribution analysis of the freeboard and roughness per SAR pixel done to ensure that the mean and standard deviation are representative statistical measures for these data? If the distribution is heavily skewed, it may be more appropriate to use the median, for example.
- Page 4, line 92: Again, it would be helpful to have an example of these time differences (minutes, hours?).
- Page 4, line 95: It would be good to explicitly name the conventions you are pulling from and provide an associated reference.
- Page 5, line 111: It is unclear to me why you are assuming a Gaussian distribution when the density functions in Figure 2 are non-Gaussian. Your metrics (e.g., standard deviation) are likely to be heavily influenced by outliers.
- Page 6, Figure 3: Please add the mathematical notations for freeboard and SAR backscatter to the caption here so it is clear what the figure axes are referring to.
- Page 6, Figure 4: Similarly, it would be good to note in the caption that PDF refers to probability density function for the general reader.
- Page 7, line 123: Does favoring equal class performance affect how the model will perform operationally, where classes are almost always not equal?
- Page 7, line 125: This sentence is unclear to me. Perhaps the second mention of “backscatter” should actually be “topography”?
Section 3. Results
- Page 8, first paragraph: Referring to the VGG16, etc. as pixel-wise classification approaches is confusing here, especially since under the Section 2.3 you describe these classification approaches as predicting over patches as a whole (or just the center), and the segmentation approaches as predicting a label for every pixel in a patch. Given that, seems backwards to refer to the center-pixel classification approaches as pixel-wise classification. I would try using a different descriptor if possible.
- Page 9, Figure 5: Do you have ground truth labels for this example? If so, it would be helpful to include them in the figure.
Section 4. Discussion
- Page 10, line 204: Can you elaborate more on how your results compare to the results from previous studies that you are replicating?
- Page 11, Figure 7: A reference for the misclassification of water and old ice being a common issue would be helpful to include here.
- Page 12: It would be great to see some discussion on how the temporal span of the dataset may affect the results, as well as how you expect the results to change (or not change) when applied to different seasons. For example, do you think certain models would be more robust to the existence of melt ponds on older ice during the summer months?
- Page 12: Likewise, a discussion on the spatial coverage of the ALS dataset and how that could affect your results, especially when compared to previous studies, would be useful. Did this dataset only cover a particular region of the Arctic, and would you expect results to differ if you had data from other regions?
Technical Corrections:
- Page 3, line 49: Should be “neural networks”.
- Page 3, line 67: There is a space missing between the citations.
- Page 5, Figure 2: Do not need the word “the” before “April” in the first line of the caption.
- Page 5, line 111: Gaussian should be capitalized.
- Page 8, line 164: The word “table” should be capitalized.
- Page 8, line 179: The comma after “shows” is unnecessary.
- Page 10, line 183: The comma after “apparent” is unnecessary.
- Page 10, line 188: The comma after “results” is unnecessary.
- Page 12, line 214: Unet should be capitalized to be consistent with the rest of the text, and “to” is repeated near the end of the line.
- Page 10, line 215: The comma after “labels” is unnecessary. There should be a comma after “For example”, though.
Citation: https://doi.org/10.5194/tc-2023-72-RC1 - AC1: 'Reply on RC1', Karl Kortum, 22 Nov 2023
-
RC2: 'Comment on tc-2023-72', Anonymous Referee #2, 26 Oct 2023
- AC2: 'Reply on RC2', Karl Kortum, 22 Nov 2023
Status: closed
-
RC1: 'Comment on tc-2023-72', Anonymous Referee #1, 16 Aug 2023
Summary:
The authors are addressing the lack of a standard deep learning approach for sea ice classification in SAR imagery by assessing existing deep learning approaches with an improved data set. By utilizing airborne laser scanner (ALS) measurements from the MOSAiC expedition that are closely timed with TerraSAR-X collects, they derive their own near-coincident training label data set. They then evaluate a number of existing approaches on this improved data set to determine which approaches are more informative.Broad Comments:
The manuscript covers an exciting topic that it is an open problem in the sea ice and SAR community. However, the text itself is currently lacking some much-needed details that would help the reader identify the full contributions of this work. The authors state in the Introduction that they are assessing existing deep learning approaches for sea ice classification in SAR imagery by testing them on a more reliable data set. The goal seems to be to advance our understanding of which deep learning approaches are actually advantageous for sea ice classification, leading to a standard for the community. However, there wasn’t enough detail provided about the previous studies nor this study’s methods to know if enough was kept consistent when repeating the analysis to have comparable results (e.g., did the previous studies also use X-band data? Were model parameters and frameworks kept consistent? etc.). The authors provide some discussion towards their differing results, especially in terms of resolution, but more information would be useful.Specific Comments:
Section 1. Introduction- Page 2, Figure 1: The caption mentions a red circle, but it was not visible to me. Perhaps check printed color (and consider using a different color for those who are colorblind).
- Page 3, lines 47-48: Traditionally, these types of datasets tend to be too sparse to provide robust training sets for image-based ML models. How does this dataset differ?
- Page 3, line 57: How close is near-coincident? An example in parentheses would be helpful here.
Section 2. Methodology
- Page 3, first paragraph: More information is needed here on how the TSX data is processed (what corrections were performed, filtering, etc.) and what the effective resolution of the data is after such processing. Were you considering backscatter as your data? If so, what was it normalized to (e.g., , etc.)?
- Page 3, first paragraph: It is stated that only co-polarized channels are used. How do you mitigate effects from wind-roughened waters and other confounding factors which are more prevalent in co-polarized data?
- Page 3, line 68: The ALS dataset seems to focus on the winter seasons. Is any comparison done for seasonality transfer?
- Page 3, second paragraph: What was the regional coverage of the ALS dataset? Did it cover all representative portions of the Arctic, or was it constrained to a particular region?
- Page 3, second paragraph: A verbal comparison between ALS dataset footprint size versus the footprint of a TSX image would be useful for context.
- Page 4, line 87: Was a distribution analysis of the freeboard and roughness per SAR pixel done to ensure that the mean and standard deviation are representative statistical measures for these data? If the distribution is heavily skewed, it may be more appropriate to use the median, for example.
- Page 4, line 92: Again, it would be helpful to have an example of these time differences (minutes, hours?).
- Page 4, line 95: It would be good to explicitly name the conventions you are pulling from and provide an associated reference.
- Page 5, line 111: It is unclear to me why you are assuming a Gaussian distribution when the density functions in Figure 2 are non-Gaussian. Your metrics (e.g., standard deviation) are likely to be heavily influenced by outliers.
- Page 6, Figure 3: Please add the mathematical notations for freeboard and SAR backscatter to the caption here so it is clear what the figure axes are referring to.
- Page 6, Figure 4: Similarly, it would be good to note in the caption that PDF refers to probability density function for the general reader.
- Page 7, line 123: Does favoring equal class performance affect how the model will perform operationally, where classes are almost always not equal?
- Page 7, line 125: This sentence is unclear to me. Perhaps the second mention of “backscatter” should actually be “topography”?
Section 3. Results
- Page 8, first paragraph: Referring to the VGG16, etc. as pixel-wise classification approaches is confusing here, especially since under the Section 2.3 you describe these classification approaches as predicting over patches as a whole (or just the center), and the segmentation approaches as predicting a label for every pixel in a patch. Given that, seems backwards to refer to the center-pixel classification approaches as pixel-wise classification. I would try using a different descriptor if possible.
- Page 9, Figure 5: Do you have ground truth labels for this example? If so, it would be helpful to include them in the figure.
Section 4. Discussion
- Page 10, line 204: Can you elaborate more on how your results compare to the results from previous studies that you are replicating?
- Page 11, Figure 7: A reference for the misclassification of water and old ice being a common issue would be helpful to include here.
- Page 12: It would be great to see some discussion on how the temporal span of the dataset may affect the results, as well as how you expect the results to change (or not change) when applied to different seasons. For example, do you think certain models would be more robust to the existence of melt ponds on older ice during the summer months?
- Page 12: Likewise, a discussion on the spatial coverage of the ALS dataset and how that could affect your results, especially when compared to previous studies, would be useful. Did this dataset only cover a particular region of the Arctic, and would you expect results to differ if you had data from other regions?
Technical Corrections:
- Page 3, line 49: Should be “neural networks”.
- Page 3, line 67: There is a space missing between the citations.
- Page 5, Figure 2: Do not need the word “the” before “April” in the first line of the caption.
- Page 5, line 111: Gaussian should be capitalized.
- Page 8, line 164: The word “table” should be capitalized.
- Page 8, line 179: The comma after “shows” is unnecessary.
- Page 10, line 183: The comma after “apparent” is unnecessary.
- Page 10, line 188: The comma after “results” is unnecessary.
- Page 12, line 214: Unet should be capitalized to be consistent with the rest of the text, and “to” is repeated near the end of the line.
- Page 10, line 215: The comma after “labels” is unnecessary. There should be a comma after “For example”, though.
Citation: https://doi.org/10.5194/tc-2023-72-RC1 - AC1: 'Reply on RC1', Karl Kortum, 22 Nov 2023
-
RC2: 'Comment on tc-2023-72', Anonymous Referee #2, 26 Oct 2023
- AC2: 'Reply on RC2', Karl Kortum, 22 Nov 2023
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 366 | 154 | 24 | 544 | 23 | 14 |
- HTML: 366
- PDF: 154
- XML: 24
- Total: 544
- BibTeX: 23
- EndNote: 14
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1