the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
GLAcier Feature Tracking testkit (GLAFT): A statistically- and physically-based framework for evaluating glacier velocity products derived from satellite image feature tracking
Whyjay Zheng
Shashank Bhushan
Maximillian Van Wyk De Vries
William Kochtitzky
David Shean
Luke Copland
Christine Dow
Renette Jones-Ivey
Fernando Pérez
Abstract. Glacier velocity measurements are essential to understand ice flow mechanics, monitor natural hazards, and make accurate projections of future sea-level rise. Despite these important applications, the method most commonly used to derive glacier velocity maps, feature tracking, relies on empirical parameter choices that rarely account for glacier physics or uncertainty. Here we test two statistics- and physics-based metrics to assess velocity maps from a range of existing feature-tracking workflows at Kaskawulsh Glacier, Canada. Based on inter-comparisons with ground-truth data, velocity maps with metrics falling within our recommended ranges contain fewer erroneous measurements and more spatially correlated noise than velocity maps with metrics that deviate from those ranges. Thus, these metric ranges are suitable for refining feature-tracking workflows and evaluating the resulting velocity products. We have released an open-source software package for computing and visualizing these metrics, the GLAcier Feature Tracking testkit (GLAFT).
Whyjay Zheng et al.
Status: final response (author comments only)
-
RC1: 'Comment on tc-2023-38', Tazio Strozzi, 05 May 2023
The manuscript introduces a method that can be used to evaluate the quality of glacier velocity maps derived from satellite image feature tracking. The method includes two numbers that we can calculate for each velocity map, one based on the statistics on ice-free regions and one based on the ice flow physics. The method was tested using satellite optical data (Landsat 8 and Sentinel-2) from Kaskawulsh glacier, Canada. Recommendations on the use of the method are given and an open-sourced software tool was released to support users assess their velocity maps.
The characterization of the errors of ice velocity maps derived from satellite image feature tracking is indeed currently still challenging. In the literature various methods to characterize the quality of flow velocities are proposed, including local measure of correlation quality estimate (e.g. CC or SNR), fraction of area with valid measurements of total glacier area, statistical measures of the velocity over stable terrain and intercomparison/validation with in-situ data or products from different sensors. But overall, for a proper validation of ice velocity maps it is hard to get access to coincident independent data in time and space. In addition, currently the know-how of the operators is often more important than a proper independent evaluation of the results for the selection of the most important parameters to be considered in the traking algorithm. Therefore, the contribution of the manuscript is welcome by the user community.
The paper is well structured and the aims of the work are clearly introduced with a comprehensive discussion of the technical limitations of the current literature. The open-sourced software tool should be very useful to users to check their results, but I must admit that I did not check it. I have also not checked the mathematics of the two metrics, but everything seems plausible. This paper is more of a sort of welcome and enjoyable niche investigation with tools useful to the user community rather than a research with broad scientific implications, but I recommend accepting it for publication in TC after moderate review and consideration of three main points and a few minor ones.
1) The first comment can be probably addressed with an answer rather than with a proper revision of the manuscript, a part from adding a few words at the beginning of the paper. When reading title and abstract I was expecting to see also results obtained from satellite SAR data, in particular Sentinel-1. But this is not the case and only results from optical data (Landsat 8 and Sentinel-2) are analysed. I was rather disappointed by this as a comparison between the optical and SAR results would have increased the interested of the paper. But I can accept the decision of the authors to concentrate on satellite optical images and I just suggest to add “optical” before “satellite image feature tracking” in the title of the paper and again “based on satellite optical images” before “at Kaskawulsc Glacier” in the abstract unless you want to include in the paper also tests based on results from satellite SAR image feature tracking (e.g. from http://retreat.geographie.uni-erlangen.de).
2) In Section 2 only the uncertainty (i.e. precision) of the feature tracking algorithm is introduced but other aspects of the accuracy of the results are not discussed. Some aspects of the later are actually discussed in the continuation of the paper, e.g. at l. 255 (bias correction for image misalignment) and l. 327 (larger errors of matches on the glacier surface rather than over ice-free terrain). I suggest to discuss a little bit more in detail the aspects of precision versus accuracy already in Section 2, possibly also mentioning aspects related to geolocation (e.g. use of outdated DEM) and atmospheric disturbances (clouds in optical images or ionosphere for SAR images).
3) Finally, I recommend making Section 4.1 (Recommended strategy to evaluate velocity map quality) more self-reading, e.g. by saying again explicitly what are all variables and write again in Table 2 the equations. I agree that this might be a repetition, but for someone interested to quickly implement the proposed metrics or reviewing what the open-sourced software tool is computing having a self-reading short section could be quite useful instead of having to go back and forth over the entire manuscript.
Here a list of other minor points that should be considered in the revision of the manuscript.
l. 53. Include something like “using different parameter settings of various software packages” before “172 glacier velocity maps”. At first I could not really understand how you computed 172 velocity maps from two Landsat 8 and two Sentinel-2 image pairs.
l. 102. I suggest including reference to the “multivariate kernel density estimation (KDE)”.
l. 341. What are the vertical bars in Figure 5b?
l. 382. Make reference to https://www.mdpi.com/2072-4292/10/6/929 as a previous intercomparison exercise.
l. 383. I am not convinced that the assumption about “coherent … flow pattern” (see l. 130) of metric 2 would be still valid for "different use cases (e.g., sand dune mapping or earthquake displacement)". Remove this or explain why metric 2 is still valid for other uses.
Citation: https://doi.org/10.5194/tc-2023-38-RC1 -
RC2: 'Comment on tc-2023-38', Suzanne Bevan, 09 May 2023
This paper defines metrics for assessing and comparing the quality of surface velocities measured using feature-tracking of satellite images. The first method allows uncertainty to be estimated using only ‘correct’ matches over stationary areas. The second metric is based on an assessment of how realistic the derived strain-rate fields are. Both metrics can be used to refine the choice of empirical parameters for feature-tracking algorithms.
The metrics are demonstrated and tested using 172 examples feature-tracking of optical satellite data for Kaskawulsh Glacier, Canada. The ensemble of results is comprised of different sensors, different surface conditions, different tracking parameters, and different algorithms. The results are validated against in-situ GNSS data, and against a synthetic velocity field.
It is concluded that both metrics can be used to benchmark feature-tracking algorithms and to facilitate intercomparison exercises.
The software for generating the metrics is called the GLAcier Feature Tracking testkit (GLAFT) and is provided on Ghub and Github. I did not download or test this software so cannot comment on how easy it is to use.
The paper is well written and organised and worth publishing. However, while the metrics could prove very useful to practitioners of feature-tracking, I’m not sure how much interest they would be to the end-user as they do not, ultimately, allow an objective uncertainty to be delivered with the data.
As the authors state ‘accurate maps of ice velocity with rigorous uncertainty propagation are needed’. Including the metrics described in this paper as metadata with supplied velocity maps would not meet this requirement. Whilst metric 1 provides uncertainty associated with correct matches, as acknowledged, the measured velocity fields over moving terrain are a combination of correct and incorrect matches. The metrics would allow users to compare velocity products in terms of quality, but more often than not, velocity products are chosen for reasons of temporal and spatial coverage and resolution.
The examples are limited to optical feature tracking of one glacier. The authors should comment on what issues there might be with applying these metrics to SAR feature-tracking?
How feasible is it to use either metric over ice sheet flow? With respect to lack of stationary areas, and also very different strain-rate fields in comparison with glacier flow.
Some comment on how metric 1 could improve bias removal/calibration of measured velocities would be useful.
More specific comments:
Title – remove hyphens, adverbs do not need hyphens.
Line 74. Delete ‘and calculate uncertainty for correct matches’. This phrase is not relevant in this paragraph and by removing it the next sentence makes sense.
Line 84. Not sure this sentence makes sense. ‘…should provide a global estimate…’ of what? Needs rewriting somehow to make sense and to provide a better lead into the following 2.1 and 2.2 subsections. Also, DO the presented metrics provide image-wide estimates of incorrect matches? I don’t think any of the figures show examples of this.
Line 99. The last sentence ‘A metric involving the total number and distribution of incorrect matches…’ . Where is this metric presented? The following paragraphs of this section rely on identifying uncertainty of correct matches.
Line 100 . ‘feature-tracking workflow’. Here and throughout.
Line 129. Recast sentence to begin with ‘For computation simplicity’.
Line 131. Change ‘the flow pattern’ to ‘the measured flow pattern’.
Section 3.2
Explain here how the percentages of incorrect matches are calculated. How is incorrect determined?
Is rectangular meaning non-square?
Fig. 6. Would be useful to add the polygons of static and flow areas to these figures.
It would be useful to have the supplementary material available as a pdf without having to go through github, or make the directions how to reach the Supplementary figures clearer. It took me a while of searching to find them.
Line 350. More accurate to say ‘non-square’ than rectangular.
Citation: https://doi.org/10.5194/tc-2023-38-RC2
Whyjay Zheng et al.
Data sets
GLAFT data repository Whyjay Zheng, Shashank Bhushan, Maximillian Van Wyk De Vries, William Kochtitzky, David Shean, Luke Copland, Christine Dow, Renette Jones-Ivey, and Fernando Pérez https://doi.org/10.17605/OSF.IO/HE7YR
Model code and software
GLAcier Feature Tracking testkit: glaft The GLAFT team https://github.com/whyjz/GLAFT
Ghub - Resources: GLAcier Feature Tracking testkit The GLAFT team https://theghub.org/resources/glaft
Executable research compendia (ERC)
whyjz/GLAFT: GLAFT 0.2.0 Whyjay Zheng, Shashank Bhushan, and Erik Sundell https://doi.org/10.5281/zenodo.7527957
Whyjay Zheng et al.
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 298 | 192 | 14 | 504 | 8 | 7 |
- HTML: 298
- PDF: 192
- XML: 14
- Total: 504
- BibTeX: 8
- EndNote: 7
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1