I read this brief comment with interest, and found some core issues troubling.

Fundamentally, the authors discuss 'actionable' science, but they discuss it stripped of context. Actions are defined by the American Heritage Dictionary as 'organized activity to accomplish an objective'. Science cannot be judged to be actionable, or not, outside the context of an organized activity and an objective. It makes little sense to talk about something being 'actionable' in general, outside of a specific decision context.

The authors neglect the extensive literature on decision science and risk analysis relevant to using sea-level projections in adaptation decision making. For a relatively recent review, see Keller, K., Helgeson, C., & Srikrishnan, V. (2021). Climate risk management. Annual Review of Earth and Planetary Sciences, 49, 95-116, https://www.annualreviews.org/doi/abs/10.1146/annurev-earth-080320-055847.

In the specific context of communicating sea-level uncertainty and ambiguity, the authors should also see Kopp, R. E., Oppenheimer, M., O’Reilly, J. L., Drijfhout, S. S., Edwards, T. L., Fox-Kemper, B., ... & Xiao, C. (2023). Communicating future sea-level rise uncertainty and ambiguity to assessment users. Nature climate change, 13(7), 648-660, https://www.nature.com/articles/s41558-023-01691-8. Given the direct relevance, this latter omission is particularly surprising.

Why do the organized activity and the objective matter?

Broadly, high-end sea-level rise scenarios, including low-confidence processes, are valuable in flexible, adaptive decision-making. This is shown by a number of papers, but perhaps most clearly and directly for this context in a preprint by Feng et al. (https://doi.org/10.22541/essoar.170914510.03388005/v1 ). 

Among other analyses, Feng et al. compare idealized protection schemes for Manhattan under (1) a static optimal approach, where a single sea wall elevation must be picked based on available knowledge today, and (2) a variety of dynamic approaches, where sea wall height can be periodically adjusted based on new information. (I focus particularly on the 'reinforcement learning' approach described therein).

They consider two cases where projects are planned under inaccurate sea-level rise projections: (A) where planning takes place under the SSP5-8.5 low-confidence projections but the reality corresponds to SSP2-4.5 medium-confidence projections, and (B) where planning takes place under the SSP2-4.5 medium-confidence projections by reality corresponds to the SSP5-8.5 low-confidence projections.

In the former case -- where high-end projections are used and reality underperforms -- the expected net present value cost is $2.3 billion, $1.0 billion more than with the correct (lower) distribution, if a static approach is taken. With a flexible approach, the expected net present value cost is $1.0 billion, just $0.1 billion more than if the correct distribution is chosen.

However, in the latter case -- where middle-of-the-road projections are used and reality overperforms -- the expected net present value cost is $15 billion, $12 billion more than with the correct (high-end) distribution if a static optimal approach is taken. With a flexible approach, the expected net present value cost is $3.9 billion, $0.9 billion more than if the high-end distribution had been used.

Thus, with a dynamic approach, using high-end projections that capture low-confidence processes makes a lot of economic sense. Such an approach cuts of the tail risk at relatively small additional cost. (In fact, the cost of a static optimal approach using the correct distribution in a middle-of-the-road world is more than the cost of using a dynamic approach with the overestimated, high-end distribution.)

However, with a static approach, the costs of getting the distribution wrong are more substantial (though an order of magnitude larger if the distribution is underestimated than if it is overestimated).

In truth, I think the concern the authors address is not one with scientists offering practitioners low-confidence, high-end projections as part of the domain of plausible futures. It is with how these projections are then used.

As the Feng et al. analysis, and others, indicate, the most economic approach given substantial uncertainty and ambiguity is most often the dynamic one. Where a static approach must be used, whether due to inability to undertake a dynamic approach or regulatory inflexibility, then benefit-cost theory tells us what needs to be taken into account in order to determine the best option. This includes:

1) The benefit in terms of reduced risk associated with choosing different adaptation levels

2) The cost in terms of additional adaptive expenditures in terms of choosing different adaptation levels

3) The discount rate used to tradeoff present adaptation costs and future harms

4) The risk aversion that determines how much weight is given to the high-end of the cost distribution

5) The ambiguity aversion that determines how much weight is given to different alternative probability distribution for sea level and thus for cost.

Where the costs and benefits of adaptation are comparable, discomfort will arise if regulatory guidance specifies a single adaptation target stripped of context, because a user's risk and ambiguity aversion applies to both the costs and benefits of adaptation, not just the benefits.

I suspect that the authors' concern with the actionability of projections incorporating low-confidence processes is misaimed. Given appropriately flexible decision frameworks, as Feng et al. show, we are better off incorporating such high-end projections. 

Regulations that rigidly prescribe the use of specific high-end projections in static contexts, however, run the risk of leading to sub-optimal outcomes. It may be appropriate for policy to set discount, risk aversion, and ambiguity aversion levels for specific contexts; this is a matter where different political philosophies will lead to different judgements. However, given these parameters, identifying the benefit-cost optimal outcome requires considering the net value of adaptation benefits and adaptation costs under these parameters. If costs and benefits are comparable, overly rigid targets might cut off the long tail of sea-level harms but create a long tail of adaptation cost overruns.

In short, the authors have chosen the wrong target. Scientists should strive to communicate not just projections that incorporate processes for which there is a high degree of evidence, but also processes that are of potentially great significance but less agreement and evidence -- as AR6 has done. It is, however, important that actions be guided by decision frameworks that correctly reflect the nature of the information provided.

I appreciate the opportunity to comment, since I would like to point out, and help correct, an error in the paper.

Specifically, on page 6, the authors have made the following statement about the influence of DeConto and Pollard (2016) on the U.S. interagency sea level rise scenarios report of Sweet et al. (2017) (my emphasis):

“To be consistent with “recent updates to the peer-reviewed scientific literature”, they issued an “Extreme” global mean sea-level projection of 2.5 m by 2100 for RCP8.5, exceeding the previous upper bound of 2.0 m based on Pfeffer et al. (2008). Their Extreme projection relied on a large AIS contribution, based primarily on DP16. [Footnote 4]

[Footnote 4] The 2.0 m upper bound of Pfeffer et al. (2008) assumed large contributions from both ice sheets: 0.54 m from the GrIS and 0.62 m from the AIS. In AR5, Church et al. (2013) estimated a likely upper bound of just 0.21 m for the GrIS, since process models do not support “the order of magnitude increase in flow” in Pfeffer et al. (2008). To reach an upper bound of 2.0 m or more, S17 therefore needed an increased AIS contribution of ∼1.0 m or more. To support this increase, they cited DP16 along with the expert-judgment assessment of Bamber and Aspinall (2013). However, the latter study gave a high-end (95^th percentile) estimate of 0.84 m SLR from the two ice sheets, less than Pfeffer et al. (2008). Other studies cited by S17 did not give independent evidence of a large AIS contribution. Thus, both the “High” projection of 2.0 m and the “Extreme” projection of 2.5 m in S17 relied on DP16’s claim that the AIS could contribute at least a meter of SLR by 2100."

The highlighted statement is a misstatement of fact. The accompanying footnote is also erroneous, as well as the part of the statement, on page 7, that references DeConto and Pollard (2016) in the context of Sweet et al. (2017), i.e., “(2) S17 and other reports that were published in 2016–2020 and relied on DP16 for the AIS contribution.”

Here is the relevant paragraph from page 14 in Sweet et al. (2017):

“The growing evidence of accelerated ice loss from Antarctica and Greenland only strengthens an argument for considering worst-case scenarios in coastal risk management. Miller et al. (2013) and Kopp et al. (2014) discuss several lines of arguments that support a plausible worst-case GMSL rise scenario in the range of 2.0 m to 2.7 m by 2100: (1) The Pfeffer et al. (2008) worst-case scenario assumes a 30-cm GMSL contribution from thermal expansion. However, Sriver et al. (2012) find a physically plausible upper bound from thermal expansion exceeding 50 cm (an additional ~20-cm increase). (2) The ~60 cm maximum contribution by 2100 from Antarctica in Pfeffer et al. (2008) could be exceeded by ~30 cm, assuming the 95th percentile for Antarctic melt rate (~22 mm/year) of the Bamber and Aspinall (2013) expert elicitation study is achieved by 2100 through a linear growth in melt rate. (3) The Pfeffer et al. (2008) study did not include the possibility of a net decrease in land-water storage due to groundwater withdrawal; Church et al. (2013) find a likely land-water storage contribution to 21st century GMSL rise of -1cm to +11 cm. Thus, to ensure consistency with the growing number of studies supporting upper GMSL bounds exceeding Pfeffer et al. (2008)’s estimate of 2.0 m by 2100 (Sriver et al., 2012; Bamber and Aspinall, 2013; Miller et al., 2013; Rohling et al., 2013; Jevrejeva et al., 2014; Grinsted et al., 2015; Jackson and Jevrejeva, 2016; Kopp et al., 2014) and the potential for continued acceleration of mass loss and associated additional rise contributions now being modeled for Antarctica (e.g., DeConto and Pollard, 2016), this report recommends a revised worst-case (Extreme) GMSL rise scenario of 2.5 m by 2100.”

In developing the report, we wished to provide a number of scenarios that could be used to fully bracket the evidence base for the physically possible 21-st century sea level rise, as well as providing expert judgement about the central tendency/best guess trajectory. These ranged from 0.3 m at the lowest of the lower bounds to 2.5 m at the uppermost of the upper bounds. Briefly, the motivation was to support as wide a possible range of decision contexts as existed at the time in coastal risk planning and management (e.g., see Hinkel et al., 2015, Nature Climate Change, and many others), including long-term adaptation pathways approaches and “stress test” type applications, both of which often use a “not-to-be-exceeded,” upper bound metric of performance.

As described in the quoted paragraph from Sweet et al. (2017), above, we arrived at the 2.5 m upper bound by synthesizing a number of lines of evidence from numerous studies, as well as the IPCC AR5, to individually interrogate the physically possible ranges of the contributing components to global-mean sea level rise. This was new evidence, and/or new synthesis of that evidence, since Pfeffer et al. (2008), the study that helped define the physically possible upper bound for a preceding U.S. interagency sea level rise scenarios report (Parris et al., 2012).

All of these studies predated the publication of DeConto and Pollard (2016); we had already decided on the 2.5 m upper bound, and completed most of the work of developing the global and regional scenarios, before that paper was published. As just one example, Kopp et al. (2014) estimated 2.45 m as the 99.9 percentile outcome for global-mean sea level rise in 2100 under RCP8.5. Once DeConto and Pollard (2016) was published, we added it to our citation list as another piece of evidence, but the conclusions of that paper had no influence on our choice of 2.5 m as the upper bounding scenario. The successor report to Sweet et al. (2017), i.e., Sweet et al. (2022), stated this clearly, as well (e.g., see page 11): “In Sweet et al. (2017), these scenarios were developed to span a range of 21st-century GMSL rise from 0.3 m to 2.5 m. Sweet et al. (2017) built these scenarios upon the probabilistic emissions scenario–driven projections of Kopp et al. (2014).”

The bottom line is that, if DeConto and Pollard (2016) had never been published, we would have written exactly the same report at the time that we wrote it.

In closing, I wanted to note that, on the initiative of one of the authors of this brief communication (DB), he and a number of others of us (including myself and Kopp, as well as DeConto) spent substantial time in productive discussions of the very points I have just summarized, and related topics, in the broader context of the nuances of using cutting-edge sea level rise science to support decision-making. These extensive discussions following the publication of Sweet et al. (2017) resulted in an AGU presentation by DB (see https://par.nsf.gov/servlets/purl/10066643), and a written summary of our engagement (see https://acwi.gov/climate_wkg/minutes/final_agu_consensus_statement_probabilisitic_projections_dec_2017.pdf), both of which reflected a useful integration of our diversity of perspectives as scientists and practitioners.

Actionable science calls for a higher standard for communication of science, including rating findings for reliability (as suggested by the authors). This is both because (a) even a great single paper typically only addresses a portion of earth system components and because (b) guidelines actually help articulate exceptions.

On the complexity of scientific findings vs. adaptation  

Presumably, we would not present decision-makers with non-peer-reviewed SLR estimates and expect them to decide its merit based on the specific decision context. If we accept the current social production of science which is “peer review” (also imperfect), a higher standard for “actionable science” is simply a logical extension.

Keller et al., 2021 (mentioned by Dr. Kopp) specifically discusses the importance of “Linking the Required Disciplines” in the management of climate risk. As an example: climate change introduces both direct impacts of SLR to coastlines and to enhanced weather extremes inland. The impact on weather extremes is not typically the purpose of any singular scientific paper strictly focused, for example, on the impacts of ice sheet loss on SLR.  To focus on the potential opportunity cost: If efforts are focused on coastlines now in response to extreme SLR scenario, what of the potential loss of funding to adapting to extreme weather scenarios inland? This problem is more acute in regions with fewer resources than discussed here (New York City), who may benefit from a clearer communication of the scientific consensus.

On guidelines and exceptions

Importantly, the authors do not advocate against limiting novel science, but rather avoiding its misrepresentation, i.e. “It is better to discuss such a claim, including the gaps in the evidence, than to disregard it.”.  In another comment, Dr. Kopp suggests (in summary) that it is critical to present the long tail and leave room for a dynamic response, even where evidence is lacking, based on the extent of potential risk (and the associated benefit of more extensive adaptation). I see no meaningful contradiction between the need for a guideline presented by the authors and the presence of exceptions. In fact, the “exceptionality” here is still defined in reference to some guideline and underlying rationale, thus underlining the need for the guideline.

The IPCC acts as a first, but possibly inadequate, level of synthesis. In fact, the purview of the IPCC has expanded over time precisely to accommodate evolving needs. A continued discussion of not just the “what” of actionable science, but also the “why” (the philosophical underpinning, as this paper explores) can inform precisely when it is important to break with consensus and to focus research into novel claims. This is particularly true because, while the presumed audience for IPCC predictions and adaptation recommendations is “decision-makers”, there is a far larger population which needs to understand the philosophy governing adaptation priorities (and has less time to examine footnotes in the IPCC and specific case studies): voters.

This short paper describes a framework for actionable science.  I am writing this comment as a glaciologists who also works broadly in the field of adaptation and science usability. I would really like to encourage the authors and editors to expand the paper to full length to give service to the ideas because in the short format currently there is much ambiguity and with significant room for mischief and harm.  

To start, the study of how knowledge systems are applied to decision making often goes under the name “science usability”, but I have also seen it referred to as “actionable science”.  There is substantial literature about the process of knowledge creation for planning and decision making.  Foundational to this field, is the concept that for knowledge to be applied in a decision making or planning concept requires credibility, salience and legitimacy (see, e.g., Cash et al., 2003).  To quote directly from Cash et al., (2003) “. . . . credibility involves the scientific adequacy of the technical evidence and arguments. Salience deals with the relevance of the assessment to the needs of decision makers. Legitimacy reflects the perception that the production of information and technology has been respectful of stakeholders' divergent values and beliefs, unbiased in its conduct, and fair in its treatment of opposing views and interests.”  

The definition of “actionable” that the authors present only includes what would be traditionally called “credibility”, omitting salience and legitimacy completely.  This is a fatal omission from a usability standpoint because there is abundant, credible literature that demonstrates the importance of salience and legitimacy (e.g., Cash, 2003).  To give two examples of salience, communities adapt to local sea level not global sea level and thus global sea level rise might have low salience and thus low usability for many decision makers.  Or century and longer sea level rise projections have little salience for, say 30 year home mortgages. Legitimacy is also important as it impacts the process and values of science creation and there are numerous examples where low legitimacy jeopardized science usability.

Clearly usability depends on the specific decisions and the community making decisions and, as noted by another commenter,  usability cannot be divorces from this context: actionable in one decision context does not mean actionable in a different decision context.  We also know that the way to increase the usability of knowledge is through co-generation, but that this is inefficient.  Usually it is knowledge brokers and boundary organizations that play the role of interpreters and translators of knowledge to users.  There is, again, abundant literature in this field and it is a shame to have no dialogue with the many previous studies that identify criteria AND the process of usable knowledge production.

My view is that the present study has *much* more value if this study  is reframed more specifically around credibility rather than more broadly about usability/actionable.  I would even recommend switching out the term "actionable" for "credible" to better reflect existing terminology.  Even here, I would have really like the authors to examine the role of values and tradeoffs in defining credibility, especially recognizing that scientific uncertainty can often be weaponized to delay action.

Determining if a study has multiple lines of evidence to support it is, as we can see from the comments, subjective and the authors have not demonstrated that their definition improves decision making.  As the current paper reads, it seems more like it is specifically designed as a refutal of DeConto and Pollard’s (2016) projections and the impact they have had as a high-end scenario in planning and adaptation decisions rather than as a general framework.  If that is the goal then the study should, perhaps be framed as such. 

To give some additional examples, the explosive disintegration of the Larsen B ice shelf and subsequent acceleration of tributary was not anticipated by any models nor was the sudden acceleration and retreat of Jakobshavn, Pine Island or Thwaites Glacier.  The disintegration of Conger ice shelf was also a surprise, although perhaps shouldn’t have been.   Going beyond glaciology, the pre-eminent physicists of the time Lord Kelvin famously said that “Heavier than air flying machines are impossible” a mere 8 years before the first airplane successfully flew (Shoemaker, 1995).  The Harvard Economic Society announced that “A severe depression like that of 1920-1921 is outside the range of probability” on November 16, 1929 (Shoemaker, 1995).  I could go on with other examples in which real world events failed to adhere to the academic consensus.  Here the point isn’t that scientists can be wrong (of course we are!), but that how communities, stakeholders and decision makers choose to incorporate information depends heavily dependent on values, resources and objectives.  Would we better off stress testing, strategizing or planning to  deal with low probability, high impact events?  I don’t know and I I would be loath for scientists to insist on being on the arbiter of these value laden decisions.  Actionable doesn't always mean infrastructure and it might mean longer term strategic thinking to be better positioned to incorporate new information as it becomes available.  Perhaps one useful framing for the authors is that their approach might be most suited to mega infrastructure investments that are expensive and take decades to plan and build.  Different criteria would be appropriate for different scales of intervention.

My penultimate point, illustrates the degree of mischief that is possible if terminology and definitions aren’t tightened up and illustrated with multiple case studies.  The authors argue that DeConto and Pollard’s study of sea level rise isn’t credible (actionable in their words) because there isn’t multiple lines of evidence to support ice cliff instability.  I suspect that the paleo-record results in more ambiguity than the authors concede here,  but nonetheless this is a fair point.  However, we can also say that there is zero evidence that the calving front will remain stationary and very little evidence to support *any* calving law used in ice sheet modeling.  By the authors same logic, we might conclude that *none* of the projections of sea level rise are credible (actionable in the authors words).  We can apply this same reasoning to other processes.  Should we doubt the entire enterprise of climate modeling because of the quasi-empirical treatment of clouds , precipitation and aerosols?  Clearly, this is not what the authors intend, but it might be an unintentional side effect if not clarified.  To this end, the IPCC, NOAA, NASA Sea level team and multiple organizations are involved in summarizing literature and play a well documented role in determining “credibility” and these organizations form one step in a chain of boundary organizations.  Here there needs to be dialogue with the role played by these existing structures and organizations and the interplay with other boundary organizations.    

My final point relates to the definition of “action”.  As noted by Knaggard (2014), one of the common actions that stakeholders take is decide that the scientific uncertainty is currently too large and devote resources to research to better quantify and hopefully reduce uncertainties.  By this definition, the action associated with high impact/low probability impacts might be more research (potentially coupled with adaptive decision making).  However, the most common action that decision makers take, however, is simply to do what is currently politically and technically feasible (Knaggard, 2014).  These decisions, based on political expediency and co-benefits depend more on the solution space than scientific uncertainty and so establishing credibility is less important than salience and legitimacy.

I’m very sympathetic to the authors goals, but I’m not sure that, to use the authors definition, their definition of "actionable" has multiple, independent lines of evidence to support its use and adoption.  I think it would be hard to demonstrate this in the limited space available, but I think it would be a great addition if the authors built their case through a larger number of case studies and literature review and applied their own criteria of multiple lines of evidence to their own proposed definition of actionable science. 

Refs

Cash, David W., et al. "Knowledge systems for sustainable development." Proceedings of the national academy of sciences 100.14 (2003): 8086-8091. And many more on usability!

Schoemaker, Paul JH. "Scenario planning: a tool for strategic thinking." MIT Sloan Management Review (1995).

Åsa Knaggård (2014) What do policy-makers do with scientific uncertainty? The incremental character of Swedish climate change policy-making, Policy Studies, 35:1, 22-39, DOI: 10.1080/01442872.2013.804175

All the best,

Jeremy Bassis

Comment on Lipscomb, Behar, Morrison Brief Communication

 By Judy Lawrence, Marjolijn Haasnoot, Robert Lempert

We are researchers and science brokers who work with SLR science in the context of decision making under uncertainty (DMDU). We recognize the challenge of dealing with multiple scenarios that are frequently updated as new understanding emerges. However, the proposal set out by LBM runs the risk that we wait until there is certainty before taking adaptation action. This would also not meet the precautionary test in the UNFCC which underpins the Paris Agreement, and lead to adaptation decision delay.

There are approaches that have emerged and assessed in IPCC AR6 for considering the dilemma that decision makers find themselves in as new science continues to emerge and decisions have to be made under uncertainty over the trajectory and pace of change. These DMDU approaches and methods are able to deal with large uncertainties and updated (climate) information. The LBM paper mentions adaptive planning and integrating SLR information into planning but does not elaborate what this means or how these approaches can address the problem of over and under investment in adaptation. This omission leads to a very directive solution that would not be decision relevant (salient nor legitimate).

Furthermore, we agree with Bassis's comment that the LBM paper only focuses on the matter of credibility and that salience and legitimacy are critical for decision making under uncertainty to enable local context to inform decisions. The response from LBM that the paper is not about how the science is used, misses the point that science does not sit in a vacuum outside how it may be used. In fact the authors have defined their problem with an example of how the projections were ‘misused’ in the San Francisco example. Science and its use are inextricably linked.

One cannot set a fixed criteria/standard for a changing context (both science and societal values-which decision makers weigh up). The question of the standard of the science cannot be divorced from a discussion about how sea-level rise projections are used. Using DMDU methods such as Dynamic Adaptive Pathways Planning (DAPP; Haasnoot et al. 2019) or Robust Decision Making (Lempert et al. 2019) to stress test adaptation options against a range of scenarios gives decision makers an idea of how sensitive each strategy is and then this information can be weighed against the other considerations that the decisionmakers must take into account as representatives of their communities, now and in the future. Given that surprises around polar ice processes and feedback are occurring, even high-end/low confidence scenarios are useful to remind decision makers that one cannot rule out such outcomes, as also stated in IPCC AR6. A DAPP approach enables adaptation to be broken down into near-term actions and mid-to long-term options to avoid lock in of investments that are costly to adjust in the future and which shift the adaptation costs to future generations. It helps to prepare and keep options open or create them through innovation and planning, allowing further adaptation if necessary. DAPP thus enables more robust decisions to be made and contingency actions to be ready.

Given the time it takes for long lived infrastructure projects to be designed and implemented,  especially for coastal adaptation, to high and rapid sea level rise, a precautionary approach has merit. Not considering such scenarios can run the risk of being too late or invest in the wrong measures resulting in high sunk costs and transfer costs. Considering them on the other hand gives decision makers greater confidence in a changing situation (science and societal values).

The nature of the decision process is lightly addressed in the paper beyond one example. A range of approaches as to how SLR projections can be used should be proffered. These can be found in the literature.

For example, in the Netherlands a group of experts (the Signal Group) advise the government on relevant new research which is then assessed for its potential implication and need for further research or actions (Haasnoot et al. 2018). The follow-up research from Deltares and KNMI lead to further assessment on the need to reassess the adaptation strategy. It also raised awareness of the long-term higher sea levels, even if they were in low global warming scenarios, and that the current adaptive plan would not be sufficient, and that transformative adaptation would be needed. While near-term actions were not changed immediately, it was recognised that preparations were needed to be able to further adapt, if such an accelerated SLR became a reality.

In New Zealand, revised national coastal hazards and climate change guidance (Ministry for the Environment 2024) has adopted a tailored approach for different types of decisions, using DAPP to stress test adaptation options with downscaled global scenarios that enable polar ice responses and vertical land movement to be incorporated into SLR which vary around the coast. Periodic revisions are undertaken to reflect new science and a precautionary approach is adopted considering at least a 100year timeframe to account for change and uncertainty. Specific infrastructure guidance is also available (Lawrence and Allison, 2024).

Adaptive pathways planning and monitoring for signals of is also done in practice in the Thames Estuary plan (Environment Agency, 2012; Ranger et al 2013) and New York (Blake et al., 2019; Rozenzweig). They also plan regular ongoing assessment and evaluation of changes (e.g. ongoing and every 6 six years in New York and in the Netherlands).

References

Environment Agency. (2012). TE2100 Plan. Managing Flood Risk Through London and the Thames Estuary.

Haasnoot, M., van ’t Klooster, S., & van Alphen, J. (2018). Designing a monitoring system to detect signals to adapt to uncertain climate change. Global Environmental Change, 52, 273–285. https://doi.org/https://doi.org/10.1016/j.gloenvcha.2018.08.003

Haasnoot, M., Warren, A., & Kwakkel, J. H. (2019a). Dynamic Adaptive Policy Pathways (DAPP) - Decision Making under Deep Uncertainty: From Theory to Practice (V. A. W. J. Marchau, W. E. Walker, P. J. T. M. Bloemen, & S. W. Popper, Eds.; pp. 71–92). Springer International Publishing. https://doi.org/10.1007/978-3-030-05252-2_4

Lawrence, J., & Allison, A. (2024). Guidance on adaptive infrastructure decision-making for addressing compound climate change impacts. https://deepsouthchallenge.co.nz/wp-content/uploads/2024/04/Guidance-on-adaptive-decision-making-for-addressing-compound-climate-change-impacts-for-infrastructure.pdf

Lempert, R. (2013). Scenarios that illuminate vulnerabilities and robust responses. Climatic Change, 117(4), 627–646. https://doi.org/10.1007/s10584-012-0574-6

Ministry for the Environment. 2024. Coastal hazards and climate change guidance. Wellington: Ministry for the Environment. Pub. 1805 https://environment.govt.nz/assets/publications/Coastal-hazards-and-climate-change-guidance-2024-ME-1805.pdf

Ranger, N., Reeder, T., & Lowe, J. (2013). Addressing `deep’ uncertainty over long-term climate in major infrastructure projects: four innovations of the Thames Estuary 2100 Project. EURO Journal on Decision Processes, 1(3), 233–262. https://doi.org/10.1007/s40070-013-0014-5