ATAT 1.1, the Automated Timing Accordance Tool for comparing ice-sheet model output with geochronological data

Ely, Jeremy C.; Clark, Chris D.; Small, David; Hindmarsh, Richard C. A.

doi:https://doi.org/10.5194/gmd-12-933-2019

GMD | Articles | Volume 12, issue 3

Geosci. Model Dev., 12, 933–953, 2019
https://doi.org/10.5194/gmd-12-933-2019
© Author(s) 2019. This work is distributed under
the Creative Commons Attribution 4.0 License.

GMD | Articles | Volume 12, issue 3

Methods for assessment of models 12 Mar 2019

Methods for assessment of models | 12 Mar 2019

ATAT 1.1, the Automated Timing Accordance Tool for comparing ice-sheet model output with geochronological data

Jeremy C. Ely et al.

Download

Final revised paper (published on 12 Mar 2019)
Supplement to the final revised paper
Discussion paper (discussion started on 14 Feb 2018)
Supplement to the discussion paper

Interactive discussion

Status: closed

AC: Author comment | RC: Referee comment | SC: Short comment | EC: Editor comment

- Printer-friendly version

- Supplement

RC1: 'Review of Ely et al', Evan Gowan, 12 Mar 2018
RC2: 'Review of ATAT 1.0, 1 an Automated Timing Accordance Tool .., Jeremy Ely et al', Lev Tarasov, 23 Mar 2018
AC1: 'Author response to reviewers', Jeremy Ely, 10 May 2018

Peer review completion

AR: Author's response | RR: Referee report | ED: Editor decision

AR by Jeremy Ely on behalf of the Authors (14 May 2018) Author's response Manuscript

ED: Referee Nomination & Report Request started (18 Jun 2018) by Didier Roche

RR by Lev Tarasov (06 Jul 2018)

Suggestions for revision or reasons for rejection

Though the revised submission and softwared is an improvement, there
remains a core problem. As detailed below, there is a fundamental
contradiction in the paper that needs to be addressed. The authors
rigorously defend their inclusion of discussion on model and data
uncertainties but then ignore these uncertainties in their
metrics. For model calibration in the context inferring past ice sheet
evolution, a metric needs to take into account all uncertainties,
otherwise the resulting inferences will be invalid. And even for the
context of just data/reconstruction comparison, inferential
uncertainties (measurement, dating, downscaling). need to be accounted
for. The two RMSE metrics (equations 1 and 2) could easily be
modified to account for observational and downscaling uncertainties.
The user also needs to be provided with clear simple instructions for
modifying the metric to do so to insert their own estimates for
structural uncertainty. Without these corrections, I suspect ATAT will
foster invalid model/data based inferences and thereby do a
disservice to the community.

# detailed comments : responses and revised text

In reality, with a high-resolution ice-sheet model (say 5 km) it is
unlikely that two equally reliable dates will be contained within a
cell – radiocarbon in a core for example should just use the date that
is oldest, closest to the glacial contact.
# The expense of 5 km resolution paleo ice sheet models precludes
# their current useage for the large ensembles needed for paleomodel
# calibration. Also, in regions with high-topographic variance (eg
# most of the Greenland margin), there may easily be two relevantly
# valid dates within 5 km proximity at different elevations.

Tarasov et al. 2012 run ice sheet models that are not independent of
the dated chronology (there is a margin raster, their Fig 2, which
“nudges” the ice sheet into place based upon Dykes reconstruction).
# Not clear what you mean by "independent", the margin chronology
# (with uncertainties) is used for nudging the surface mass-balance
# within climate forcing uncertainties (nudging isn't unbounded), but
# the amount of nudging then goes into the cost function for the
# calibration.

>If this tool is meant to be used by those doing model calibration
>against paleo observations, then model uncertainty and downscaling
>error needs to be explicitly accounted for in the metric.

We have shortened the length of this discussion, but retain the
section as we think that understanding the uncertainty of the model is
important when comparing to data. Uncertainty handling will come with
ensemble design, the tool asks which ensemble member fits the data
best.

# The authors never respond to my concern above about model
# uncertainty (ie structural uncertainty). They thereby also
# contradict their own stated requirement in the text: "In order
# maximise the 68 benefit to all users, any comparisons between
# palaeo-ice sheet model output and empirical data should ideally 69
# consider the inherent uncertainties of both."

# Furthermore, if the user of ATAT is going to rely on either the
# original or current descriptions of dating and modelling uncertainty
# as their main source of understanding these critical issues, then
# the paper will be doing a disservice to the community. The main
# message should be that users need to invest the time to really
# understand these uncertainties or include a collaborator who does.
# The provision of a set of in depth appropriate references on this
# topic would therefore be of much more use than the current or past version
# cursory discussions.

Uncertainty handling will come with ensemble design, the tool asks
which ensemble member fits the data best.
# The first phrase is incorrect and the 2nd means the tool is useless
# for model calibration in the context of inferring past ice sheet evolution.
# Read Rougier, 2007 to understand what structural uncertainty means and
# why it has to be included in the likelihood function. The 2nd phrase
# what are the comparative fits of each ensemble member to the constraint
# data given all uncertainties.

It is also important keep the agree/disagree metric for the
following reason: you may do 100 simulations of a palaeo ice sheet and
keep getting the same sites that disagree.
# I don't follow the logic here. What does agree/disagree mean? Your
# responses seem to put a lot of emphasis on user choice, but here
# you are imposing a choice on what level of misfit constitutes disagreement
# instead of having the user apply their "expert judgement"...

Three classes of data are of particular use for constraining
palaeo-ice sheets; 46 (i) geomorphological data, (ii) relative sea
level history, and (iii) geochronological data
# This list is too limited. Present-day rates of vertical uplift are
# also a powerful constraint (cf my 2012 paper) for the North American
# and Eurasian ice sheets. So change RSL -> geophysical constraints
# (including RSL and present-day vertical velocities).

applying offsets derived from ice core records to contemporary climate
(Hubbard et al., 2009) and scaling 202 between present-day conditions
and uncoupled global-circulation-model simulations at maximum glacial
203 conditions (Gregoire et al., 2012; Gasson et al., 2016).
# The usual standard for example references is either a recent detailed
# review or first use. The above citations do not follow either logic.

Since all dating 390 techniques only record the absence of ice,
geochronological data provides only a one-way constraint on palaeo-391
ice sheet activity. For deglacial ages, deglaciation could occur any
time before the geochronological data provided 392 and within the
error of the date,

# Incorrect. As detailed:
Figure 3
# I am concerned about the metric indicated in this figure. It seems
# to indicate that a model with say last retreat for a given grid cell
# at say 30 ka, will be given the same score for this site as for a
# model that retreats ice at say 19 ka (for the given sample date of
# 18 +/- 1 ka). If my interpretation is correct, this needs to be
# remedied. This should be obvious for eg cosmogenic dates where
# inheritance will make the date if anything too old (and therefore
# cosmogenic dates may be maximum limiting depending on type of sample
# and location). But even for C14 dates, where the issues of sample
# availability, time required for in-migration of plants,.. mean that
# the dates are generally minimum limiting, I can't see anyone saying
# that a model with a 12 kyr misfit with a minimum limiting age should
# score the same as a model that fits the sample within sample age
# uncertainties.

# equations 1 and 2 are highly problematic given that observational
# and model uncertainties are ignored. And this again contradicts the
# authors own statements about the importance of these uncertainties.

Hide

RR by Evan Gowan (30 Jul 2018)

ED: Reconsider after major revisions (22 Aug 2018) by Didier Roche

AR by Jeremy Ely on behalf of the Authors (01 Oct 2018) Author's response

ED: Reconsider after major revisions (19 Oct 2018) by Didier Roche

Dear Jeremy Ely and co-workers,

I have read in depth your manuscript, the concerns expressed by reviewer #2 and your response to reviewers.

I still find two aspects problematic:

1-/ Reviewer #2 highlighted that the RMSE metrics on which you base the ATAT tool needs to include for observational and downscaling uncertainties. Though you discuss at length in your response to reviewers the different sources of uncertainties and their origin, you did not reply to this specific request in positive or negative. I am sure the reviewer is well aware that structural uncertainty is present in any modeling work so this is not the point for discussion. You advocate for the user of your tool to have run ensemble experiments of their ice-sheet model which I understand, but how and where does these ensemble runs come into the uncertainty in your tool? Do you consider running your tool for each of the member independently and accounting for such uncertainty a posteriori? But how? I feel that the request of reviewer #2 in this context regarding equations 1&2 is a valid one, unless I miss part of the reasoning of course.

2-/ In the new paragraphs that have been implemented in your manuscript I read:
"Structural uncertainty is related to parametric uncertainty, but has a broader remit, and is defined here as all uncertainty that arises within a model due to a lack of physical understanding of the system in question." pp line and following. Does this means you have, for this particular manuscript, specific definition of structural uncertainty which is not the common one? Could you highlight the reason for doing so and what is the exact difference with the common definition?

Best wishes,
Didier ROCHE

Hide

AR by Jeremy Ely on behalf of the Authors (14 Nov 2018) Author's response Manuscript

ED: Publish as is (25 Feb 2019) by Didier Roche

Journal metrics

Journal metrics

Abstracted/indexed

ATAT 1.1, the Automated Timing Accordance Tool for comparing ice-sheet model output with geochronological data