Interactive comment on “ BRICK v 0 . 1 , a simple , accessible , and transparent model framework for climate and regional sea-level projections ”

The submitted manuscript, “BRICK, a simple, accessible, and transparent model framework for climate and regional sea-level projections,” presents a good example for other geophysical modelers to follow. The authors describe a modular and transparent framework for projecting changes in regional sea level under different uncertain future scenarios, and they also give an example of how the framework can be used to plug in other modules enabling decision support based on the climate model outputs. While the flood risk management example is simplistic, it is illustrative of the potential for the BRICK model to be leveraged in a variety of useful applications. Further, the authors make a nice case for the value and importance of open-source, transparent, and simple modeling.

Computational constraints often impose hard trade-offs between physical model complexity and statistical model complexity.For example, a sizable allocation of computational time could be spent running a small number of simulations using a high-complexity physical model.Highly detailed simulations are useful to better understand the complex system, but with just a small number of simulations, only weak ensemble statistics can be drawn.In contrast, numerous realizations of a less detailed physical model could be run.This would provide the opportunity for more advanced ensemble statistical techniques including the characterization and quantification of uncertainties.It is important in climaterelated applications such as mitigation of greenhouse gas emissions or adaptation to sea-level rise that the relevant un-certainties are explored and communicated clearly to policymakers (e.g., Garner et al., 2016;Gauderis et al., 2013;Goes et al., 2011;Hall et al., 2012;Lempert et al., 2004).
Several studies have broken important new ground in tackling these challenges.For example, Nauels et al. (2017) present a platform of sea-level emulators (i.e., simple models of complex models) that efficiently produces future projections and characterizes key model structural uncertainties using statistical calibration methods.Semi-empirical modeling (SEM) approaches trade detailed physics for a model that can efficiently project sea level using statistical, but mechanistically motivated, relationships between sea-level changes and climate conditions such as temperature and radiative forcing (Grinsted et al., 2010;Jevrejeva et al., 2010;Kopp et al., 2016;Rahmstorf, 2007).Recent work has expanded upon the SEM approach to use simple models to resolve individual contributions to global sea level (Bakker et al., 2017;Mengel et al., 2016;Nauels et al., 2017).
Studies based on simple, mechanistically motivated models have the potential to be transparent and reproducible when presented in open platforms and when the underlying data are readily available.Yet, although there is an increasing tendency to share scientific code, it can be (perhaps surprisingly) hard to get the models running and to reproduce the results.A likely cause of this is that not enough attention is given to the scientific coding itself.Careful coding, documentation, and review require a dedicated commitment of time, but scientific incentives to do so can be weak.
Here we describe in detail BRICK (Building blocks for Relevant Ice and Climate Knowledge, Bakker et al., 2017) v0.2, a model framework that focuses on accessibility, transparency, and flexibility while maintaining, as much as possible, the computational efficiency that makes simple models so appealing.As compared to Bakker et al. (2017), BRICK v0.2 accounts for land water storage with the other components kept unchanged.There is a wide range of potential applications for such a model.A simple framework enables uncertainty quantification via statistical calibration approaches (Higdon et al., 2004;Kennedy and O'Hagan, 2001), which would be infeasible with more computationally expensive models.A transparent modeling framework enables communication between scientists as well as communication with stakeholders.This leads to potential application of the model framework in decision support and education (Fischbach et al., 2012;Johnson et al., 2013;Weaver et al., 2013).The present work expands on previous studies by (i) providing a platform of simple but mechanistically motivated sealevel process models that resolve more processes, (ii) providing a model framework that can facilitate model comparisons (for example, between our models and those of Nauels et al., 2017), (iii) exploring combined effects of key structural and parametric uncertainties, (iv) explicitly demonstrating the flexibility of our framework for interchanging model components, and (v) explicitly demonstrating the utility of our model framework for informing decision analyses.
In this model framework, we present a set of existing, well-tested, and easy-to-couple simple models for climate and flood risk management.They simulate global mean surface temperature and contributions to global mean sea-level rise.BRICK also includes a regional sea-level rise module, which translates the global mean sea-level contributions to regional sea level at a user-defined location.We use these regional sea-level projections to demonstrate how the physical model may be linked to decision-making and impacts.We implement a Bayesian calibration approach with an aim to adequately resolve the tails of the distribution of future sea levels because these low-probability areas represent highrisk events.In robust decision-making approaches, it can be favorable to be underconfident as opposed to overconfident, e.g., by applying conservative estimates in the sense of being risk-averse (Herman et al., 2015).We hence include in our Bayesian approach wide, mechanistically motivated prior parameter probability distributions (Bakker et al., 2017).Yet, the flexibility of the BRICK model framework also enables the implementation of other calibration schemes.This paper is intended to showcase a useful model framework that is attractive for a sustainable approach to model development, for example by inspiring fellow researchers to contribute to the framework, to rethink their coding practice, and maybe even to adopt some of the demonstrated design objectives in their future research proposals.
The hindcast skill of the BRICK model has been previously demonstrated (Bakker et al., 2017).Thus, the present work focuses on outlining a set of epistemic modeling values that we believe facilitates advances in the modeling community.The remainder of this work is organized as follows.In Sect.2, we describe these values and the ways in which the BRICK model implementation strives to attain them.Section 3 contains an overview of the BRICK model components for climate and the contributions to sea-level rise.Section 4 describes and presents the results of a set of model experiments conducted to demonstrate how BRICK lives up to our epistemic modeling values.Section 5 summarizes the findings of this work and provides conclusions and guidance for future work.

Model design
The essence of the BRICK physical model is to simulate changes in global mean surface temperature and sea level, in response to perturbations in radiative forcing.The socioeconomic impacts of the simulated temperature and sea-level changes may then be assessed.This is depicted in Fig. 1.The climate component, each individual contribution to global sea-level rise, and an impacts module are sub-models of BRICK, or "BRICKs."We defer details of the specific submodels to Sect. 3. The physical model (climate and sea-level  (Meinshausen et al., 2011a); TE is the Thermal Expansion model (Grinsted et al., 2010;Mengel et al., 2016); SIMPLE is the Simple Icesheet Model for Projecting Large Ensembles (Bakker et al., 2016); ANTO is the ANTarctic Ocean temperature model; DAIS is the Danish Center for Earth System Science Antarctic Ice Sheet model (Shaffer, 2014); regional sea-level fingerprinting downscales from global sea-level contributions to regional (Slangen et al., 2014); and the model of Van Dantzig (1956) assesses flood risk.rise) components of BRICK are intentionally simple.This choice is guided by the epistemic modeling values outlined below.

Accessibility
We selected R (R Core Team, 2016) as the base language for BRICK because it is (i) stable, (ii) freely available and open source, (iii) relatively easy to use, and (iv) easy to call subroutines written in faster languages.In the BRICK source code accompanying this study, the physical submodels within the climate and sea-level rise modules are all provided as both R and Fortran 90 routines.It is our aim that the full physical-statistical model of BRICK will be accessible using a modern laptop.This means that sizable Monte Carlo simulations (on the order of a million samples) must be possible on a timescale of hours.This is made possible by calling Fortran 90 sub-models from the base code in R.
In addition to conceptual accessibility, it is our view that useful model codes should be physically accessible too.Openness with scientific codes is likely to lead to higher quality codes (Easterbrook, 2014).In an effort to be truly open source and freely available, all codes -including the physical model, statistical model, and processing and plotting scripts used for the results shown here -are available through a download server as well as the Github repository provided in the Code Availability section of this article.Providing all code and data necessary to recreate this study is a critical component of reproducible research (Murray-Rust and Murray-Rust, 2014) and can help to build trust between the general public and the scientific community (Easterbrook, 2014;Grubb and Easterbrook, 2011).

Transparency
We aim to achieve transparency in two areas: the physical modeling, including the related model code, and the communication of scientific findings.
T. E. Wong et al.: BRICK v0.2, a simple, accessible, and transparent model framework With regards to transparent physical modeling, we use simple numerical integration schemes whenever possible.We use as few global variables as possible, in order to "write programs for people, not computers" (Wilson et al., 2014).The essence of these authors' advice is that users should not be expected to remember more than a few pieces of information as they read and develop code.To this end, in BRICK we aim to give appropriately suggestive names to our variables within the code, such that a human intuitively understands what the quantity at hand represents.For example, when naming a logical or Boolean variable, we prefer for its name to read as a question that the variable itself answers, and begin the variable name with the letter "l" to imply it is a "logical" variable.One example of this in the BRICK source code is the variable "l.project", which is true when the model is configured to make projections of future sealevel rise and climate, and false when the model is set up for hindcast simulations.While it may seem fussy to review these points, practices such as this will facilitate the sharing of scientific codes and enable the community to build stronger and more efficient collaborations.
Transparency also serves to link the findings of a physical model to decision-making and policy impacts.BRICK can be a useful tool to link climate changes (global temperature and sea-level rise) to decision-making frameworks through a clear outlet for coupling to socioeconomic models.Perhaps most importantly, the coupled physical-statistical framework in BRICK incorporates many sources of uncertainty into the physical findings on which the decisions will be based.It is important that these uncertainties in climate projections are represented in the decision-making framework (Lempert et al., 2004).

Flexibility
A modular programming approach is taken with BRICK, which allows each component sub-model to be exchanged for alternative models.In this way, as the scientific forefront progresses, the BRICK sub-models may advance as well.The flexible BRICK framework also permits a quantitative evaluation of model structural differences, which is valuable in the event that it is unclear which of two candidate models should be chosen.In these cases, the BRICK framework is valuable for model comparison and quantification of structural uncertainty.As new data sets for the calibration of the sub-models become available, these can also be incorporated instead of or in addition to the current data sets.We demonstrate the flexibility of the BRICK framework through a series of modeling experiments (Sect.4).

Efficiency
Code efficiency is enabled primarily through (i) the use of simple models and (ii) using model versions written in R for easy preliminary experimentation, and Fortran 90 versions for production simulations.This practice also follows the advice of Wilson et al. (2014) for code developers to "write code in the highest-level language possible, and shift to lower-level languages like C and Fortran only when they are sure the performance boost is needed."This boost indeed enables the generation of production simulations on most modern laptops.The simulation of 1 million model iterations spanning from 1850 to the present, performed on each of four CPUs (two cores and two threads per core), yields an ensemble of 4 million model realizations.This procedure requires less than an hour on a model year 2012 laptop with a 2.9 GHz dual-core processor with 16 GB of RAM.Paleoclimatic simulations require longer wall clock times, but can still be completed in less than a day.All simulations for this study were completed on this machine.
Providing computationally efficient code simplifies its use.For example, there may be limitations on the computing resources allocated for a particular project, or an instructor might be interested in enhancing coursework by incorporating computer modeling exercises.In these cases, transparency is critical (as mentioned above), but the model must also be sufficiently efficient that it neither (i) expires the computational allotment for the experiment nor (ii) takes too long to be of any educational use.Our epistemic modeling values of accessibility, transparency, flexibility, and efficiency motivate the choice of a relatively simple physical modeling framework.Accordingly, a detailed statistical calibration framework is implemented.Within this framework, physical model and statistical model parameters are calibrated using observational data sets and mechanistically motivated prior ranges.The statistical model is reviewed at greater length by Bakker et al. (2017), so we provide only an overview in Sect.4.1.

Code review and sharing
We invite the readers to download and test our code, as well as provide feedback on how best to further develop BRICK to fulfill the four epistemic values outlined above.Frequent and thorough code review by other team members as well as outside agents is another critical step towards good scientific coding practices (Wilson et al., 2014), and "peer review needs to be supplemented with a number of other mechanisms that help to establish the correctness and credibility of scientific research" (Grubb and Easterbrook, 2011).Wilson et al. (2014) also note that a number of high-profile research articles have been retracted or revised because of errors in the code.The likelihood of these errors may be greatly reduced by thoroughly testing other group members' codes.In our own experience conducting the experiments described in this study, we have anecdotal evidence for the value of testing one another's code.Some errors were corrected through this process, and many more pieces of code were modified for clarity.We continue to invite all comments and suggestions for improvements and modifications (to the corresponding author).
The use of a version control system greatly expands the accessibility of a code base, and also facilitates continuous improvement of the modeling framework itself.This is true and useful before, during, and after the peer-review process.Mistakes are inevitable and we assume that BRICK still contains some minor errors, ambiguities, and pieces of code that do not fully comply with our own standards.Openly sharing the code and documentation will help to address these issues.It is our hope that BRICK may be further developed as a community modeling tool, and that other users may contribute to the framework through added or revised models and data, or improved functionality.The use of a version control system facilitates this type of community effort (Wilson et al., 2014).

Global mean climate
We adopt DOECLIM (Diffusion Ocean Energy balance CLIMate model, Kriegler, 2005) as a starting point for a simple climate model (Fig. 1).DOECLIM is a zerodimensional energy balance model coupled to a three-layer, one-dimensional diffusive ocean model.The DOECLIM physical model outputs are global mean surface temperature anomaly ( • C) and ocean heat uptake (10 22 J).Calibration data for DOECLIM include both global surface temperature (Morice et al., 2012) and ocean heat uptake (Gouretski and Koltermann, 2007).We use a 1-year time step for the DOECLIM model, and the required input to drive the model is the radiative forcing time series (W m −2 ).This forcing is partitioned into aerosol and non-aerosol components, to enable a representation of the uncertainty associated with these forcings.The BRICK model considers this as an uncertain model parameter denoted as the aerosol forcing scaling factor (α DOECLIM ).This aerosol scaling factor has been used elsewhere in the literature (Urban et al., 2014;Urban and Keller, 2010) and accounts for some uncertainty in the radiative forcing of aerosols (Meinshausen et al., 2011b).The interested reader is directed to Kriegler (2005) and Tanaka and Kriegler (2007) for more information about the DOECLIM model.
We fit a first-order autoregressive (AR1) error model to the model-data discrepancy between temperature and ocean heat uptake model output and calibration data.We estimate the first-order lag autocorrelation parameters (ρ T and ρ H ) and homoscedastic component of the AR1 innovation variance (σ T and σ H ) within the calibration framework as statistical model parameters.We add the heteroscedastic observational error estimates for global mean surface temperature from Morice et al. (2012) and for ocean heat uptake from Gouretski and Koltermann (2007) in quadrature to σ T and σ H (respectively) for the complete heteroscedastic temperature and ocean heat uptake error estimates.The model calibration approach implemented here assumes normally distributed model-data residuals with mean 0 (Higdon et al., 2004).The AR1 error model has the effect of "whitening" the residuals to satisfy this assumption.This type of calibration approach for DOECLIM has been implemented previously in the literature (Urban et al., 2014;Urban and Keller, 2010), and we direct the interested reader to these studies for further details.

Sea-level components
The BRICK global mean sea-level module calculates global sea-level change as the sum of four individual components: glaciers and ice caps (GIC), the Greenland Ice Sheet (GIS), the Antarctic Ice Sheet (AIS), and thermal expansion (TE).These component sub-models are described in the following sections.BRICK accounts for land water storage contributions to global mean sea level using mass balance trends from the International Panel on Climate Change (IPCC) Fifth Assessment Report (AR5, Church et al., 2013) and from the work of Dieng et al. (2015).The differential equations for the GIC, GIS, AIS, and TE contributions to global mean sea level are integrated into BRICK using first-order numerical integration schemes with a 1-year time step.Initial conditions are specified at a year dictated by the sub-model's assumed reference point.This differs, in general, among the sub-models, and some model parameters depend on preserving this reference year.Starting from this initial condition, a first-order explicit numerical integration method integrates forward in time to the end of the simulation and a first-order implicit (backward differentiation) method integrates backward in time to the earliest year of the simulation.Preliminary experiments (not shown) demonstrated that the 1-year time step is sufficiently short to maintain numerical stability.The total global mean sea-level rise from the coupled BRICK model is where S(t) is the global mean sea level (m) in year t, S GIC is the sea-level contribution from GIC (m), S GIS is the sea-level contribution from the GIS (m), S AIS is the sea-level contribution from the AIS (m), S TE is the sea-level contribution from thermal expansion (m), and S LWS is the sea-level contribution from changes in land water storage.We report projections of future sea level relative to the 1986-2005 mean.
T. E. Wong et al.: BRICK v0.2, a simple, accessible, and transparent model framework

Glaciers and Ice Caps
We adopt a simple zero-dimensional sub-model for the contribution to global sea-level rise from Glaciers and Ice Caps (GIC) from Wigley and Raper (2005).This same formulation is used in the MAGICC climate model (Meinshausen et al., 2011a).The parameterization for the GIC contribution to global sea-level rise is In Eq. ( 2), S GIC is the cumulative sea-level contribution from GIC (m), β 0 is the initial mass balance sensitivity to global temperatures (m • C −1 yr −1 ), T eq,GIC is the theoretical equilibrium temperature at which the GIC mass balance is at steady state ( • C), V 0,GIC is the initial total volume of GIC available in 1990 (m sea-level equivalent (SLE)), and n is an exponent parameter for area-to-volume scaling.An initial condition, S 0,GIC , is provided as an uncertain model parameter.T eq,GIC is taken equal to −0.15 • C (Wigley and Raper, 2005).Note that in this formulation for GIC contribution to sea-level rise, whether the GIC mass is increasing or decreasing depends only on T g (t) relative to T eq,GIC ; it is independent of the current state S GIC (t).Within this model for the GIC sea-level contribution, T g is relative to the 1850-1870 mean global surface temperature (Wigley and Raper, 2005).The uncertain physical model parameters for GIC-MAGICC (which will be tested in Sect.4.2) are β 0 , V 0,GIC , S 0,GIC , and n.We fit an AR1 model to the model-data discrepancy between GIC model output and calibration data (Dyurgerov and Meier, 2005) in the same manner as the temperature and ocean heat uptake calibration (Sect.3.1).Uniform prior distributions are used for the GIC-MAGICC physical and statistical model parameters.These prior distributions, as well as calibrated posterior medians, 5, and 95 % quantiles, are given in Appendix A.

Greenland Ice Sheet
BRICK uses the mechanistically motivated, zerodimensional SIMPLE (Simple Ice-sheet Model for Projecting Large Ensembles) model as the parameterization for the Greenland Ice Sheet (GIS) contribution to global mean sea-level change (Bakker et al., 2016).SIMPLE estimates the GIS response to changes in global mean surface temperature by first estimating an equilibrium ice sheet volume (V eq, GIS , m SLE) at which the sea-level contribution from the GIS is 0, and estimating the e-folding timescale of GIS volume changes due to changes in global temperature (τ GIS , yr −1 ).
In Eqs. ( 3) and ( 4), c GIS , b GIS , α GIS , and β GIS are uncertain physical model parameters.c GIS is the sensitivity of the equilibrium volume to changes in temperature (m SLE • C −1 ); b GIS is the equilibrium volume V eq, GIS for 0 temperature anomaly (m SLE); α GIS is the sensitivity to temperature of the timescale of GIS volume response to changes in temperature ( • C −1 yr −1 ); and β GIS is the equilibrium (T g = 0 • C) timescale of GIS volume response to changes in temperature (yr −1 ).Global mean surface temperature, T g , is taken relative to the 1961 to 1990 mean.The GIS volume changes can then be written as The initial condition V 0,GIS is provided as an uncertain model parameter (m SLE).Using this initial condition, designated in the year 1961, the sea-level rise due to the GIS is calculated as the change from V 0,GIS to the current GIS volume, V GIS (t).This formulation, of course, assumes that all GIS volume lost makes its way into the oceans.An AR1 model is fitted to the GIS model-data residuals.Due to poor convergence, the first-order lag autocorrelation parameter (ρ GIS ) is held constant at a value determined by a preliminary model simulation that is optimized using a differential evolution optimization algorithm (Storn and Price, 1997).The GIS training data set does not provide heteroscedastic error estimates, so the AR1 innovation variance is taken to be the estimated statistical parameter σ GIS added in quadrature to the provided error estimate (Sasgen et al., 2012).All GIS physical and statistical model parameters are assigned uniform prior distributions.The ranges for these priors and posterior distribution medians, 5, and 95 % quantiles are given in Appendix A. Further details regarding SIMPLE area provided in Bakker et al. (2016).

Antarctic Ice Sheet
We employ the Danish Center for Earth System Science Antarctic Ice Sheet (DAIS) model to simulate the Antarctic Ice Sheet contribution to global sea level (Shaffer, 2014).This is a two-dimensional model for the Antarctic Ice Sheet that assumes an axisymmetric geometry, shown graphically in Shaffer (2014) When calibrated as a stand-alone model, the DAIS forcings are provided based on temperature reconstructions (see Shaffer, 2014).When the DAIS model is run as a component in the coupled BRICK model, a separate sub-model is needed to convert the global mean surface temperature from the climate model (DOECLIM) to the Antarctic surface and ocean subsurface temperatures required by the DAIS model.The Antarctic surface temperature is estimated from a linear regression with global mean surface temperature (Morice et al., 2012;Shaffer, 2014).The Antarctic Ocean temperatures (T ANTO ) are modeled through a simple relation with the global mean surface temperature, T g (relative to the 1850-1870 mean).T ANTO is bounded below at the freezing point of salt water (T f = −1.8• C): Equation ( 6) is a modified linear regression between the global mean surface temperature T g and the Antarctic Ocean temperature T ANTO , such that the Antarctic Ocean temperature is bounded below by the freezing temperature of seawater, T f .In Eq. ( 6), a ANTO is the sensitivity of the Antarctic Ocean temperature to global mean surface temperature (unitless), and b ANTO ( • C) is the approximate Antarctic Ocean temperature for T g = 0 • C. b ANTO is the approximate ocean temperature because the relationship in Eq. ( 6) is not a simple linear regression.a ANTO and b ANTO are both estimated as uncertain model parameters.The DAIS model contains 11 physical parameters and 1 statistical parameter, for a total of 14 Antarctic Ice Sheet parameters to be estimated.The heavily parameterized Antarctic Ice Sheet module reflects our focus on including a broad range of model and observational uncertainties and considering the critical role of the Antarctic Ice Sheet in driving substantial uncertainty in future sea levels (Church et al., 2013).
Here, we use an updated and corrected version of the DAIS model (Ruckert et al., 2017;Shaffer, 2014).In the original formulation of the DAIS model, the input forcing from year t is used to determine the AIS contribution to sea-level rise in year t.This implicit numerical scheme assumes sea level and temperatures for the current year are known during that model iteration.For this study, in which temperatures and sea level originate in other BRICK model components, the DAIS model is re-cast using an explicit numerical scheme.The sea level and temperatures from the year t − 1 are used to calculate the AIS contribution in year t.Antarctic shoreaverage local mean sea level functions as the input to DAIS when run as a sub-model of the coupled BRICK model.This is estimated as described in Sect.3.3.
The dynamical core of the DAIS model is more detailed than the GIC, GIS, and TE emulators given above.For this reason, we do not undertake a full review of the model equations here.The interested reader is directed to Shaffer (2014) and Ruckert et al. (2017) for further details regarding the DAIS model and its hindcast forcings.Eq. (3) of Shaffer (2014) is the main equation of state for the Antarctic Ice Sheet volume (V AIS , m 3 ): In Eq. ( 7), B tot is the total rate of accumulation of mass on the Antarctic Ice Sheet (m 3 yr −1 ), T A is the Antarctic surface temperature reduced to sea level ( • C), S is the sea level (m), R is the Antarctic Ice Sheet radius (m), and F is the ice flux at the grounding line (m 3 yr −1 ).Following Shaffer (2014), we take the present sea-level equivalent Antarctic Ice Sheet volume to be 57 m SLE, and the initial ice sheet volume (V 0,AIS , m 3 ) to be consistent with an initial ice sheet radius of 1.86 × 10 6 m.Thus, the Antarctic Ice Sheet contribution to global sea level may be calculated as

Thermal expansion
BRICK uses a simple parameterization for the contribution of thermal expansion (TE) of the Earth's oceans to sea-level rise.We make the simplifying assumption that thermal expansion of the oceans occurs uniformly around the globe.While this is, of course, not strictly true, the next obvious step forward in model complexity would be to use a vertically and latitudinally resolved model for thermal expansion, incorporating the DOECLIM model output for ocean heat uptake.This two-dimensional ocean model is beyond the scope of the simple model framework described presently, but is an excellent subject for future work.Here, we employ a simple zero-dimensional thermal expansion emulator based on the parameterizations of the sea-level rise sub-models of Mengel et al. (2016) and that was originally used by Grinsted et al. (2010) to model the total global mean sea-level changes.First, an equilibrium sea-level rise from thermal expansion, due to changing global surface temperature (S eq,TE , m), is calculated as In Eq. ( 9), a TE is the sensitivity of the equilibrium sea-level rise from thermal expansion, due to changing global surface temperatures (m • C −1 ), and b TE is the equilibrium sea-level rise from thermal expansion with no temperature anomaly (m).The sea-level rise due to thermal expansion evolves with time as where the quantity τ TE is the e-folding timescale with which the current sea level adjusts to the equilibrium state, and 1/τ TE is taken as an uncertain model parameter.This parameter is assigned a gamma prior distribution with shape 1.81 and scale 0.00275, which places the 5th and 95th quantiles for τ TE at 82 and 1290 years (Mengel et al., 2016).This www.geosci-model-dev.net/10/2741/2017/Geosci.Model Dev., 10, 2741-2760, 2017 choice of prior distribution is motivated by the fact that τ TE functions similarly to the uncertain timescale associated with an exponentially distributed random variable.A gamma distribution is the conjugate prior for such a random variable.
The initial condition S 0,TE is provided as an uncertain model parameter (m), designated in year 1850.To match this accounting for sea-level rise relative to pre-industrial, forcing temperature is taken relative to its 1850-1870 mean.We calibrate the thermal expansion component of sea-level rise using trends reported by the IPCC (Church et al., 2013).

Regional sea-level patterns
In order to link the projections of global mean sea-level change from BRICK to a local coastal adaptation, information on regional sea-level change is needed.Thus, the global mean sea level from BRICK is downscaled to regional sea level using previously published maps of scaling factors for the glacier and ice sheet components of sea-level change (Slangen et al., 2014).Any redistributions of mass between the cryosphere and the ocean (e.g., ice melt) lead not only to a change in the total mass of the ocean, but also to changes in regional sea level as a result of variations in the gravitational field of the Earth, which in turn affect the solid Earth and the rotation of the Earth (e.g., Mitrovica et al., 2001).This typically (and counterintuitively) leads to a sea-level fall close to the source of mass loss and larger-than-average sealevel rise at larger distances (> 6700 km) from the source.These so-called regional sea-level "fingerprints" are constant for the timescales used in this study, as long as the location of the ice mass change remains the same.The fingerprints can therefore be used to relate global glacier and ice sheet contributions to sea level (Sects.3.2.1-3.2.3) to their regional sea-level contribution.We couple changes in global sea level to the Antarctic Ice Sheet model using an Antarctic shoreaverage fingerprint ratio of −1.0 for the AIS contribution to global sea level, and Antarctic shore-average fingerprint factors of 1.0 for the other contributions to sea-level rise from all BRICK sub-models (Slangen et al., 2014).Preliminary experiments indicated that our results are not sensitive to the precise choices of these fingerprints.
The glacier fingerprint is based on projected changes in glacier mass in 2100 using a glacier model driven by temperature and precipitation information from the Fifth Climate Model Intercomparison Project database (Taylor et al., 2012) under the Representative Concentration Pathway 8.5 climate change scenario (RCP8.5,Moss et al., 2010), as presented in Slangen et al. (2014).It is assumed that the mass change ratios between the different glacier regions on Earth remain the same throughout the 20th and 21st centuries, which is a valid assumption as long as none of the glacier regions "finish" (which is not expected to happen in the next century).For the Greenland and Antarctic ice sheets, it is assumed that ice melt takes place uniformly over the ice sheet surface.Within the BRICK model structure, users may define a latitude and longitude to obtain regional sea-level change.
4 Model experiments

Model calibration
We calibrate the model through a coupled physical-statistical calibration framework.The relatively simple physical modeling framework of BRICK is motivated by our epistemic modeling values (Sect.2.1).This efficient model permits the use of a sophisticated model calibration technique.The calibration uses a robust adaptive Markov chain Monte Carlo (MCMC) approach (Vihola, 2012).The specifics of how it is applied to the BRICK model as well as a demonstration of calibrated BRICK model hindcast skill are documented in Bakker et al. (2017).
The vastly different timescales and characterizations of uncertainty in the Antarctic paleoclimatic calibration period and the modern period (1850 to present) lead to two separate sets of calibration parameters: (i) DAIS parameters, calibrated using paleoclimatic data, and (ii) DOECLIM, GIC, GIS, and TE parameters, jointly calibrated using modern data.The paleoclimatic calibration is done using four parallel MCMC chains of 500 000 iterations each.The first 120 000 iterations of each chain are removed for burn-in.The paleoclimatic calibration requires about 10 h on a laptop with a 2.9 GHz dual-core processor with 16 GB of RAM.The modern calibration is done using four parallel MCMC chains of 1 × 10 6 iterations each.The first 500 000 iterations of each chain are removed for burn-in.This requires less than 1 h on the same machine as the paleoclimatic calibration.Convergence and burn-in lengths are assessed using Gelman and Rubin diagnostics (Gelman and Rubin, 1992).
We combine these two disjoint sets of parameters to form concomitant full BRICK model parameter sets, and calibrate these to global mean sea-level data (Church and White, 2011) using rejection sampling (Votaw Jr. and Rafferty, 1951).Prior to rejection sampling, contributions from land water storage are estimated using trends from the IPCC (Church et al., 2013) and subtracted from global mean sea level.When projecting global mean sea-level rise, we estimate land water storage contributions by extrapolating using the 2003-2013 trend of 0.30 ± 0.18 mm yr −1 found by Dieng et al. (2015).This approximation may not hold in reality (Wada et al., 2012), but serves as a starting point for future model developments.The use of rejection sampling and the estimation of land water storage contributions to sea level are the two aspects in which our calibration approach differs from that of Bakker et al. (2017).In this rejection sampling step, each full BRICK parameter set is constructed by parsing a random draw from the calibrated DAIS parameter sets with a random draw from the DOECLIM-GIC-GIS-TE calibrated parameter sets.This full BRICK model has the major components of global mean sea-level rise represented, so only at this stage is calibration using global mean sea-level data possible.The calibration to global sea-level data initially proposes 135 000 full BRICK model parameter sets.We use a joint Gaussian normal likelihood function centered at the time series of the global mean sea-level data, with standard deviation given by the observational uncertainty of the sea-level data (corrected to account for land water storage).For rejection sampling, the enveloping distribution is this likelihood function evaluated at the observed sea-level time series itself.Thus, no model simulation can yield a realization of the likelihood function that exceeds this value.Rejection sampling accepts each model simulation with probability equal to the ratio of the likelihood function evaluated at the selected model simulation to the maximal value of the likelihood function; 10 589 ensemble members remain after the calibration to global mean sea-level data.These model realizations serve as the control ensemble for analysis.The entire analysis for the control model, including paleoclimatic simulations and the risk assessment presented in Sect.4.4, requires about 4 h on a modern laptop, but constructing smaller ensembles is much faster (an ensemble of about 600 members requires less than 10 min).
In the spirit of our epistemic values, calibration routines are provided with the available BRICK source code.These routines use modern methods readily available in R. It is our aim that the interested user can easily substitute their own likelihood function (as physical scientific knowledge progresses), a new calibration method (as the statistical state-ofthe-art progresses), or both.To this end, we provide a subroutinized likelihood function, called from an R-packaged calibration method (Vihola, 2012).We also provide individual likelihood functions and calibration scripts for each submodel of BRICK individually, to enable interested users to perform experiments using stand-alone sub-models or precalibration (Edwards et al., 2011).
In the interest of accessibility and transparency, with the available BRICK source code we also provide the sets of calibrated model parameters for all experiments presented here.The purpose of this is 2-fold.First, it greatly enhances the reproducibility of these results.Second, these data sets enable users who would like to run their own ensembles and make projections of local sea levels to do so.This supports our goal of accessibility.The calibrated parameter sets are provided in netCDF format, with ensemble member as the "unlimited" dimension.This permits concatenation of multiple data sets by using netCDF operators (NCO) such as "ncrcat" (Zender, 2008).These are freely available tools for manipulating data stored in netCDF format.

Experimental description
We achieve the accessibility, transparency, and computational efficiency of the BRICK modeling framework through use of simple models written in a simple programming environment (R, R Core Team, 2016).It remains to be demonstrated that this framework is flexible and efficient in postprocessing.
We demonstrate BRICK's flexibility and efficiency by implementing and switching in an alternative formulation for the global mean sea level, S(t).We exchange the more detailed model configuration for global mean sea level (the BRICK control; see Fig. 1) for the simple emulator described by Rahmstorf (2007).This is where t is time (years), S is the global mean sea level (m), a GMSL is a sensitivity constant (m • C −1 yr −1 ), T g is the global mean surface temperature anomaly ( • C), and T eq,GMSL is the theoretical temperature at which the global sea level is steady ( • C).The parameters a GMSL and T eq,GMSL , as well as the statistical parameters ρ GMSL (the first-order lag) and σ GMSL (the homoscedastic component of the innovation variance), are calibrated using the same global mean sea-level data set as the full BRICK sea-level rise module (Church and White, 2011).The BRICK-GMSL model performance using Eq. ( 11) for the sea-level rise module (while still coupled to DOECLIM as the climate module) is compared against the full BRICK model configuration.This BRICK-GMSL model configuration is calibrated using four parallel MCMC chains of 100 000 iterations each.The first 50 000 iterations are removed for burn-in, as determined using Gelman and Rubin diagnostics (Gelman and Rubin, 1992).We randomly sample from the resulting posterior distribution to form an ensemble for analysis of 10 589 model realizations.This ensemble size is chosen to be consistent with the BRICK control model ensemble size.The prior ranges and posterior medians, 5 %, and 95 % quantiles for the BRICK-GMSL parameters are provided in Appendix A. Note that the Rahmstorf (2007) emulator is arguably not the state-of-the-art anymore (Grinsted et al., 2010;Kopp et al., 2016).However, it serves here the purpose of demonstrating the ease with which alternative model formulations can be tested.This greatly simplifies, for example, model intercomparisons and improvements.Some advantages of a simple emulator such as this include fewer parameters to estimate and a transparent analysis.Disadvantages of such a model include the inability to resolve individual contributions to global mean sea level.This disables the use of sea-level fingerprinting to obtain regional sea-level patterns.

Metrics for model-data comparison
Many goodness-of-fit metrics are available for the comparison of models and data.We focus on three metrics that are motivated by the heavily parameterized full BRICK model framework.There are 39 free parameters in the coupled climate/sea-level rise model.By contrast, BRICK-GMSL has 13 free parameters.We use the global mean sea-level time series of Church and White (2011) for the model-data comparisons in skill hindcasting global mean sea level.
Root-mean-squared error (RMSE) is a commonly used error metric, so we employ it here.For consistency with other error criteria defined below, we define the RMSE for a model as the RMSE of the model ensemble member that maximizes the likelihood function.
The Akaike information criterion (AIC) is a measure of the relative goodness-of-fit between two potential models for the same data (Akaike, 1974).
In Eq. ( 12), L max is the maximum value of the likelihood function and N is the number of model parameters.Lower values of the AIC provide a better match between model output and data, and consider a penalty for over-parameterizing a model.The Bayesian information criterion (BIC) is formulated similarly to the AIC, but enacts a different penalty for overparameterization (Schwarz, 1978).
In Eq. ( 13), N obs is the number of observational data points used in the model-data comparison.Thus, for N obs > e 2 , the BIC metric penalizes over-parameterization more harshly than does the AIC.

Experimental results: sea-level rise module intercomparison
The full BRICK sea-level rise module (Fig. 1) performs better than the GMSL emulator (Eq.11) according to RMSE; the full sea-level rise module has an RMSE of 0.0057 m, which is about half the GMSL emulator RMSE of 0.015 m (Fig. 2).These hindcasts are presented as sea level relative to 1961-1990 global mean sea level.This is of course expected, because the number of free model parameters in the full BRICK model is 39, while the GMSL emulator contains only 13 free parameters.The BIC metric gives the expected result for this disparity in model complexity.The BIC for the full BRICK model with respect to the sea-level data is 60.4 higher than the BIC for the GMSL emulator.The AIC is actually lower (by 14.2) for the full BRICK model than for the BRICK-GMSL emulator.These mixed results for the model comparison metrics indicate that the full BRICK sea-level rise module is not unreasonably over-parameterized; if the full BRICK model were obviously over-parameterized, we would expect the AIC for the GMSL emulator experiment to be lower than for the full BRICK model.These results also show that the sea-level hindcast in the full BRICK model smoothes much of the year-to-year variability in sea-level rise.This can be seen by contrasting the full BRICK maximum likelihood ensemble member (solid blue line) in Fig. 2a with the BRICK-GMSL emulator maximum likelihood ensemble member in Fig. 2b.The full BRICK simulation does not capture the annual variation in global mean sea level that the BRICK-GMSL simulation successfully captures.This is attributed to the smoothing effect of averaging over the model ensemble the four major contributions to global mean sea level, as opposed to calibrating the BRICK-GMSL simulations directly to global mean sea-level data.This does not affect ensemble statistics, however, which can be seen from the shaded envelopes around the model simulations in Fig. 2. The BRICK model has been developed with efficiency and large ensemble simulations in mind, so missing annual variability is of little concern.This demonstrates the ease with which model intercomparisons may be undertaken using BRICK.Deactivating the glaciers and ice caps, thermal expansion, and Greenland and Antarctic Ice Sheet components and integrating the GMSL emulator into BRICK involves low overhead in computer code.GMSL is the main output of the BRICK physical model.As such, it is our aim to provide a framework in which users can easily integrate new processes and models into the climate and sea-level rise modules as the scientific forefront progresses.This GIC-SIMPLE model configuration calibrates GIC-SIMPLE using the same observational data as the control GIC-MAGICC setup.One key difference is that the prior distributions of the model parameters for GIC-SIMPLE were modified to be specific to the GIC conditions instead of the GIS.These prior distributions are given in Appendix A. The same calibration method and likelihood functions are used for the GIC-SIMPLE experiment as in the GIC-MAGICC control model.We use the same calibration approach as in the control ensemble, which yields an ensemble of 10 483 model realizations for analysis in the GIC-SIMPLE experiment.As in Sect.4.2, we focus on RMSE, AIC, and BIC as model goodness-of-fit metrics.The GIC-MAGICC model has six model parameters (four physical parameters, two statistical ones) and the GIC-SIMPLE model has seven parameters (five physical parameters, two statistical ones).

Experimental results: glaciers and ice caps sub-model intercomparison
When the GIC-MAGICC model is used, RMSE, AIC, and BIC are all lower than when the GIC-SIMPLE model is used (Fig. 3).But the AIC and BIC are not drastically lower for GIC-MAGICC than for GIC-SIMPLE.This indicates that the addition of a model parameter (GIC-SIMPLE) may not be justified (Kass and Raftery, 1995).The GIC contribution to global sea level in Fig. 3 is presented relative to 1960 GIC sea-level rise.The median, 5 %, and 95 % quantiles of the calibrated GIC-SIMPLE parameters are given in Appendix A.
The two models display similar levels of underconfidence, illustrated by the wide model ensemble envelope around the narrower range of observational data (Fig. 3) (Dyurgerov and Meier, 2005).That both models show under-confidence is often judged to be preferable to over-confidence, especially when physical models are linked to applications-oriented decision-making frameworks (Herwww.geosci-model-dev.net/10/2741/2017/Geosci.Model Dev., 10, 2741-2760, 2017 Table 1.Parameter descriptions and prior probability distributions for flood protection cost-benefit analysis.

Parameter Description Distribution
p 0 Initial flood frequency (yr −1 ) with 0 heightening log N(log µ = log(0.0038),log σ = 0.25) α Exponential flood frequency constant (m −1 ) N(µ = 2.6, σ = 0.1) V Value of goods protected by dike ring (billion USD) U(5, 30) δ Net discount rate (-) U(0.02, 0.06) I unc Investment uncertainty (-) U(0.5, 2) r subs Land subsidence rate (m yr −1 ) log N(log µ = log(0.0056),log σ = 0.4) man et al., 2015).This experiment demonstrates BRICK's flexibility and ability to allow the user to isolate and examine any source of uncertainty or dissatisfaction in the modeling framework.These results also provide guidance for the use of the BRICK model framework for model intercomparison and selection experiments.At present we do not make any recommendations regarding which GIC sub-model to use.The GIC-MAGICC component has both strengths (e.g., fewer parameters and appropriate in melting regimes) and weaknesses (unphysical GIC growth, does not encourage growth beyond V 0,GIC , state-independent equilibrium).
4.4 Linking an impacts and decision-analysis module to BRICK

Experimental description
We demonstrate the ability of the BRICK framework to incorporate additional structure to link the physical model for surface temperature and sea-level rise (climate and sea-level modules, Fig. 1) to socioeconomic implications (impacts module, Fig. 1).In this example application, we use the calibrated ensemble in the BRICK control configuration to obtain local sea level projections for New Orleans, Louisiana (29 • 57 N, 90 • 4 W).We use a common didactic model for coastal flood protection (Van Dantzig, 1956;Jonkman et al., 2009).In this flood risk model, the policy lever available to decision-makers is the amount by which to heighten the dikes protecting the coastal community.We consider a previously published simple analysis that focuses on the northern dike ring in central New Orleans (Jonkman et al., 2009).
We use this illustrative cost-benefit approach to calculate an economically efficient dike-heightening by weighing the decrease in probable losses due to flooding achieved by building taller dikes against the increase in costs due to investments in construction.
The flood risk model implemented here follows a commonly used simple approach (Van Dantzig, 1956).The present implementation considers the current year as 2015 and a time horizon of 85 years (to 2100).We consider discrete dike heightenings in increments of 5 cm, between 0 and 10 m.The average annual flood probability is calculated from the simulated local sea-level rise, the land subsidence rate (Dixon et al., 2006), and flood frequency parameters (Jonkman et al., 2009).We calculate the expected losses (US dollars) for each proposed dike heightening from the flood probabilities for each heightening, the value of goods protected by the dike ring, and the net discount rate (Jonkman et al., 2009).The total expected costs are the sum of the expected losses and the expected investments.In this simplified model, the investment costs only depend on dike heightening and are approximated by linear interpolation between data points provided by Jonkman et al. (2009) (and linear extrapolation for dike heightenings outside this range), and the expected losses are an exponentially decreasing function of dike height above mean sea level.The minimum total expected cost then is the economically efficient dike heightening strategy in the framework of this simple illustrative model (Eq.14 of Van Dantzig, 1956).
The uncertain parameters considered in this cost-benefit analysis include the initial flood frequency with no heightening (yr −1 ); the exponential flood frequency constant (m −1 ); the value of goods protected by the dike ring (billion US dollars); the net discount rate (%); the uncertainty in investment costs (a unitless multiplicative factor); and the land subsidence rate (m yr −1 ) (Table 1).The central estimates for the exponential flood frequency constant (α) and the initial flood frequency with no heightening (p 0 ) are taken from Van Dantzig (1956).The exponential flood frequency constant relates the increase in flood probability that results from an increase in sea level relative to the dike height.We make the assumption that this factor should scale (to first order) relatively well from the Dutch case considered by Van Dantzig (1956) to the test case of New Orleans considered presently.The initial flood frequency with no heightening (p 0 ) may not translate directly between these two cases, but highlights our intent for this experiment to serve as an example of future applications of the BRICK model to inform decision analyses.The admittedly ad hoc distributions assumed for α and p 0 were selected to sample tightly around the central estimates from Jonkman et al. (2009).A more detailed treatment of this risk management problem would include using methods from extreme value theory to address the risks posed by storm surges (Coles, 2001).
The investment uncertainty considered in the sensitivity tests of Jonkman et al. (2009) included a base case, 50 % lower, and 100 % higher than the base case.We use this range for the investment uncertainty, applied as a multiplicative factor ranging from 0.5 to 2. The range for the value of goods protected by the dike ring is taken from Jonkman et al. (2009), where the lower bound is the lowest estimate of the value of goods protected by the three dike rings considered in that work (USD 5 billion), and the upper bound is the estimated combined value protected by all three dike rings (USD 30 billion).The net discount rate range is centered at 4 %, the estimate from Jonkman et al. (2009) accounting for inflation and interest rate.Those authors' net discount rate is decreased to 2 % due to economic growth (1 %) and increased flooding probability due to sea-level rise (1 %).Our demonstrative example endogenizes the effects of sea-level rise and accounts for parametric uncertainty in the value of goods protected by the dike ring.Hence, we center our range for the net discount rate at 4 % but allow for a ±2 % uncertainty range.The rate of land subsidence is based on the estimates of Dixon et al. (2006), with mean 5.6 mm yr −1 and standard deviation 2.5 mm yr −1 .We transform this to a lognormal distribution to disallow negative rates of land subsidence.We sample the uncertainty in these parameters via a Latin hypercube, where the population size is given by the number of sea-level rise ensemble members that are present (10 589 for the control BRICK ensemble).The distributions from which the economic model parameters are drawn are given in Table 1.Each realization of regional sea level is assigned a concomitant sample of flood risk model parameters.An economically efficient dike heightening is calculated for each ensemble member."Return periods" (years) correspond to the frequency of storms with the potential to overtop dikes with the corresponding dike height -essentially, the inverse of the annual flood probability.Return periods are a convenient and intuitive way to view the probabilities of flooding in this economic analysis.
We present results for the flood risk management experiment using sea-level projections under RCP8.5.We note that many factors are not incorporated into this analysis, and this simple illustration is not designed to be used for real decision-making.For example, storm surge non-stationarity and structural failure are not considered (Grinsted et al., 2013;Moritz et al., 2015).The purpose of this illustration is to demonstrate the flexibility and transparency of the BRICK model framework.This experiment highlights the importance of transparency in particular when linking physical modeling results to the impacts on socioeconomic modeling and policy decision-making.

Experimental results: regional sea-level changes
In order to link projections of sea-level rise to problems of local coastal adaptation, regional sea level is projected to 2100 under the climate change scenarios of RCP2.6, 4.5, and 8.5 (Fig. 4).These projections use the control configuration of www.geosci-model-dev.net/10/2741/2017/Geosci.Model Dev., 10, 2741-2760, 2017 the model, with GIC-MAGICC and the full sea-level rise sub-model setup depicted in Fig. 1.The ensemble median projection is shown in Fig. 4. Sea level rises by 2100 globally by about 55 cm (43-72 cm), 74 cm (56-100 cm), and 130 cm (93-177 cm) under RCP2.6,4.5, and 8.5, respectively (ensemble median and 5-95 % range in parentheses).The Arctic Ocean is an obvious exception to the rest of the ocean.Due to the Greenland ice mass loss, Arctic regional sea level will fall as a result of the loss of gravitational attraction.However, the addition of mass raises sea level in other parts of the ocean farther away.Arctic sea level (median sea level of all latitudes higher than 60 • N) increases by 7 cm under RCP2.6, but falls by 2 cm under RCP4.5 and by 30 cm under RCP8.5.By contrast, the tropical sea level (median of all latitudes between 30 • S and 30 • N) rises by 57, 82, and 147 cm under RCP2.6,4.5, and 8.5, respectively, which is greater than the global mean rise.Due to the asymptotically increasing gravitational effects in proximity to the melting Greenland Ice Sheet, sea-level fall below −1.5 m is cut off at −1.5 m.

Experimental results: link to coastal defense strategies
We now focus on the regional sea-level projections for the grid cell containing New Orleans, Louisiana, under RCP8.5 (Fig. 4c), to demonstrate the use of these sea-level projections in a common local flood risk management example.We find the economically efficient (i.e., cost-minimizing) dike heightening to be 1.5 m (ensemble mean; the 90 % range is 0.75 to 1.95 m; Fig. 5).This heightening corresponds to a return period of about 760 years (ensemble mean; the 90 % range is roughly 200-3000 years; Fig. 5).The simple analysis presented here should not be used to inform on-theground decisions in New Orleans.This experiment is meant to demonstrate BRICK's ability to contribute in risk assessment applications.

Conclusions
We present BRICK v0.2, a modeling framework for global and regional sea-level change.BRICK has been designed with four epistemic modeling goals: accessibility, transparency, efficiency, and flexibility.BRICK can skillfully match observational data for individual sea-level contributions in hindcasts (Bakker et al., 2017).Here we focus on how BRICK achieves our epistemic values using a set of modeling experiments.BRICK is coded in the widely available and simple coding language R (R Core Team, 2016), to achieve the goals of accessibility and transparency.The main physics (global mean temperature and sea-level rise) codes are also (redundantly) transcribed in Fortran 90, for more efficient simulations.BRICK is designed to be transparent, as well as efficient, by coupling previously published simple, mechanisti-cally motivated models for the major contributors to global sea level.The efficient physical modeling approach provides the opportunity to incorporate a rigorous statistical calibration framework as well, wherein various sources of uncertainty are incorporated into model projections (see Bakker et al., 2017, for a more detailed discussion of this).Finally, the model comparison experiments in Sect.4.2 and 4.3 demonstrate the flexibility of the BRICK modeling framework.These sections bring into focus the importance of these epistemic modeling values.A modeling framework that is (in particular) transparent and accessible can help to streamline the process of quantifying the local impacts of the physical model results, to link to decision-analytical models, and to communicate these results to stakeholders and decisionmakers.
We hope that the accessibility and transparency of BRICK are helpful to others, and will stimulate the continuous peerreviewing, challenging, and improving of the BRICK framework.Of course, although we tried to couple models that fit our epistemic model values as closely as possible, we assume that others may prefer other models and may have different epistemic values.Our framework is designed in such a way that it is possible to plug in other model components to reflect these different values.For example, it would be very interesting to add the component models used for the semiempirical model frameworks of Mengel et al. (2016) and Nauels et al. (2017).
We demonstrated the flexibility and transparency of BRICK in connecting projections from the physical model to the impacts on a local risk and decision-analysis problem.The simple probabilistic calibration method and costbenefit analysis that we adopted for the simple demonstration can be expanded to incorporate aspects of deep uncertainties (Lempert et al., 2004;Weaver et al., 2013) as well as more complex decision-making frameworks (e.g., considering multiple objectives, beyond only expected total costs) (Kasprzyk et al., 2013;Lempert, 2014;Lempert and Collins, 2007).Climate change poses decision problems where strong connections across academic disciplines are critical.Further, the study of climate modeling relies on communal modeling efforts.The need for transparent communication among modelers and between disciplines is where the BRICK framework and the epistemic modeling values presented here can facilitate future developments.Above all, we hope that BRICK inspires the involved communities to pay careful attention to enhance flexibility, transparency, and accessibility of modeling frameworks.
Appendix A: Prior probability distribution ranges for the sub-model parameters, and median, 5th, and 95th quantiles of the calibrated posterior parameter distributions

Figure 1 .
Figure 1.BRICK model structural diagram.Dashed connectors indicate couplings that are non-essential for projections of global mean sea level.These dashed couplings are required for projecting regional sea-level and climate impacts.DOECLIM is the Diffusion-Ocean-Energy balance CLIMate model (Kriegler, 2005); GIC-MAGICC is the Glaciers and Ice Caps module from the MAGICC climate model(Meinshausen et al., 2011a); TE is the Thermal Expansion model(Grinsted et al., 2010;Mengel et al., 2016); SIMPLE is the Simple Icesheet Model for Projecting Large Ensembles(Bakker et al., 2016); ANTO is the ANTarctic Ocean temperature model; DAIS is the Danish Center for Earth System Science Antarctic Ice Sheet model(Shaffer, 2014); regional sea-level fingerprinting downscales from global sea-level contributions to regional(Slangen et al., 2014); and the model of Van Dantzig (1956) assesses flood risk.

Figure 2 .
Figure 2. Comparison of global mean sea-level rise hindcast skill relative to sea-level data (Church and White, 2011), using (a) the full sub-model approach (GIC, GIS, TE, and AIS) and (b) the model for global mean sea-level rise of Rahmstorf (2007).Sea level is relative to 1961-1990 global mean sea level.Both model configurations use DOECLIM as the climate module.Lower values of the Akaike information criterion (AIC), the Bayesian information criterion (BIC), and root-mean-squared error (RMSE) indicate a better model fit to the data.These error metrics are all calculated using the maximum likelihood ensemble member, which is represented by the solid blue line.Green highlighting indicates the model structure suggested by each comparison metric.
BRICKs and sub-model intercomparisons4.3.1 Experimental descriptionWe conduct an experiment to demonstrate the flexibility of BRICK to permit easy exchanging of a single sub-model for one component of global sea-level rise.In the control BRICK model setup, SIMPLE is used to emulate the sea-level rise contributions from the Greenland Ice Sheet (GIS) and GIC-MAGICC is used to emulate the contributions from glaciers and ice caps (GIC).In this model intercomparison experiment, a second version of SIMPLE is calibrated to represent the GIC component of sea-level rise.This experiment is motivated by potential structural shortcomings of the GIC-MAGICC model.In Eq. (2), the implied GIC volume equilibrium depends only on the current surface temperature relative to the fixed parameter T eq,GIC .If the GIC volume is quite low (almost entirely melted), this structure potentially enables unphysically fast growth of GIC volume.The SIM-PLE model (Eqs.3-5) contains an arguably more realistic representation of the relaxation of ice sheet volume towards an equilibrium.In this formulation, the timescale of the relaxation and the equilibrium itself both depend on the surface temperature state.This type of potential disagreement within the scientific community regarding model structure is precisely where the BRICK model framework can be useful.The flexibility of BRICK enables easy exchange of one component sub-model (GIC-MAGICC) for another (GIC-SIMPLE).This enables experiments examining the impacts of model structural choices.

Figure 3 .
Figure 3.Comparison of (a) GIC-MAGICC vs.(b) GIC-SIMPLE model performance in hindcasting the glaciers and ice caps (GIC) contribution to sea-level rise.GIC sea-level rise is presented relative to 1960 GIC sea-level contribution.Lower values of the AIC, BIC, and RMSE indicate a better model fit to the data (Dyurgerovand Meier, 2005).These error metrics are all calculated using the maximum likelihood ensemble member, which is represented by the solid blue line.Green highlighting indicates the model structure suggested by each comparison metric.

Figure 5 .
Figure 5. Illustrative cost-benefit analysis for the economically efficient dike heightening (lower horizontal axis) and return period (upper horizontal axis) for the northern-central dike ring in New Orleans, Louisiana.The bold dot denotes the economically efficient (i.e., cost-minimizing) solution.The shaded region gives the 90 % ensemble range of trade-off curves, and the bold line denotes the ensemble mean trade-off curve.

Table A1 .
Prior probability distribution ranges for the DOECLIM climate model parameters, and median, 5th, and 95th quantiles of the calibrated posterior parameter distributions.The priors are all uniformly distributed.

Table A2 .
Prior probability distribution ranges for the thermal expansion model parameters, and median, 5th, and 95th quantiles of the calibrated posterior parameter distributions.The prior distribution for 1/τ TE is a gamma distribution (see main text).The other priors are all uniformly distributed.

Table A3 .
Prior probability distribution ranges for the GIS-SIMPLE Greenland Ice Sheet model parameters, and median, 5th, and 95th quantiles of the calibrated posterior parameter distributions.The priors are all uniformly distributed.Due to convergence issues, ρ GIS is held fixed at a value calculated from a preliminary optimized model simulation (see main text).

Table A4 .
Ruckert et al., 2017)tribution ranges for the DAIS Antarctic Ice Sheet model parameters, and median, 5th, and 95th quantiles of the calibrated posterior parameter distributions.An inverse gamma prior distribution is used for σ 2 DAIS (seeRuckert et al., 2017).All other prior distributions are uniform.

Table A5 .
Prior probability distribution ranges for the GIC-MAGICC Glaciers and Ice Caps model parameters, and median, 5th, and 95th quantiles of the calibrated posterior parameter distributions.The priors are all uniformly distributed.

Table A6 .
Prior probability distribution ranges for the GIC-SIMPLE model parameters, and median, 5th, and 95th quantiles of the calibrated posterior parameter distributions.The priors are all uniformly distributed.

Table A7 .
Rahmstorf (2007)y distribution ranges for theRahmstorf (2007)global mean sea-level model parameters, and the median, 5th, and 95th quantiles of the calibrated posterior parameter distributions.The priors are all uniformly distributed.