Four simulations with the ECHAM/MESSy Atmospheric Chemistry (EMAC)
model have been evaluated with the Earth System Model Validation
Tool (ESMValTool) to identify differences in simulated ozone and
selected climate parameters that resulted from (i) different setups
of the EMAC model (nudged vs. free-running) and (ii) different
boundary conditions (emissions, sea surface temperatures (SSTs) and
sea ice concentrations (SICs)). To assess the relative performance
of the simulations, quantitative performance metrics are calculated
consistently for the climate parameters and ozone. This is important
for the interpretation of the evaluation results since biases in
climate can impact on biases in chemistry and vice versa. The
observational data sets used for the evaluation include ozonesonde
and aircraft data, meteorological reanalyses and satellite
measurements. The results from a previous EMAC evaluation of a model
simulation with nudging towards realistic meteorology in the
troposphere have been compared to new simulations with different
model setups and updated emission data sets in free-running time slice
and nudged quasi chemistry-transport model (QCTM) mode. The latter
two configurations are particularly important for chemistry-climate
projections and for the quantification of individual sources
(e.g., the transport sector) that lead to small chemical perturbations of
the climate system, respectively. With the exception of some
specific features which are detailed in this study, no large
differences that could be related to the different setups (nudged vs.
free-running) of the EMAC simulations were found, which offers
the possibility to evaluate and improve the overall model with the
help of shorter nudged simulations. The main differences between the
two setups is a better representation of the tropospheric and
stratospheric temperature in the nudged simulations, which also
better reproduce stratospheric water vapor concentrations, due to
the improved simulation of the temperature in the tropical
tropopause layer. Ozone and ozone precursor concentrations, on the
other hand, are very similar in the different model setups, if
similar boundary conditions are used. Different boundary conditions
however lead to relevant differences in the four simulations. Biases
which are common to all simulations are the underestimation of the
ozone hole and the overestimation of tropospheric column ozone, the
latter being significantly reduced when lower lightning emissions of nitrogen
oxides are used. To further investigate possible other reasons for
such bias, two sensitivity simulations with an updated scavenging
routine and the addition of a newly proposed
A correct representation of tropospheric and
stratospheric ozone is crucial for reproducing past trends in climate
variables (e.g., temperature) as well as for providing reliable projections of the
chemistry-climate system in the 21st century. Tropospheric ozone
burden has increased by around 30
Here, we evaluate simulations performed with the ECHAM/MESSy Atmospheric
Chemistry (EMAC) model, which is a numerical chemistry and climate simulation
system that includes submodels describing tropospheric and middle atmosphere
processes and their interaction with oceans, land and human influences
This paper is organized as follows: the model and model simulations
are described in Sects.
EMAC uses the Modular Earth Submodel System
The versions of MESSy used in this study include more than 30
submodels, with different functions and purposes. The submodels that
are used in the simulations evaluated in this work are summarized in
Table
List of the MESSy submodels used in the simulations. See
Gas-phase chemistry is calculated with the submodel MECCA
The four EMAC simulations discussed in this study have the same
resolution but differ from each other in their setup. Two nudged,
transient simulations (EVAL2 and QCTM) driven by the same meteorology
(including SSTs) and emission inventories are compared to two
free-running time slice simulations (ACCMIP and TS2000). As
a reference, we use the nudged experiment described in
In the following, the specific features that characterize each EMAC simulation are briefly summarized (see also Table S1 in the Supplement). A more detailed description of the general model setup which applies to all the experiments is provided in the Supplement (Sect. S1). The four simulations were conducted as part of various projects. The specific requirements of each project (e.g., ACCMIP) motivated the different configurations that were applied.
Overview of the four EMAC simulations evaluated in this study. All experiments have a spin-up year at the beginning of the simulated period which is not considered in the analysis.
This simulation has been previously evaluated by
As this experiment is designed to (approximately) reproduce the
meteorology and the atmospheric composition of the individual years,
transient (i.e., varying year by year) emission data are used where
available. For anthropogenic non-traffic emissions, we use the CMIP5
emission inventory of
The QCTM simulation covers a period of 10
The QCTM mode is realized by driving the radiation with external
climatological fields for the radiatively active gases (
Like EVAL2, this simulation was carried out to approximate meteorology
and atmospheric composition for individual years, therefore it is
performed in nudged mode and using transient emissions. We use the same nudging coefficients as for EVAL2. The emission
setup is also identical to EVAL2, with the exception of aviation emissions
which were taken from QUANTIFY
In contrast to the nudged simulations (EVAL2 and QCTM), the TS2000
simulation is a time slice experiment, performed in free-running mode
over a period of 10
This time slice simulation was performed in support of the Atmospheric
Chemistry and Climate Model Intercomparison Project
In order to quantitatively assess and compare the ability of the different EMAC simulations in representing key features of observed climate and chemical composition, basic statistical measures are calculated in addition to the diagnostic plots that provide more detailed insights. For each diagnostic, the root-mean-square difference (RMSD), the overall mean bias, and the Taylor diagram are presented. The RMSD and bias metrics are calculated considering the space–time field (latitude, longitude plus annual cycle) where available, or only the annual cycle otherwise.
Following
All diagnostics and performance metrics shown in this paper have been
implemented into the Earth System Model Validation Tool (ESMValTool). This ensures that the analysis presented in this
paper can be applied to other EMAC simulations and other ESMs in
a routine manner. The ESMValTool was originally based on the previously-developed CCMVal Diagnostic
Tool for chemistry-climate models
The ESMValTool is designed to work on model output formatted according
to the Climate Model Output Rewriter (CMOR) tables metadata (see
The ESMValTool is developed as an international community tool by
multiple institutions with the goal to enhance routine benchmarking
and evaluation of ESMs. The priority of the effort so far has been to
target specific scientific themes focusing on selected essential
climate variables (ECVs), tropical variability (e.g., Monsoon),
Southern Ocean, continental dry bias and soil hydrology–climate
interactions, carbon dioxide (
A variety of different observations are used for the model evaluation. For most variables, we choose a reference and an alternative data set in order to estimate differences and uncertainties in observations.
A summary of the main diagnostics applied in this study is given in
Table
For global temperature, winds, geopotential height and specific
humidity, meteorological reanalyses are the best available reference
data. Reanalysis projects provide spatially complete and coherent
records of atmospheric variables. Given the improvement of models,
input data and assimilation methods, reanalyses have significantly
improved in reliability, cover longer time-periods and have increased
in spatial and temporal resolution
We use two different reanalysis data sets (ERA-Interim and NCEP/NCAR,
see below) for the comparisons to simulated temperature, winds,
geopotential height and specific humidity. The differences between the
climatologies derived from these fields are an indicator of the
uncertainties in the meteorological analyses. ERA-Interim reanalysis
is produced by the ECMWF and covers the period from 1979 to present
In addition, the NCEP/NCAR reanalysis is applied, which covers the
period from 1948 to present
For specific humidity, we follow
Vertical and meridional profiles of climatological zonal mean water
vapor volume mixing ratios are compared to measurements taken by the
HALogen Occultation Experiment (HALOE) on board of the Upper
Atmosphere Research Satellite (UARS), launched in 1991
For evaluating radiation fluxes, our primary data set is taken from the
Surface Radiation Budget project
For the evaluation of total column ozone, we use the NIWA combined total
column ozone data set over the period 1998–2010 as the reference data set
List of the diagnostics applied in this work and for which
a quantitative evaluation based on performance metrics has been applied. The
climatological mean field considers both the time (annual cycle) and the
space (latitude-longitude) coordinate, or only time in some cases. Regions
are defined as follows: Glob (90
For the evaluation of tropospheric column ozone we use a global
climatology based on the Aura ozone monitoring instrument (OMI) and
microwave limb sounder (MLS) ozone measurements for the period
2005–2012
For the comparison of ozone vertical profiles in the troposphere, we
use a recently updated global climatology by
In addition, we use ozone data from a collection of aircraft campaigns
For the evaluation of ozone precursors, we use the
For the evaluation of
In the following subsections, we first evaluate how well the mean
climate state in selected basic climate variables such as temperature,
eastward and northward wind, geopotential height, specific humidity
and radiation is represented in the four simulations. In the choice of
the tropospheric diagnostics and performance metrics we closely follow
those that were applied by
For the calculation of the eastward and northward wind components,
a 10
Temperature (ta) is evaluated by investigating the climatological mean
annual cycle at the four selected pressure levels 850, 200, 30 and 5
Annual cycle of temperature climatology at 850, 200, 30 and
5
The annual cycle is in general well reproduced by all simulations at
all levels and in all regions, with the exception of the
200
At 200
Stratospheric temperatures at 30 and 5
Annual mean of zonally averaged temperature profile. The upper left plot shows ERA-Interim absolute
values; all other plots show differences between the model
simulations (or NCEP/NCAR) and ERA-Interim. Differences between the
two fields which are not statistically significant according to the
The above mentioned biases are also visible in the zonally-averaged
temperature profiles in Fig.
A warm bias can be identified in the polar SH stratosphere
(50–100
Annual cycle of water vapor climatology at 200
All experiments are characterized by a cold bias in the extratropical lower
stratosphere. This feature is common to many of the CMIP3 and CCMVal models
The temperature of the tropical tropopause layer is an important
aspect of model representation since it has strong implications for
the water vapor distribution in the stratosphere. The
lower-stratospheric water vapor mixing ratios are generally
a function of the model temperature near the tropical tropopause at
100
Annual cycle of temperature (top) and water vapor (bottom)
climatology at 100
Root-mean-square difference of the chosen basic climate
parameters over the global domain, the tropics, and the NH and SH
extratropics (from left to right). Columns and rows of
each panel represent the EMAC simulations and the given diagnostics
(see Table
As in Fig.
The relative performance of the four simulations in reproducing
temperature at the four pressure levels (850, 200, 30 and
5
Taylor diagrams of temperature (top row) and eastward wind
(bottom row) over the four chosen domains (global, tropics, NH and
SH extratropics, from left to right) and height-levels
(850, 200, 30, and 5
As in Fig.
The eastward wind (ua) as simulated by EMAC is in good agreement with
both reanalysis data sets at 850
As in Fig.
The agreement is still good at 200
In the stratosphere, where the nudging is much weaker, all the
simulations show a similar behavior, and no significant improvement
is obtained from the nudged simulations with respect to the free-running
ones. On the contrary, the QCTM simulation has some problems in
reproducing the annual cycle in the tropics in particular at the
5
Figures
The better performance of the nudged simulations with respect to the
free-running simulations in the lower troposphere (850
Northward wind, geopotential height and specific humidity are evaluated mainly to assess whether there are some serious limitations in the representation of the mean climate by the model and only discussed briefly.
The northward wind (va) at the four selected levels (850, 200, 30, and
5
The comparison of simulated geopotential height (zg) with observations shows a generally good agreement (see Figs. S6 and S7), with relative differences of the order of a few per cent. The annual cycle is mostly captured. Differences of the same order, however, can also be found when comparing ERA-Interim with NCEP data, revealing some uncertainties in the meteorological reanalyses as well.
The annual cycle of the specific humidity (hus) is mostly captured by
the EMAC simulations (Fig. S8), with the exception of the tropical
domain, in particular at the 30
Climatological mean maps of outgoing long-wave clear-sky radiation at
the ToA (rlutcs) are shown in Fig. S10, compared with SRB and
CERES. The observational data (Fig. S10, upper row, left) displays its
highest values in the tropics (about 300
The outgoing long-wave all-sky radiation at the ToA (rlut) is compared
again to SRB and CERES (Fig. S11). The observations show a maximum
value over the tropics (250–300
Total column ozone climatology for the EMAC simulations compared to the NIWA combined total column ozone database and GTO-ECV data. The values on top of each panel show the global (area-weighted) average, calculated after regridding the data to the horizontal grid of the model and ignoring the grid cells without available observational data in the GTO-ECV data.
Another important quantity for the evaluation of the radiation budget
is the reflected short-wave all-sky radiation (rsut, Fig. S12). The net
short-wave radiation is primarily determined by solar incoming
radiation and by the presence of clouds. The general pattern is
therefore a combination of the variation of incoming solar radiation
with latitude/season and of cloud cover. The EMAC simulations
reproduce this pattern well. The observations show their
highest values (around 120–150
In this paper we focus on tropospheric ozone, and consider the
stratosphere only in the context of total column ozone. Biases in
tropospheric ozone found in all four EMAC simulations led to two
additional simulations (ACCMIP-S1 and ACCMIP-S2) to explore related
model uncertainties. These two simulations are included in the
figures, but discussed separately in
Sect.
Root-mean-square difference (top) and overall mean bias
(bottom) for total and tropospheric column ozone (left), ozone profiles
(middle) and surface CO diagnostics (right). Columns and rows of
each panel represent the EMAC simulations (including the sensitivity
experiments) and the given diagnostics (see
Table
Zonal mean total column ozone (toz) climatologies from the different
EMAC simulations are compared to the NIWA assimilated data and
to GTO-ECV satellite observations in Figs.
Taylor diagrams for total and tropospheric column ozone (left), ozone profiles (middle) and surface CO diagnostics (right).
Stratospheric ozone is mainly affected by emissions from long-lived
species (
Tropospheric column ozone in the EMAC simulations compared to MLS/OMI observations. The values on top of each panel show the global (area-weighted) average, calculated after regridding the data to the horizontal grid of the model and ignoring the grid cells without available observational data.
The geographical pattern and annual cycle of tropospheric column ozone
(toztrop) from the EMAC simulations is compared to MLS/OMI
measurements on board the Aura satellite in
Figs.
Estimated methane and MCF lifetimes for the EMAC simulations.
Annual cycle of the tropospheric column ozone climatology in the EMAC simulations compared to MLS/OMI observations. The values on top of each panel show the global (area-weighted) average, calculated after interpolating the observations on the model grid and ignoring the grid cells without available observational data.
The near-global mean in EMAC EVAL2 (36.7
In agreement with observations, lower values are simulated in the
tropics and in the SH compared to NH mid-latitudes. However,
significant differences in the pattern are simulated, with correlation
values around
The annual cycle (Fig. 14) is overall well reproduced by the EMAC simulations, showing
two distinct maxima during spring in the SH and during spring/summer in the
NH. This seasonal increase in tropospheric column ozone is due to an increase
of photo-chemical production and stratosphere-troposphere exchange
Annual cycle of ozone climatology in three regions (tropics,
NH and SH extratropics) at three pressure levels (250, 500 and
700
Similar to Fig. 6 in
Ozone vertical profile climatology from selected aircraft
campaign observations by
Simulated vertical profiles of ozone are also compared to in situ
measurements from aircraft campaigns, which have been mapped onto
a
Similar to ozone, simulated vertical profiles of ozone precursors are
compared to in situ measurements for aircraft campaigns by
Similar to Fig.
Nitrogen oxides serve as catalyst in the photochemical cycles relevant
for the production and destruction of tropospheric ozone. Ozone
production depends non-linearly on
Annual cycle of
The hydroxyl radical (
Similar to Fig.
Another indirect indicator for tropospheric oxidation capacity is carbon
monoxide (
Vertical profiles of
Non-methane hydrocarbons (NMHCs) also affect ozone chemistry through
a large number of complex reactions. Several species of this family
(ethylene (
The high bias in tropospheric column ozone identified in particular in
the ACCMIP simulation motivated two additional sensitivity simulations
to explore related model uncertainties. Both are identical to the
ACCMIP simulation and cover the same time period (10
The SCAV modification avoids the use of unrealistically high
convective liquid and ice water contents for scavenging, which is
expected to result in reduced uptake and less subsequent removal of
nitric acid, particularly in the tropical upper troposphere/lower
stratosphere (UTLS). The ACCMIP-S1 simulation serves two purposes: (1)
comparing to the otherwise identical ACCMIP simulation, in order to
estimate the uncertainty imposed by the reduced uptake on the results
in all other simulations; (2) as a reference for the sensitivity
simulation ACCMIP-S2, which is also performed with the updated
scavenging code. The code modification for ACCMIP-S1 results in less
and more realistic convective cloud water and cloud ice
concentrations, and consequently less scavenging of
ACCMIP-S2 is a sensitivity simulation to quantify the uncertainty
imposed by a possible
The effects on ozone precursors are mainly determined by a decreased
oxidizing capacity in an atmosphere with the additional
Overall, introducing the
Four present-day simulations with
different setups of the ECHAM/MESSy Atmospheric Chemistry (EMAC) model have
been evaluated in this study through a comprehensive comparison to
observations. In particular, results from a previous EMAC evaluation of
a model simulation with nudging towards realistic meteorology in the
troposphere by
The two nudged simulations (EVAL2 and QCTM) are transient and driven
by the same SSTs and (transient where available) emission
inventories (with the exception of aviation). The previously evaluated
EVAL2 simulation that covers the
time period 1999–2009
In addition to a qualitative evaluation showing figures for a variety of different selected diagnostics, a quantitative evaluation has been performed to summarize the results. In particular, the normalized root-mean-square difference (RMSD) between model simulation and observations as well as the overall mean bias have been calculated consistently for climate parameters and ozone for certain domains and height-levels. Where possible, an alternative observational data set was used in addition to the reference data set to consider observational uncertainty that is introduced by differences between different instruments or meteorological reanalyses. In addition, Taylor diagrams, which are a common graphical summary to evaluate climate models, have been shown. These diagrams display the normalized standard deviation, the centered RMSD and the pattern correlation between the model simulations and the observations.
The main differences due to the setup of the simulations (free-running
vs. nudged) are introduced through differences in the meteorology. The
evaluation of the mean state of basic climate parameters is therefore
important in addition to the evaluation of ozone. This study show that
the mean state of temperature, eastward wind, northward wind,
geopotential height, specific humidity, and radiation is in general
well represented by the four EMAC simulations. Some differences exist
in specific regions and altitudes which are related to the different
setups. In particular we find a cold bias
(
The evaluation of tropospheric ozone and ozone precursors
(
Tropospheric column ozone is generally overestimated compared to
satellite observations, but the annual cycle of total column ozone is
well represented. The high bias in tropospheric column ozone motivated
two additional simulations that are identical to the ACCMIP simulation
except for a code modification to avoid unrealistically high
convective cloud water and ice contents for scavenging (ACCMIP-S1),
and an additional modification in the chemical mechanism
(ACCMIP-S2). ACCMIP-S2 includes a possible
Biases in ozone precursors exist but are strongly dependent on the
inventory used. For example, the evaluation of CO showed an
underestimation compared to observations in all EMAC simulations,
particularly in regions with anthropogenic influence.
The ACCMIP simulation with its
different emission inventory from
Evaluating ozone and ozone precursors with aircraft data has been
proven as important in this and many previous studies. It would be
important to update existing climatologies like the one by
In addition, with growing complexity of chemistry-climate and earth
system models, we advocate routine evaluation of models to be
facilitated by common software tools that are made available to the
community. All diagnostics and performance metrics shown in this paper
are now implemented in the Earth System Model Validation Tool
(ESMValTool). They can be routinely reproduced and applied to new EMAC
simulations or other ESMs such as those participating in CCMI
The root-mean-square difference (RMSD), which is commonly used to
quantify performance of climate and numerical weather forecast models,
is defined as follows:
This metric has been considered (among others) by
Additionally the overall mean bias is calculated according to
Finally, in order to be able to focus on relative performance among
the different EMAC simulations, we normalize the RMSD and the overall
mean bias by dividing through the average across the
A further possibility to graphically summarize how closely a set of
modeled patterns matches observations is provided by the so called
Taylor diagram, which was originally proposed by
Thus, each model and each diagnostic will provide a distinct point on
the diagram. The closer the position of this point to the reference
position of the observation (
Note that the statistics given above are not independent,
particularly, adding the centered RMSD and the overall mean bias
Additionally to the already mentioned statistics, the Welch's
The difference of the mean between two variables
An error affecting the EMAC model output of the horizontal wind
components has been recently reported (M. Kunze, personal communication, 2014).
For the corresponding quantities, the intermediate state within the
leapfrog time filter was output instead of the finalized value. According to
the applied leapfrog time filter, this introduces an error of about 10
It is important to stress that this error does not affect the internal consistency of the model dynamics in any way, but concerns only the way the output is written. This error will be corrected in the upcoming release of EMAC.
The Modular Earth Submodel System (MESSy) is continuously further developed
and applied by a consortium of institutions. The usage of MESSy and access
to the source code is licensed to all affiliates of institutions which are
members of the MESSy Consortium. Institutions can be a member of the MESSy
Consortium by signing the MESSy Memorandum of Understanding. More information
can be found on the MESSy Consortium web-page (
The ESMValTool is currently under development and will be publicly
released only at a later stage. A stable version of the tool can be made
available upon request for development purposes. Interested users and developers
are welcome to contact the lead author. For further information and updates, see the
ESMValTool web-page at
This work was funded by the German Aerospace Center (DLR) Earth
System Model Validation (ESMVal) project. The implementation of the
performance metrics and diagnostics into the Earth System Model
Validation Tool (ESMValTool) was also supported by the European
Commission's 7th Framework Programme, under Grant Agreement number
282672, Earth system Model Bias Reduction and assessing Abrupt
Climate change (EMBRACE) project. We thank Diego Loyola (DLR,
Germany) for providing the GTO-ECV data (