MACC-II,III, Monitoring Atmospheric Composition and Climate, is the current pre-operational Copernicus Atmosphere Monitoring Service (CAMS). It provides data records on atmospheric composition for recent years, present conditions and forecasts (a few days ahead). To support the quality assessment of the CAMS products, the EU FP7 project Network Of ground-based Remote-Sensing Observations (NORS) created a server to validate the gridded MACC-II,III/CAMS model data against remote-sensing observations from the Network for the Detection of Atmospheric Composition Change (NDACC), for a selected set of target species and pilot stations. This paper describes in detail the algorithms used in this validation server. Amongst others, the algorithms take into account the horizontal displacement of the measured profiles from the location of the instrument, the vertical averaging and uncertainty propagation.
MACC-III, Monitoring Atmospheric Composition and Climate
(
List of NORS pilot instruments, stations, and target parameters. UVVIS.DOAS.ZENITH (OFFAXIS) stands for UV–visible (MAX)differential optical absorption spectroscopy (DOAS), FTIR for Fourier transform infrared spectrometer and MWR for microwave radiometer.
The validation processes carried out in NORS were created to be compliant
with best practices as defined by the international community: all validation
results include traceability information (see the Global Earth Observation
System of Systems (GEOSS), Quality Assurance for Earth Observation,
NORS is a demonstration project: it focuses on a limited number of
target data products from a limited number of pilot NDACC stations
representative of four major atmospheric regimes
(Table
The validation service is built such that it is easily expandable to a larger number of stations and instruments and to additional CAMS data products for which NDACC can provide independent reference data. The MACC-II,III validation subproject (VAL) has a focus on reactive gases and aerosol composition on a global scale, which coincides partially with the NORS/NDACC target species. Duplication with existing validation tasks in MACC is not an issue because MACC-II,III VAL is mostly using in situ surface data (AERONET, AErosol RObotic NETwork) or satellite total column data (e.g., Measurements of Pollution in the Troposphere (MOPITT) and Infrared Atmospheric Sounding Interferometer (IASI) CO data), as reference data for the validation of the MACC-II,III products.
VAL produces 3 monthly validation reports for the near-real-time services of
MACC-II,III, and 6 monthly validation reports (updates) for the reanalysis
services of MACC-II,III
(
The paper is organized as follows: in Sect.
List of GEOMS variable names common for all GEOMS templates (dimensions are indicative for 100 measurements on a grid with 47 layers).
Regarding notation conventions, all multiplications are scalar; matrix
multiplication is denoted by a central dot
MACC variables specifications. MMR is mass mixing ratio. The dimensions are indicative for a model with IFS resolution T255N128.
List of default
NORS data files are delivered in rapid delivery mode (not later than 1
month after acquisition) to the NDACC database rapid delivery directory, if
not fully in final form, or to the corresponding NDACC database station
directory if in its final form both with respect to data versioning (PI
reviewed vs. operational) and file temporal coverage. For each measurement
technique the NDACC data format has to be compliant with a pre-defined
template (see
Construction of MACC data times for type FC (forecast).
For each measurement technique, site and target species, a specific
sensitivity range can be determined. For further details on sensitivity
ranges and typical AVK's, see the data user document that was developed
within the framework of the NORS project
At present, the validation server validates in operational mode the following
forecast runs: near-real-time operation suite (NRT o-suite), the experiment
running the TM5 3D atmospheric chemistry-transport model and the experiment
with MOZART chemistry (for further details see
An IFS resolution is
denoted by T
For each of these experiment versions, the validation server generates
a time sequence of MACC model data with a time interval of 3
Table
The surface height variable
This section contains a detailed description of the calculation of
a vertical height grid for MACC data, starting from the vertical
pressure coordinate. The algorithm described here calculates directly
the height coordinate for the model pressure levels (i.e., the middle
of a layer) and not for the pressure interfaces (layer boundaries) of
the model as it is described in the IFS documentation. Throughout this
document
The pressure
All profile data (for the MARS variables go3, co, ch4, etc., except
aergn03) are defined on model pressure levels
To construct a vertical height vector out of the pressure grid, the server calculates the molar mass of humid air.
In the next algorithm, we use the fraction
The outer boundaries are determined from
The server checks that the lowest boundary does not become negative
and that the upper boundary does not exceed the top of atmosphere
The validation server will align the MACC data with the measurement data.
This means that all model profile data are re-gridded to the measurement's
vertical grid and converted to measurement's units. Typically the MACC
profile data are given in mass mixing ratio (MMR,
The partial column profile is derived from the ND profile, using the
layer thickness
The above formula with the layer thickness is also used to scale a profile of optical depths to an optical thickness profile.
Interpolation factors for grid layers: green source grid
layers are only partially overlapped by the
Re-gridding of profile data are done with conservation of mass in mind (or
total optical depth, in the case of aerosol data). As an example, assume that
a target profile (in partial column units for a concentration profile or
optical depth for an aerosol extinction profile) is defined on a vertical
height grid
If there is no overlap between
The dark gray layers in Fig.
The sum of all rows in the transformation matrix is a vector with the
dimension of the source grid that contains a coefficient of 1 for every
source layer that is completely covered by an external layer. The re-gridded
profile is obtained from matrix multiplication:
Section
Temporal co-location of MACC (model, black) and NORS (measurement, green) data.
A NORS measurement at time
This choice implies that a single MACC data instance at time
This type of co-location is used for all NDACC products; however,
depending on the nature of the species and the availability of
measurements, the window
Not all measurement techniques measure the state of the atmosphere directly above the instrument's location, e.g., FTIR measurements measure direct sunlight, and the probed column of air varies with the local measurement time. A similar situation occurs for UVVIS.DOAS.ZENITH and UVVIS.DOAS.OFFAXIS measurements. The vertical profile that is extracted from the MACC model at the site's location, should take this off location of the measured air mass into account.
The UVVIS template includes the latitude and longitude coordinate for the probed air mass at each height grid point (the latitude and longitude GEOMS variables). For FTIR measurements, the server uses an off-line routine to calculate the latitude and longitude GEOMS variables when not available in the NDACC file. Depending upon the availability of these variables, the server distinguishes two situations.
In this case a vertical profile is extracted at the site's location by
means of a standard bilinear interpolation to get from the MACC
latitude–longitude grid to a vertical profile at the instrument's
latitude and longitude at all altitudes. The horizontally interpolated
profile is re-gridded to the measurement vertical grid (i.e., the
external grid is
If the horizontal coordinates of the probed air mass for each measurement
layer are available, the co-located model profile is constructed per
measurement layer; i.e., the MACC re-gridded profile value for the
The values for the measurement grid that are not (or only partially) covered by the MACC grid are void.
The re-gridding algorithm described in Sect.
This implies that the validation is based on the model's partial column/optical depth profile and that further manipulation on both model and measurement data are done using equal conversion factors.
Aerosol optical depth profiles require one further manipulation. Typically,
the measurement's optical depth profile is measured at a specific wavelength
(say
Under the assumption that the Ångström coefficient is height
independent and the measurement's wavelength
In the following, it is understood that the re-gridded model profile, e.g.,
Example of a model
In some cases, the measurement data only provide a retrieved total column
(e.g., the UVVIS.DOAS.ZENITH measurements). In that case the above formula is
adapted such that the column averaging kernel is used. In this case
In order to get statistics on the validation results
As a general rule, the representation vertical grid is either a chosen
fixed grid from a measurement or from the MACC model. This choice is
such that the coarsest grid is chosen for representation purposes: for
LIDAR, the vertical grid of the measurement may be finer than the
model grid and in this case the representation grid is chosen to be
a fixed model grid. Re-gridding towards this fixed representation grid
Example of a
NDACC uncertainties can be reported as standard deviation values
The measurement uncertainty covariance matrix
The covariance matrices in the formula above are in partial column units
(e.g.,
To transform a covariance matrix from optical thickness units towards optical
depth, the same formula is used where
Depending on the measurement technique, site and species, the measurement
sensitivity may differ. Table
For long time periods, it is required to average data in time to improve the
readability or visibility of the validation statistics. For example, assume
that the monthly mean of a time series of O
This differs from the systematic uncertainty on the monthly mean:
This paper documents in detail generic tools for comparison between data sets
related to atmospheric composition, focusing on ground-based remote-sensing
data vs. gridded model data. Although comparisons between data sets from two
different sources have been performed for many years in the
atmospheric scientific community and the basic
concepts of co-location and comparisons are known, this is the first time
that generic tools have been developed, fully documented and implemented
successfully. It is also the first time that the effective location of the
remotely sensed air masses is taken into account for UVVIS and FTIR
measurements, at least in an approximate way. Differences in vertical
resolution of the data are also accounted for. The tools comply with the
QA4EO guidelines (see
The automatic application of the tools requires that the reference data formats comply with the GEOMS generic guidelines and specific templates per data type. During the development of the tools, it appeared that some GEOMS guidelines needed more precise specifications. Also, many inconsistencies in the data files have shown up and were corrected.
The addition of a new data type to the set already covered, is an easy task, because the tools consist of a succession of basic algorithms, of which many are identical for different data types. The tools can also be extended easily to comparisons between the ground-based remote-sensing data and satellite data (instead of gridded model data), by adding different co-location algorithms. Such algorithms have been developed previously in the context of the Generic Environment for Calibration/Validation Analysis (the GECA project, funded by ESA) and will be integrated in the toolset in the near future.
The NORS validation server is operational and example validation reports can
be viewed online at
List of GEOMS variable names, notations and corresponding GEOMS templates.
The code used by the validation server is built on the existing GECA Toolset
(Generic Environment for Calibration and Validation Activities), developed
within the ESA GECA project. It is foreseen that the GECA Toolset will become
publicly available as a separate component in BEAT (Basic Envisat Atmospheric
Toolbox) (
This work was developed under a EU FP7 project, NORS, under grant
agreement no. 284421. The NORS project relies on the data publicly available in
NDACC (