This paper presents a system to perform large-ensemble climate stochastic forecasts. The system is based on random analogue sampling of sea-level pressure data from the NCEP reanalysis. It is tested to forecast a North Atlantic Oscillation (NAO) index and the daily average temperature in five European stations. We simulated 100-member ensembles of averages over lead times from 5 days to 80 days in a hindcast mode, i.e., from a meteorological to a seasonal forecast. We tested the hindcast simulations with the usual forecast skill scores (CRPS or correlation) against persistence and climatology. We find significantly positive skill scores for all timescales. Although this model cannot outperform numerical weather prediction, it presents an interesting benchmark that could complement climatology or persistence forecast.

Stochastic weather generators (SWGs) have been devised to simulate many and
long sequences of climate variables that yield realistic statistical
properties

There are many categories of SWGs

SWGs that use observations as input could in principle be used to forecast
variables. This is the case for analogue weather generators

Due to uncertainties in observations and the high sensitivity to initial
conditions

The most trivial prediction systems are based on either climatology
(i.e., predicting from the seasonal average) or persistence (i.e., predicting
from the past observed values)

Machine learning algorithms were recently devised to simulate complex systems

This paper presents tests of such a system to forecast temperatures in Europe
and an index of the North Atlantic Oscillation (NAO). The NAO controls the
strength and direction of westerly winds and location of storm tracks across
the North Atlantic in the winter

Since the setup of such a system is fairly light, it is possible to test it for time leads from a meteorological forecast (5 days ahead) to a seasonal forecast (80 days ahead). We test this system in hindcast experiments to forecast climate variables between 1970 and 2010. The tests are performed with the usual skill scores (continuous rank probability score and correlation).

The paper is organized as follows. Section

We used data from different sources for sea-level pressure (SLP), NAO index, and temperatures. SLP data are used for analogue computations as a predictor. The NAO index and temperatures are the predictands (i.e., variables to be predicted). It is important that they share a common chronology, in order to allow their simulation because the NAO index and temperatures are simulated from SLP analogues.

We use the reanalysis data of the National Centers for Environmental
Prediction (NCEP)

One of the caveats of this reanalysis dataset is the lack of homogeneity of
assimilated data, in particular before the satellite era. This can lead to
breaks in pressure-related variables, although such breaks are mostly
detected in the Southern Hemisphere and the Arctic regions

The NAO is a major mode of atmospheric variability in the North Atlantic

We retrieved the daily NAO index from the NOAA web site:

The procedure to calculate the daily NAO teleconnection indices is detailed
on the NOAA web site. In short, a rotated principal component analysis (RPCA)
is applied to monthly averages of geopotential height at 500 hPa (Z500)
anomalies

The geographical domain on which this NAO index is computed is larger than
the one for SLP data.

We took daily averages of temperatures from the ECAD project

they start before 1948 and end after 2010. This allows the computation of analogue temperatures with the SLP from the NCEP reanalysis, which includes that period, and

they contain less than 10 % of the missing data.

Upper panel: North Atlantic region (blue rectangle) and western European region (red rectangle) on which analogues are computed.

Analogues of circulation are computed on SLP data from NCEP
(Sect.

Ensembles of simulations of temperature or the NAO index can be performed
with the rules illustrated by

inversely proportional to the calendar distance of the analogue dates

inversely proportional to the correlation of the analogue with the SLP
pattern at time

a zero weight if

Schematic of the iteration procedure to simulate one random
trajectory of temperature (TG) from SLP analogues. The values of

If we want to simulate a daily sequence starting at time

In this paper, the lead time

The

For comparison purposes, climatological and persistence forecasts are also
computed. The climatological forecast for a lead time

Illustration of average forecast for daily mean temperature (TG) in
Toulouse, for 1 January 2007. The continuous black line indicates the
observations of TG for the first 90 days of 2007. The colors indicate lead
times

This non-parametric weather generator (based on data resampling) is compared
to a parametric autoregressive simple model, based on a similar principle of
a relation between SLP and variables like temperature and NAO index. We build
a multi-variate autoregressive model of order 1

We then perform a multilinear linear regression between the five mean daily
temperature records (TG at Berlin, Toulouse, Orly, Madrid, and De Bilt) and
the NAO index:

The simplest score we use is the temporal correlation between the median of the ensemble forecast and the observations. Due to the autocorrelation and seasonality of the variables we try to simulate (temperature and NAO index), we consider the correlations for the forecast in the months of January and July.

The continuous rank probability score (CRPS) compares the cumulated density
functions of a forecast ensemble and observations

The score is the average over all times:

The CRPS is a fair score

The CRPS can be decomposed into reliability, resolution, and uncertainty
terms

The units of CRPS are those of the variable to be forecast; therefore, its
interpretation is not universal, and comparing the CRPS values for the NAO
index and temperatures is not directly possible. Therefore, it is useful to
compare the CRPS of the forecast with the one of a reference forecast. A
normalization of CRPS provides a skill score with respect to that reference:

The CRPSS indicates an improvement over the reference forecast. A perfect forecast has a CRPSS of 1. A positive improvement over the reference yields a positive CRPSS value. A value of 0 or less indicates that the forecast is worse than the reference.

We compute CRPSS for the climatological and persistence references. We used
packages “SpecsVerification” and “verification” in R to compute CRPS
decomposition and CRPSS scores. Hence we compare our stochastic forecasts
with forecasts made from climatology and persistence. By construction, the
persistence forecast shows an offset with the actual value ahead, because the
persistence is the value of the average of observations between

We tested the ensemble forecast system on the period between 1970 and 2010.
We simulate

We recall that the tests we perform are on the

The CRPS and CRPSS are computed for each value of lead times

We performed our stochastic forecasts on the NAO index and European
temperatures with the analogue stochastic weather generator and the mAR1 model.
The two datasets (NAO and temperature) are treated separately because the
simulations are done with two different analogue computations
(Sect.

For illustration purposes, we comment on the skill of simulations of 2007.
Fig.

The

individual simulated trajectories tend to “collapse” toward a climatological value after

taking the median of all simulations also naturally reduces the variance.

Left column: time series of analogue ensemble forecasts for 2007, for
lead times

The correlation, CRPS reliability, and CRPSS values for NAO index forecast
are shown in Fig.

The CRPS reliability values range from

On the one hand, the CRPSS

The correlation scores decrease with lead time

Skill scores for the NAO index for lead times

For comparison purposes, the multivariate autoregressive model NAO time
series are shown in Fig.

Multivariate autoregressive (mAR1) model time series of ensemble
forecasts for 2007, for lead times

The correlation and CRPSS values for daily mean temperature (TG) forecast are
shown in Fig.

The CRPSS

The CRPSS values are rather consistent for four of the stations (Toulouse, De Bilt, Berlin, and Orly). The stochastic model CRPSS fares slightly worse at Madrid station.

The CRPS reliability values are shown in Fig.

The correlation scores for January and July decrease with lead time

Skill scores for mean daily temperature in Toulouse and De Bilt,
for lead times

The mAR1 system for temperature is not designed to yield a seasonal cycle (contrary to the analogue system). Therefore, the skill scores of this system for temperatures are negative (for CRPSS) or with non-significant correlations.

We have presented a system to generate ensembles of stochastic simulations of
the atmospheric circulation, based on pre-computed analogues of circulation.
This system is fairly light in terms of computing resources as it can be run
on a (reasonably powerful) personal computer. The most fundamental assumption
of the system is that the variable to be predicted is linked to the
atmospheric circulation. The geographical window for the computation of
analogues needs to be adjusted to the variable to be predicted, so that prior
expertise is necessary for this analogue forecast system. This implies that
this approach would not be adequate for variables that are not connected in
any way to the atmospheric circulation (here approximated by SLP). The use of
other atmospheric fields (e.g., geopotential heights) might increase the
skill of the system. The computation of analogues with other parameters
(geographical zone, atmospheric predictor, type of reanalysis, climate model
output, etc.) can be easily performed with a web processing service

We have tested the performance of the system to simulate an NAO index and
temperature variations in five European stations. The performance of such a
system cannot beat a meteorological or seasonal forecast with a full-scale
atmospheric model

The reason for the positive skill (especially against climatology) remains to
be elucidated, especially for lead times longer than 20 days. We conjecture
that the information contained in the initial condition (as done with regular
weather forecasts) actually controls the mean behavior of the trajectories
from that initial condition. But such a skill is actually “concentrated” in
the first few days, because the trajectories tend to converge to the
climatology after 20 days. The combination of several skill scores shows that
such a system is not appropriate for ensemble forecasts beyond lead times of
40 days, which is lower than what is reported by

Although the forecast system is random, it contains elements of the dynamics of the atmosphere, from the choice of the analogues. This system is consistently better than a simple multivariate autoregressive (mAR1) model for lead times shorter than 20 days. Since the seasonal cycle is naturally embedded in the analogue simulations, there is no need to parameterize it, in contrast to the mAR1 model.

Recent experimental results in chaotic systems have shown that a well-tuned
neural network algorithm could simulate efficiently the trajectories of a
chaotic dynamical system

This system was tested on temperature for five European datasets. This could be extended to precipitation or wind speed. If a real-time forecast is to be performed, we emphasize that only the predictor (here, SLP) needs to be regularly updated for the computation of analogues.

The goal of such a system is not to replace ensemble numerical weather/seasonal forecast. Rather, it can refine the usual references (climatology and persistence) for the evaluation of skill scores. This would create a third “machine learning” reference for CRPSS that might be harder to beat than the classical references.

The code for the computation of analogues is available at
(free CeCILL license)

The code for simulations is available at

The temperature data are available at

The NAO index data are available at

The NCEP reanalysis SLP data are available at

PY wrote the codes and designed the experiments. CD participated in the writing of the manuscript.

The authors declare that they have no conflict of interest.

This work was supported by a grant from Labex-IPSL and ERC grant no. 338965-A2C2. We thank Mariette Lamige and Zhongya Liu, who performed preliminary analyses during their training periods. We thank the two anonymous reviewers for their suggestions to use the CRPS decomposition and a simple parametric stochastic model. Edited by: James Annan Reviewed by: two anonymous referees