Estimating and diagnosing model error variances in the M et eo-France global NWP model
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Estimating and diagnosing model error variances in the Météo-France global NWP model 1
Estimating and diagnosing model error variances in the
Météo-France global NWP model
M. Boisserie∗ , P. Arbogast, L. Descamps, O. Pannekoucke, L. Raynaud
CNRM/GAME (Météo-France, CNRS), Toulouse, France
A methodology for estimating model error statistics is proposed in this paper. Its
application to the global operational model ARPEGE of Météo-France provides
valuable insights into the spatio-temporal dynamics of model error variances. In
particular larger model errors are found in the mid-latitude storm tracks (high
cyclonic activity) for dynamical variables, such as 500-hPa geopotential height
and the 850-hPa wind speed.
The average model errors over both hemispheres show a linear growth until
reaching saturation. Model errors are also shown to grow more rapidly
than predictability errors which leads, after a certain time, to a crossover
time beyond which model error contribution to forecast error starts playing
the dominant role. Moreover, model errors saturate more rapidly than
predictability errors.
On the other hand, the spectral analysis shows an upscale energy transfer and a
faster growth at synoptic scales for both model errors and predictability errors.
This indicates that, for dynamical variables, the growth of both errors is most
likely driven by baroclinic instability.
The results found in this study could provide valuable information for a
future implementation of a stochastic physics approach to account for model
errors in the operational ensemble prediction system. Copyright c 2011 Royal
Meteorological Society
Key Words: Model error, ensemble forecasting, NWP
Received . . .
Copyright c 2011 Royal Meteorological Society Q. J. R. Meteorol. Soc. 00: 2–13 (2011)
Citation: . . .
Prepared using qjrms4.clsQuarterly Journal of the Royal Meteorological Society Q. J. R. Meteorol. Soc. 00: 2–13 (2011)
1. Introduction purpose of the present paper is to fill this gap by diagnosing
model error variances in an operational NWP model.
The two basic ingredients for an accurate global weather
forecast are the production of perfect initial conditions
Daley (1992) proposed a methodology to estimate model
and a perfect numerical weather prediction (NWP) model.
error covariances using an ensemble of initial conditions
However, errors in the production of initial conditions are
evolved by a NWP model. In contrast with that study,
unavoidable since this is an under-determined problem.
which uses the so-called standard National Meteorological
Moreover, although NWP models have tremendously
Center (NMC) method (Parrish and Derber 1992), here the
increased in complexity over the years, a model is by
operational Ensemble Data Assimilation (EDA) system of
definition only an approximation of reality. Model errors
Météo-France is used (Berre et al. 2007). An EDA system
result for instance from approximations in the dynamics
is expected to provide a more reliable estimation of the
formulation, physical parameterizations, or numerical
predictability error variances and in turn a more reliable
schemes. Therefore, a weather forecast will always be
estimation of model error variances than the NMC method
associated with an error that scientists strive to reduce.
(Pereira and Berre 2006; Berre et al. 2006).
Since the early 1990s, ensemble forecasting (EF)
techniques have been commonly used by major operational
While several studies have been devoted to the
centres worldwide to estimate the uncertainty of the
examination and description of background error or
deterministic numerical weather forecast and also provide
forecast error statistics (e.g., Lorenz 1982, 1984; Tribbia
forecasters with an ensemble of possible weather scenarios
and Baumhefner 2004; Pereira and Berre 2006), few
in addition to the deterministic weather forecast. The design
studies have attempted to estimate model error statistics
of an ensemble prediction system consists of the generation
in a complex numerical weather prediction system so
of multiple model runs accounting for the major sources of
far. In the present paper, we analyze spatio-temporal
forecast error. Over the past 20 years, more efforts have
characteristics of model error variances. In particular,
been given to represent the contribution of errors arising
diagnosing the predictability errors along with the model
from the production of initial conditions (Toth and Kalnay
errors allows us to examine the impact of each error on
1993; Molteni et al. 1996). The representation of the model
the atmospheric predictability, which is a crucial issue in
error contribution is a more challenging task since the model
weather forecasting.
error sources are diverse and only partially known.
In the past decade, different methods have been
On the other hand, the results of this study can
proposed to represent model error contributions in EF. The
provide useful information for improving the model error
methods currently used include multi-model/multi-physics
representation in the operational EF of Météo-France.
approaches (Houtekamer et al. 1996; Stensrud et al. 2000),
additive and multiplicative inflations (Constantinescu
et al. 2007; Anderson 2001), stochastic physics such as The paper is organized as follows. The methodology used
Stochastic Kinetic Energy Backscatter (SKEB, Shutts 2005) to estimate model error standard deviations is described in
or the Stochastically Perturbed Parameterization Tendencies section 2. The spatio-temporal analysis along with a spectral
(SPPT, Palmer et al. 2009). However, most of these methods analysis of both model errors and predictability errors are
are rather empirical, in the sense that they are not based on discussed in section 3. Finally, conclusions and future works
the knowledge of model error statistical characteristics. The are presented in section 4.
Copyright c 2011 Royal Meteorological Society Q. J. R. Meteorol. Soc. 00: 2–13 (2011)
Prepared using qjrms4.clsEstimating and diagnosing model error variances in the Météo-France global NWP model 3
2. Methodology approximated as:
2.1. Estimation of model error covariances pi = Mi a + r[a ], (4)
In this section, a methodology to estimate model error where Mi is the tangent linear model along the trajectory
covariances at each forecast lead time (up to 10-day Mi (xa ) and r[a ] is the residual of the Taylor serie
forecast) is presented. expansion. One must note that, in constrat with several
studies on model error dynamics (e.g. Daley 1992; Nicolis
Let xt0 be the true state at the initial time t0 . At time ti ,
2003, 2009), here the predictability error is not linearized.
the true state xti can be written as:
Moreover, in this study, ai EC is assumed to be negligible
as compared to the other errors. This assumption is most
xti + qi = Mi (xt0 ), (1)
likely correct for all ranges, expect at 6 hours. More
particularly, pi is most probably larger than ai EC but
where Mi represents the numerical weather forecast model
not negligible at 6 hours. Unfortunately, no estimation of
and qi the associated model error between t0 and time ti .
ai EC has been available to verify this assumption. The
The forecast state xfi at time ti is given by:
consequence of not meeting this assumption would be that
the methodology presented in this paper underestimates
xfi = Mi (xa ), (2)
model error variances at 6 hours. Hereafter, Eq.(3) is
simplified as follows:
a a
with x the analysis state valid at t0 and its associated
error (xa = xt0 + a ). Ideally, the true state xti is given by fi = pi + qi . (5)
observations. However, there are no gridded observations
of meteorological parameters available across the globe Finally, by taking the outer product (e.g. < u.v T > where
against which global forecast error can be estimated. .T denotes the transpose operator) of Eq.(5) with itself, it
Therefore, because it provides reliable global values of comes:
meteorological parameters, in this study we chose the
European Centre for Medium-Range Weather Forecasts < fi .(fi )T >=< pi .(pi )T > + < qi .(qi )T >
(ECMWF, Molteni et al. (1996)) analyses to represent
+ < pi .(qi )T > + < qi .(pi )T > .
the true state. Then, the forecast state is expressed as
xfi = xai EC + fi with xai EC = xti + ai EC where ai EC is
This latter equation can be expressed as follows:
the ECMWF analysis associated error at time ti .
Then, by subtracting Eq.(2) from Eq.(1), it comes: Pfi = Ppi + Qi + Cqp qp T
i + (Ci ) , (6)
fi + ai EC = pi + qi , (3) where Pfi =< fi .(fi )T > is the forecast error covariance
matrix, Ppi =< pi .(pi )T > is the predictability error
where pi = Mi (xa ) − Mi (xa − a ) is the predictability covariance matrix, Qi =< qi .(qi )T > is the model error
error (i.e. forecast error arising from errors in the covariance matrix between the initial time and time ti ,
initial conditions). Using the second-order Taylor series and Cqp p q T
i =< i .(i ) > is the covariance matrix between
expansion of Mi (xa − a ), the predictability error can be model error and predictability error. Therefore, from Eq.(6),
Copyright c 2011 Royal Meteorological Society Q. J. R. Meteorol. Soc. 00: 2–13 (2011)
Prepared using qjrms4.cls4 M. Boisserie
the model error covariance matrix Qi is estimated as: 2.2.1. Experimental design
The climatology of the model error and predictability error
Qi = Pfi − Ppi − Cqp qp T
i − (Ci ) . (7)
standard deviations (simply the square root of variances) is
calculated using a 6-member EF system. The EF system is
2.2. Use of an EDA system
initialized with the operational EDA system run of Météo-
France for 10 days using the Action de Recherche Petite
One of the original aspects of this study relies on the use Echelle Grande Echelle (ARPEGE) numerical prediction
of the operational EDA system of Météo-France (Berre model operational at Météo-France. ARPEGE uses a
et al. 2007) to estimate predictability error (i.e. analysis stretched horizontal grid with a finer resolution over France
error growth) variances whereas Daley (1992) used the so- regridded to a regular grid of 1.5o horizontal resolution.
called standard NMC method. The NMC method consists The operational EDA system is based on explicit
of using an analysis ensemble by generating an ensemble perturbations of the observations in 4D-VAR assimilation
of successive forecasts valid at the same time but with cycles, while the backgrounds are implicitly perturbed. It
different ranges (e.g. 6h and 30h). In the past decade, provides six perturbed analysis states at a T399 spectral
EDA systems have been proposed as a promising alternative triangular truncation on 70 vertical levels (Berre et al.
method to generate analysis ensembles (Houtekamer et al. 2007).
1996; Fisher 2003). Berre et al. (2006) found that an To analyze the growth and the vertical variability
EDA system leads to a better representation of the analysis of model errors and predictability errors, their standard
errors compared to the NMC method. Moreover, this latter deviations are averaged across the Northern Hemisphere
paper has demonstrated formally that the evolution of (between 20o N and 90o N) and the Southern Hemisphere
the ensemble analysis perturbations is the same as the (between 90o S and 20o S). Both errors are calculated for
evolution of the exact analysis error. This indicates that, if two dynamical variables, the geopotential height at 500 hPa
observation error statistics are perfectly known, the spread (Z500) and the wind speed at 850 hPa (Ws850). Finally,
of the EDA system must reflect fairly well the predictability in order to compute robust statistics, these matrices are
error variances. However, observation error statistics are calculated over a 59-day winter period (January 16 to March
not perfectly known, which can then affect the accuracy 15 of 2010) and a 60-day summer period (June 1 to August
of the analysis error. Nevertheless, based on Desroziers 31 of 2010). Thus, this provides a climatological estimation
et al. (2005), a technique is applied at Météo-France of the error variances, denoted hereafter Q, Pf , Pp and
to tune observational error variances in order to retrieve Cqp .
correct estimates of observation error variances, which thus
hampers this latter issue. 2.2.2. Estimation of Pf
In the remainder of the paper, we will solely focus on the The forecast error corresponds to the difference between
estimation of model error variances. Therefore, although we the forecast state and the true state. As mentioned earlier,
use the same notation, the matrices Qi , Pfi , Ppi , and Cqp
i ECMWF analyses are used in this study to represent the
will be considered hereafter diagonals. Then, it yields that true state and its associated error is neglected. We choose to
Cqp
i = (Cqp
i )
T
and Eq.(7) can be written as: represent the forecast state by the forecast ensemble mean.
For each day i of the period, the mean squared forecast
Qi = Pfi − Ppi − 2Cqp
i . (8) errors are computed at 18 UTC at the forecast range r (r
Copyright c 2011 Royal Meteorological Society Q. J. R. Meteorol. Soc. 00: 2–13 (2011)
Prepared using qjrms4.clsEstimating and diagnosing model error variances in the Météo-France global NWP model 5
varies from 6 hours to 10 days) and for the considered Although model errors have been recently taken into
meteorological variable v (i.e. Z500 and Ws850) as follows: account in the operational EDA system (Raynaud et al.
2011), in this study, the EDA system is run in the perfect-
model framework. As a consequence, the ensemble is found
N
1 X f to underestimate the magnitude of the predictability errors.
Pf (v, r) = (v, r)2 =
N i=1 i
This problem was simply solved by calculating the lack
N
1 X f
{x (v, r) − xai EC (v, 0)}2 , (9) of the EDA variance by assuming that model errors are
N i=1 i
negligible at 6 hours. In orther words, we assume that
predictability error variances are equal to forecast error
where N is the number of days over which the climatology
variances as follows:
is computed, xai EC (v, 0) is the ECMWF analysis for the
variable v valid at day i, and xfi (v, r) is the forecast
Pp (v, r = 6h) + Ppdef icit (v, r = 6h) = Pf (v, r = 6h)
ensemble mean starting at i − r with forecast range r, and
thus valid at day i. It may be mentioned that the forecast
then, we deduce this lack of variance as:
error bias is neglected when estimating Pf according to
Eq.(9). It has been verified that the forecast error can
Ppdef icit (v) = Pf (v, r = 6h) − Pp (v, r = 6h)
be assumed unbiased on average over the Northern or
Southern Hemisphere (not shown). Nevertheless, in the
.
sparse regions where the forecast error bias is not negligible,
the methodology presented in this paper overestimates This lack of the EDA variance is then added to the
model error variances since part of the quantity Q is due predictability error variances at each forecast lead time as
to mean squared bias (not shown). follows:
Pp∗ (v, r) = Pp (v, r) + Ppdef icit (v)
p
2.2.3. Estimation of P
In the remainder of the paper, the climatological estimation
The predictability error corresponds to the analysis error
of predictability error variances takes this lack of variance
growth and is calculated using the operational EDA system
into account, thus Pp = Pp∗ .
of Météo-France. The predictability error variance at
forecast range r and for the considered meteorological
variable v is computed as the ensemble variance:
2.2.4. Estimation of Cqp
N
1 X p
Pp (v, r) = (v, r)2 =
N i=1 i
N M =6
As shown by Eq.(8), the estimation of covariances between
1 1 X X f,k f 2
= {xi (v, r) − xi (v, r)} , (10) model errors and predictability errors is also needed. Here,
N (M − 1) i=1
k=1
we present how these covariances are estimated.
where M is the EDA size and xf,k
i (v, r) is the forecast By taking the outer product of Eq.(5) with the
ensemble member k starting at i − r with forecast range predictability error pi , we obtain:
r, and thus valid at day i, and xfi (v, r) is the associated
ensemble mean. < fi .(pi )T >=< pi .(pi )T > + < qi .(pi )T > .
Copyright c 2011 Royal Meteorological Society Q. J. R. Meteorol. Soc. 00: 2–13 (2011)
Prepared using qjrms4.cls6 M. Boisserie
The above equation can be written as: over the Northern or Southern Hemisphere). Figure 1 shows
the correlation as a function of forecast ranges. The gray
Cfi p = Ppi + Cqp
i , (11) zone indicates at which correlation value model errors are
considered uncorrelated with predictability errors. Then,
where Cfi p is the covariance matrix between predictability it is found that, for both studied variables and in both
errors and forecast errors. Since we cannot directly estimate hemispheres and seasons, model errors are uncorrelated
the term Cqp q
i (because the value of i is unknown), from with predictability errors up to about 1 day.
Eq.(11) we deduce indirectly Cqp
i as follows:
3. Diagnosis of model error standard deviations
Cqp fp p
i = Ci − P i .
3.1. Spatial distribution
Figure 2 presents the spatial distribution of model error
Averaged over the studied winter and summer periods, the standard deviations for Z500 and Ws850 at three forecast
above equation can be written as follows: lead times (6 hours, 5 days and 10 days) for the
winter period. For both variables and at 6 hours, model
qp fp p
C (v, r) = C (v, r) − P (v, r). error standard deviations show relatively unorganized and
small-scale structures. From day 5, model error standard
Then, while the matrix Ppi is given by Eq.(10), the matrix deviations become more organized; in particular, large-scale
Cfi p can be estimated as follows: structures develop within the mid-latitude storm track. The
mid-latitude storm track is often referred to as a region of
N
1 X f 1 1 cyclonic activity where model errors are then expected to be
Cf p (v, r) = (v, r)pi (v, r) =
N i=1 i N (M − 1)
relatively large. Moreover, model error standard deviations
N M =6
are very low in the Tropics. This time evolution of error
X X
{xf,k f f aEC
i (v, r) − xi (v, r)}{xi (v, r) − xi (v, 0)},
i=1 k=1 structures then suggests an energy cascade of the model
errors from small to larger scales (this will be further
Similarly to the matrices Pf and Pp , the matrix Cqp is
detailed in section 3.3).
considered as diagonal.
Similar spatial distributions of model error standard
qp
To determine whether C can be neglected, we calculate deviations are obtained during the summer period (not
the correlation between model errors and predictability shown). Finally, model errors are found to be larger during
errors at different forecast ranges averaged in absolute the winter of both hemispheres. This may reflect the
value over all grid points in the Northern Hemisphere or cyclonic activity enhancement occurring in the mid-latitude
the Southern Hemisphere. The threshold below which the bands during the winter.
correlation can be neglected is determined by calculating
3.2. Error growth
the upper 95% quantile of the correlation between two
samples drawn from normal distributions (by definition Figure 3 presents the time evolution of the average model
these two samples are independent and thus uncorrelated) error and predictability error standard deviations across the
using a bootstrap method. The sample size is 6 (i.e. the EDA Northern Hemisphere and Southern Hemisphere for both
size used in this study to compute Cqp ) and the absolute studied summer and winter periods.
correlation between these two samples is computed and First, as expected, predictability errors dominate model
averaged over 12 280 values (i.e. number of grid points errors at very short times. At 6 hours, the magnitude of
Copyright c 2011 Royal Meteorological Society Q. J. R. Meteorol. Soc. 00: 2–13 (2011)
Prepared using qjrms4.clsEstimating and diagnosing model error variances in the Météo-France global NWP model 7
predictability errors is about three times larger than that of 3.3. Spectral analysis
model errors for both variables and both seasons.
To further examine the dynamics of predictability errors
The time evolution of each error adopts a different
and model errors, the evolution of the spectral horizontal
behaviour. The predictability errors follow to some extent
variances of both errors are computed. In particular, the
the typical error growth pattern of a nonlinear dynamical
variance spectra of both errors for the 500-hPa kinetic
model. They first grow exponentially obeying linear error
energy at lead times from 6 hours to day 10 averaged
dynamics then slow down during the nonlinear error
over the studied winter period are computed (Figure 4).
growth phase until reaching saturation. The saturation
For comparison purposes, the 500-hPa spectral density of
phase is particularly noticeable for Ws850 in the Northern
model kinetic energy and that of the ECMWF analysis
Hemisphere. For Z500 and for Ws850 in the Southern
kinetic energy at day 5 are also displayed. These spectral
Hemisphere, predictability errors must saturate beyond a
error variances are simply computed by applying the
10-day forecast range.
methodology (described in section 2) in spectral space. The
spectral forecast error variance Pfn (v, r) of the variable v
For model errors, a linear growth for short times until
with forecast range r is then expressed as the sum of the
reaching saturation is found. It is interesting to note that
individual modal variances:
this dynamical behaviour of model errors is in accordance
with theoretical results from Nicolis (2003) who formally
demonstrates using a linearized theory that model error
n N
X 1 X f
standard deviations start with a linear growth. Moreover, Pfn (v, r) = [ (n, m, v, r)]2 =
m=−n
N i=1 i
the model error growth appears to be faster than that of n N
X 1 X
predictability errors. Because of this rapid growth, after [xfi (n, m, v, r) − xai EC (n, m, v, r)]2 , (12)
m=−n
N i=1
a certain time, model errors start playing the dominant
role. This crossover time depends on the studied variable
where fi (n, m, v, r) is the spectral forecast error for the
and the spatial domain of interest. This latter result is
particular mode (m, n) with m the zonal wave number
again consistent with theoretical results from Nicolis (2009)
and n the total wave number, xfi (n, m, v, r) is the spectral
based on a linearized theory. One can also note that, for
transform of the mean forecast ensemble starting at i − r,
both variables and both hemispheres, model errors reach
and thus valid at day i and, xai EC (n, m, v, r) is the spectral
saturation more quickly than predictability errors.
transform of the ECMWF analysis valid at day i.
The comparison of the model error growth between The spectral predictability error variance Ppn (v, r) of the
the two studied seasons confirms the earlier result found variable v with forecast range r is given by:
in Figure 2 (model errors are larger in the winter of
both hemispheres). Finally, during both seasons, model
n N
errors are larger in the Southern Hemisphere than in the X 1 X p
Ppn (v, r) = [ (n, m, v, r)]2 =
N i=1 i
Northern Hemisphere. This result is not surprising since m=−n
n N M
the operational model ARPEGE used in this study is well-
X 1 1 XX f
[ [xi (n, m, v, r) − xfk,i (n, m, v, r)]2 ],
m=−n
N (M − 1) i=1
known to be less skilful in the Southern Hemisphere as it k=1
(13)
uses a stretched grid centered over France.
Copyright c 2011 Royal Meteorological Society Q. J. R. Meteorol. Soc. 00: 2–13 (2011)
Prepared using qjrms4.cls8 M. Boisserie
where xfk,i (n, m, v, r) is the spectral transform of the theoretical 2-dimensional turbulence power-law spectrum
forecast ensemble number k starting at i − r, and thus valid of n−3 verified against observations by Nastrom and
at day i. Gage (1985). However, for n > 100, the spectral density
By subtracting Eq.(13) from Eq.(12), we obtain the of the model kinetic energy starts droping off and thus
spectral variance of model errors: does not capture the transition to the n−5/3 theoretical
power-law behavior. This energy loss is related to a well-
Qn (v, r) = Pfn (v, r) − Ppn (v, r) − 2Cqp
n (v, r) (14) known source of model errors arising from numerical
diffusion altogether with subgrid-scale processes that are
Figure 4a indicates that the evolution of spectral misrepresented or unrepresented (Shutts 2005; Berner et al.
predictability error variances follows to some extent the 2009). While the spectral predictability error variances are
classical inverse energy cascade by Lorenz (1969). At first, found to be bounded by the model spectral kinetic energy
predictability errors are spread over a large range of scales density (Figure 4a), the spectral model error variances
(day 0.25) before reaching rapidly saturation (from day 1) are significantly larger the model kinetic energy density
at smaller scales (n >100). However, in contrast with the slope from about n =100 (Figure 4b). To better understand
classical study by Lorenz (1969) who found the fastest this latter result, the behavior of the spectral model error
error growth at small scales, here the fastest error growth variances has been investigated (Appendix 5.1 and 5.2).
is found in the range of wavenumbers between 6 and 80 First, it has been determined formally that, at scales for
within 5 days. This result is in agreement with Tribbia and which the model has a reasonable climatology, the spectral
Baumhefner (2004) who showed a faster error growth at model error variances cannot exceed 4 times the spectral
synoptic scales, and more particularly in the baroclinically energy density of the true flow. Figure 4b is in consistent
active band. with this theoretical result as the spectral model error
Similarly to the predictability errors, the spectral variances is not larger than 4 times the spectral energy
variances of model errors indicates an upscale energy density of the ECMWF analysis, which is considered in
cascade-type behaviour as their spectral slopes are less and this study as the true flow. Figure 4b is also in agreement
less steep than −3 with forecast ranges (Figure 4b). This with another theoretical result (see Appendix 5.2) that, for
is consistent with earlier results on model error spatial large wavenumbers, the spectral model error variances must
distributions showing the formation of more organized and be close to the spectral energy density of the true flow (i.e.
large-scale structures in the mid-latitude bands as time goes ECMWF analysis).
on (Figure 2). Another similarity with predictability errors
3.4. Error vertical profile
is that the fastest model error growth is found at synoptic
scales (n between 20 and 40). On the other hand, the main The vertical profiles of the average model error and
difference with predictability errors is the earlier saturation predictability error standard deviations across the Northern
at all scales. No significant differences in variance spectra Hemisphere and Southern Hemisphere at 6-hour and 6-day
of both errors are discerned between the two seasons (not lead times and for both periods are shown in Figure 5.
shown). First, it is clear that, for all variables, both errors show
It is interesting to note in Figure 4b that, for wavenumbers more variability with height as forecast time increases.
between 10 ans 100, the spectral density of the model For instance, while the magnitude of model errors is
kinetic energy (black thick curve) and that of the ECMWF overall constant at day 0.25, it increases significantly with
analysis kinetic energy (gray thick curve) follow the height at day 6 from the 700-hPa level until reaching
Copyright c 2011 Royal Meteorological Society Q. J. R. Meteorol. Soc. 00: 2–13 (2011)
Prepared using qjrms4.clsEstimating and diagnosing model error variances in the Météo-France global NWP model 9
a maximum at the 300-hPa level. In particular, in the mid-latitude bands can be explained by the high cyclonic
Southern Hemisphere, the model error magnitude almost activity.
doubles between 700-hPa and 300-hPa levels for Z and The relative growth of both average model error and
almost triples for Ws. This model error maximum near the average predictability error standard deviations across the
tropopause is related to the jetstream arising from intense Northern and Southern Hemispheres are compared. First, it
baroclinic instability. Then, these results are consistent with is found that, while both errors follow to some extent the
the earlier result of the fastest error growth being in the typical error growth behaviour of a nonlinear dynamical
baroclinic wavenumbers. Moreover, larger model errors are model, model errors show a faster linear growth at short
found in the winter of each hemisphere throughout the times before reaching saturation. It is also found that,
troposphere and stratosphere. after a certain time, model errors gradually build up and
start playing the dominant role in error dynamics. This is
4. Conclusion consistent with theoretical results demonstrated by Nicolis
(2003, 2009), which are based on a linearized theory.
This paper first proposes a methodology for estimating In addition, model errors saturate more rapidly than the
the model error covariance matrix. Then, using the predictability errors. However, one must keep in mind that
global operational model of Météo-France, spatio-temporal those results most probably depend on the meteorological
characteristics of model error standard deviations of two variables. In this study, the 500-hPa geopotential height
dynamical variables (Z500 and Ws850) are diagnosed and the 850-hPa wind speed are chosen to analyze model
and compared to those of predictability errors in both error characteristics, which are particularly good indicators
hemispheres and two seasons. The methodology consists of of synoptic activity. Finally, model errors are, on average,
subtracting predictability error variances from forecast error larger in the Southern Hemisphere than in the Northern
variances but without neglecting the correlation between Hemisphere. This is not surprising since the operational
these two errors. model ARPEGE used in this study is well known to be less
In contrast with Daley (1992) who uses the standard skilful in the Southern Hemisphere because of the use of a
NMC method to estimate the predictability error covari- stretched grid centered over France.
ances, the operational EDA system of Météo-France is used The spectral analysis shows that the fastest error
in this study. This choice is based on results from Berre growth for both errors occurs in about the same range
et al. (2006) who showed that an EDA system provides of wavenumbers corresponding to the synoptic scales.
a better representation of the analysis error growth than This suggests that the growth of both model errors and
the standard NMC method. In order to compute robust predictability errors is most likely driven by baroclinic
statistics, climatological model error standard deviations instability. Moreover, it has been determined formally that,
are estimated over a 2-month winter period and a 2-month at scales for which model has a reasonable climatology,
summer period of the year 2010. the spectral model error variances cannot exceed 4 times
For both dynamical variables, the spatial distribution the spectral energy density of the true flow. Beyond this
of climatological model errors standard deviations shows scale, the spectral model error variances must be close to
relatively unorganized and small-scale structures at short the spectral energy density of the true flow (i.e. ECMWF
ranges and more organized structures (large-scale features analysis in this study).
developing within the mid-latitude storm tracks) as the Finally, the vertical profiles of both errors have shown
forecast time increases. The large model errors found in the more variability with height as forecast time increases with
Copyright c 2011 Royal Meteorological Society Q. J. R. Meteorol. Soc. 00: 2–13 (2011)
Prepared using qjrms4.cls10 M. Boisserie
a maximum magnitude of both errors near the tropopause. 5. APPENDIX: Bounds of spectral model error
This maximum is most likely related to the jetstream. variances
Nevertheless, it should be remembered that the method-
5.1. Case 1: the model is close to the true weather
ology relies on several assumptions, which can limit the
reliability of the estimated model error variances. The The spectral model error variance is expressed as follows:
limitations are the following: 1) by neglecting the ECMWF
m=n
X
analysis error (used to represent the true state) the estimated q
Vn ( ) = E[ |q (n, m) − q (n, m)|2 ],
m=−n
model error variances could be underestimated at 6 hours, 2)
the lack of knowledge about observation error correlations where E and . are both the mathematical expectation, and
could reduce the reliability of the estimated predictability q (n, m) is the model error for the particular mode (n, m).
error variances, and 3) in sparse regions, the methodology The model error at time ti is defined as:
presented in this paper overestimates model error variances
since part of the quantity Q is due to mean squared bias (not q (n, m) = Mi (xt0 (n, m)) − xti (n, m).
shown).
A natural extension of this work would be to where Mi is the numerical weather forecast model,
diagnose flow-dependent model error variances instead xti (n, m), and xt0 (n, m) are the spectral true flow for the
of climatological values. In addition, while this study particular mode (m, n) at time ti and t0 respectively.
concentrates on the error variances, it will be interesting Applying the triangle inequality to A = |q (n, m) −
to also diagnose model error correlations. Finally, all q (n, m)|, we obtain the two following inequalities:
the results shown in this paper could provide valuable
information for the implementation of a stochastic method
to represent model errors in the operational EF system of ||Mi (xt0 (n, m)) − Mi (xt0 (n, m))| − |xti (n, m) − xti (n, m)|| ≤ A
Météo-France . (15)
and,
A ≤ |Mi (xt0 (n, m)) − Mi (xt0 (n, m))| + |xti (n, m) − xti (n, m)|,
(16)
Assuming that, for each mode, the spectral model flow
coincides with that of the truth in terms of ampli-
tude, it comes that |Mi (xt0 (n, m)) − Mi (xt0 (n, m))| =
|xti (n, m) − xti (n, m)|. Then, squaring the two inequalities
above, we obtain:
0 ≤ |q (n, m) − q (n, m)|2 ≤ 4|xti (n, m) − xti (n, m)|2 ,
By summing over m and taking the expectation to the
inequality above, it comes:
0 ≤ Vn (q ) ≤ 4Vn (xti ), (17)
Copyright c 2011 Royal Meteorological Society Q. J. R. Meteorol. Soc. 00: 2–13 (2011)
Prepared using qjrms4.clsEstimating and diagnosing model error variances in the Météo-France global NWP model 11
Pm=n
where Vn (xti ) = E[ m=−n |xti (n, m) − xti (n, m)|2 ]. This wavenumbers, the assumption of a reasonable climatology
upper bound is found by considering that the model flow made in the appendix above is no longer valuable. From the
and the true flow are out of phase (i.e. model error phase two inequalities in Eqs.(15 and 16), it comes:
is maximum). By definition of the variance, we can express
Vn (xti ) as follows: |Vn (Mi (xt0 )) + Vn (xti ) − 2|covn (xt0 , xti )|| ≤ Vn (q ),
(19)
and,
m=n
X m=n
X
Vn (xti ) = E[ |xti (n, m)|2 ] − E2 [ |xti (n, m)|]
m=−n m=−n
Vn (q ) ≤ Vn (Mi (xt0 )) + Vn (xti ) + 2|covn (xt0 , xti )|. (20)
and therefore,
Because the model flow and the true flow becomes
m=n
X
Vn (xti ) ≤ E[ |xti (n, m)|2 ]. (18) completely decorrelated as the lead time increases,
m=−n
covn (xt0 , xti ) = 0. The two inequalities above can thus be
Finally, from Eq.(17) it comes: simplified as follows:
m=n
|Vn (Mi (xt0 )) + Vn (xti )| ≤ Vn (q ) ≤ Vn (Mi (xt0 )) + Vn (xti ).
X
0 ≤ Vn (q ) ≤ 4E[ |xti (n, m)|2 ].
m=−n (21)
Then, using the definition of the variance, such as in
The term on the right hand-side of the inequality above
Eq.(18), from the inequality above we can write:
corresponds to the spectral energy density of the true flow
Pm=n
(hereafter called En (xti ) = E[ m=−n |xti (n, m)|2 ]). This
result indicates that, at wavenumbers where the model has
En (Mi (xt0 )) + En (xti ) ≤ Vn (q ) ≤ En (Mi (xt0 )) + En (xti ).
a reasonable climatology, the spectral model error variances
(22)
are bounded by 4 times the spectral energy density of the
Typically, in most numerical weather forecast model, a drop
true flow. By performing a similar reasoning for the spectral
off of the spectral energy density at large wavenumbers is
variances of predictability errors, we obtain:
shown as they does not capture the transition to the n−5/3
theoretical power-law behavior (thus, En (Mi (xt0 ))12 M. Boisserie
References Efron B, Tibshirani R. 1993. An Introduction to the
Bootstrap : 436pp.
Fisher M. 2003. Background error covariance modelling.
Anderson J. 2001. An ensemble adjustment filter for data
Proceedings of the ECMWF seminar on recent devel-
assimilation. Mon. Wea. Rev. 129: 2884–2903.
opments in data assimilation for atmosphere and ocean,
Berner J, Shutts GJ, Leutbecher M, Palmer TN. 2009. A
ECMWF : 45–63.
Spectral Stochastic Kinetic Energy backscatter Scheme
Houtekamer P, Lefaivre L, Derome J, Ritchie H, Mitchell
and Its Impact on Flow-Dependent Predictability in the
HL. 1996. A system simulation approach to ensemble
ECMWF Ensemble Prediction System. J. Atmos. Sci. 66:
prediction. Mon. Wea. Rev. 124: 1225–1242.
603–626.
Lorenz EN. 1969. Atmospheric predictability as revealed by
Berre L, Pannekoucke O, Desroziers G, Stefanescu SE,
naturally occurring analogues. J. Atmos. Sci 26: 636–646.
Chapnik B, Raynaud L. 2007. A variational assimilation
Lorenz EN. 1982. Atmospheric predictability experiments
ensemble and the spatial filtering of its error covariances:
with a large numerical model. Tellus 34: 505–513.
Increase of sample size by local spatial averaging.
Lorenz EN. 1984. Estimates of atmospheric predictability
ECMWF Workshop on Flow-Dependent Aspects of Data
in the medium range. Predictability of Fluid Motions:
Assimilation, Reading, United Kingdom, ECMWF : 151–
A.I.P. Conference Proceedings, 106: American Institute
168.
of Physics, La Jolla Institute, 133–144.
Berre L, Stefanescu S, Pereira MB. 2006. The represen-
Molteni F, Buizza R, Palmer TN, Petroliagis T. 1996. The
tation of analysis effect in three error simulation tech-
ECMWF Ensemble Prediction System: Methodology and
niques. T. Tellus 58A: 196–209.
validation. Quart. J. Roy. Meteor. Soc. 122: 3–119.
Candille G, Cote C, Houtekamer PL, Pellerin G. 2007.
Nastrom GD, Gage KS. 1985. A climatology of atmos-
Verification of an Ensemble Prediction System against
pheric wavenumber spectra of wind and temperature
Observations. Mon. Wea. Rev. 135: 2688–2699.
observed by commercial aircraft. J. Atmos. Sci. : 950–
Candille G, Talagrand O. 2008. Impact of observational
960.
error on validation of ensemble prediction systems. Q. J.
Nicolis C. 2003. Dynamics of model error: Some generic
R. Meteorol. Soc. 134: 509–521.
features. J. Atmos. Sci. 60: 2208–2218.
Constantinescu E, Sandu A, Chai T, Carmichael G. 2007.
Nicolis C. 2009. Dynamics of prediction errors under the
Ensemble-based chemical data assimilation. i: General
combined effect of initial condition and model errors. J.
approach. Q. J. R. Meteorol. Soc. : 1229–1243.
Atmos. Sci. 66: 766–778.
Daley R. 1992. Estimating model error covariances for
Palmer T, Buizza R, Doblas-Reyes F, Jung T, Leutbecher
application to atmospheric data assimilation. Mon. Wea.
M, Shutts G, Steinheimer M, Weisheimer A. 2009.
Rev. 120(8): 1735–1746.
Stochastic parametrization and model uncertainty, 42 pp,
Desroziers G, Berre L, Chapnik B, Poli P. 2005. Diagnosis
ECMWF, Reading, UK. ECMWF Technical Memoran-
of observation, background and analysis-error statistics
dum No. 598 .
in observation space. Q. J. R. Meteorol. Soc 131: 3385–
Parrish D, Derber J. 1992. The National Meteorological
3396.
Center spectral statistical interpolation analysis system.
Desroziers G, Ivanov S. 2001. Diagnosis and adaptive
Mon. Wea. Rev. 120: 1747–1763.
tuning of observation-error parameters in variational
assimilation. Q. J. R. Meteorol. Soc 127: 1433–1452.
Copyright c 2011 Royal Meteorological Society Q. J. R. Meteorol. Soc. 00: 2–13 (2011)
Prepared using qjrms4.clsEstimating and diagnosing model error variances in the Météo-France global NWP model 13
Pereira MB, Berre L. 2006. The use of an ensemble
approach to study the background error covariances in a
global NWP model. Mon. Wea. Rev. 134: 2466–2489.
Raynaud L, Berre L, Desroziers G. 2011. Accounting
for model error in the Météo-France ensemble data
assimilation system. Q. J. R. Meteorol. Soc. : (DOI:
10.1002.qj.906).
Shutts G. 2005. A kinetic energy backscatter algorithm for
use in ensemble prediction systems. Q. J. R. Meteorol.
Soc. 131: 3079–3102.
Stensrud DJ, Bao J, Warner TT. 2000. Using initial
condition and model physics perturbations in short-range
ensemble simulations of mesoscale convective systems.
Mon. Wea. Rev. 128: 2077–2107.
Talagrand O, Vautard R, Strauss B. 1999. Evaluation
of probabilistic prediction systems. Proc. Workshop on
Predictability, Reading, United Kingdom, ECMWF : pp.
464.
Toth Z, Kalnay E. 1993. Ensemble forecasting at NMC: The
generation of perturbations. Bull. Amer. Meteor. Soc 74:
2317–2330.
Tribbia JJ, Baumhefner DP. 2004. Scale interactions and
atmospheric predictability. Mon. Wea. Rev. 132: 703–713.
Copyright c 2011 Royal Meteorological Society Q. J. R. Meteorol. Soc. 00: 2–13 (2011)
Prepared using qjrms4.cls14 M. Boisserie
a) NH b) SH
0.7 0.5
Z500 boreal winter Z500 boreal winter
Ws850 boreal winter Ws850 boreal winter
Z500 boreal summer 0.45 Z500 boreal summer
0.6 Ws850 boreal summer Ws850 boreal summer
0.4
0.5
0.35
0.4 0.3
0.25
0.3
0.2
0.2
0.15
0.1
0.1
0 0.05
0 24 48 72 96 120 144 168 192 216 240 0 24 48 72 96 120 144 168 192 216 240
forecast lead times (hours) forecast lead times (hours)
Figure 1. Time evolution of the average correlation between model errors and predictability errors absolute value over a) the Northern Hemisphere (NH)
and b) the Southern Hemisphere (SH). The threshold value under which model errors are considered to be uncorrelated with predictability errors is 0.14
(gray zone) at a 95% confidence level using a bootstrapping method.
a) Z500, forecast lead time= +6 hours d) Ws850, forecast lead time= +6 hours
0 10 20 30 40 50 70 90 120 160 0 1 2 3 4 5 6 8 10 12
b) Z500, forecast lead time= +5 days e) Ws850, forecast lead time= +5 days
0 10 20 30 40 50 70 90 120 160 0 1 2 3 4 5 6 8 10 12
c) Z500, forecast lead time= +10 days f) Ws850, forecast lead time= +10 days
0 10 20 30 40 50 70 90 120 160 0 1 2 3 4 5 6 8 10 12
Figure 2. Model error standard deviations for a wintertime period (January 16 to March 15 of 2010) for the 500 hPa geopotential height at a) 6-hour, b)
5-day, and c) 10-day lead times. The panels d) to f) are the same as panels a) to c) but for the 850 hPa wind speed. The units are in dam for Z500 and
m.s−1 for Ws850.
Copyright c 2011 Royal Meteorological Society Q. J. R. Meteorol. Soc. 00: 2–13 (2011)
Prepared using qjrms4.clsEstimating and diagnosing model error variances in the Météo-France global NWP model 15
a) Z500, NH b) Z500, SH
90 90
Pp DJF Pp DJF
Q DJF Q DJF
80 Pp JJA 80 Pp JJA
Q JJA Q JJA
70 70
60 60
std dev
std dev
50 50
40 40
30 30
20 20
10 10
0 0
0 2 4 6 8 10 0 2 4 6 8 10
Forecast lead time (days) Forecast lead time (days)
c) Ws850, NH d) Ws850, SH
5 5
4 4
std dev
std dev
3 3
2 2
1 1
0 0
0 2 4 6 8 10 0 2 4 6 8 10
Forecast lead time (days) Forecast lead time (days)
Figure 3. Time evolution of the average model error (black curve) and average predictability error (gray curve) standard deviations across a) the Northern
Hemisphere (NH) and b) the Southern Hemisphere (SH) for the 500-hPa geopotential height. Panels c) and d) are the same as panels a) and b) but for
the 850-hPa wind speed. The units are in m for the geopotential height and m.s−1 for the wind speed.
Copyright c 2011 Royal Meteorological Society Q. J. R. Meteorol. Soc. 00: 2–13 (2011)
Prepared using qjrms4.cls16 M. Boisserie
a) Spectral predictability error variances (PP )
−12
10
n−3
−13
10
4
−14
10 3
E, E∆ (m3.s−2)
2
1 n−5/3
−15 0.25
10
−16
10
−17
10
−18
10
0 1 2
10 10 10
wavenumber (n)
b) Spectral model error variances (Q)
−12
10
4 n−3
−13
10 3
2
−14
1
10
E, EΔ (m3.s−2)
0.25
n−5/3
−15
10
−16
10
−17
10
−18
10
0 1 2
10 10 10
wavenumber (n)
Figure 4. Time evolution of the variance spectra (E∆ ) of (a) predictability errors and (b) model errors for the rotational component of the kinetic energy
averaged over the studied winter period. The spectra are plotted at 6 hours (day 0.25) and every 1-day forecast (from day 1 to day 10). In both panels, the
two thick curves represents the spectral kinetic energy density of the ensemble mean forecast (black thick curve) and the ECMWF analysis (gray thick
curve) at day 5. The two lines indicate the n−3 and the n−5/3 two-dimensional turbulence power-law behavior. All the spectra are computed for 500
hPa.
Copyright c 2011 Royal Meteorological Society Q. J. R. Meteorol. Soc. 00: 2–13 (2011)
Prepared using qjrms4.clsEstimating and diagnosing model error variances in the Météo-France global NWP model 17
a) Z, NH b) Z, SH
200 200
250 250
300 300
Pressure (hPa)
Pressure (hPa)
500 0.25 500 0.25
700 700
850 6 850 6 6
925 925
1000 1000
0 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90
std dev (m/s) std dev (m/s)
c) Ws, NH d) Ws, SH
200 200
250 0.25 0.25 6 250 0.25 0.25 6
300 300
Pressure (hPa)
Pressure (hPa)
500 500 6
700 700
850 850
925 925
1000 1000
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9
std dev (m/s) std dev (m/s)
Figure 5. Same as Figure 3. but for the vertical profile at day 0.25 and day 6.
Copyright c 2011 Royal Meteorological Society Q. J. R. Meteorol. Soc. 00: 2–13 (2011)
Prepared using qjrms4.clsYou can also read
