Estimation and calibration of observation impact signals using the Lanczos method in NOAA/NCEP data assimilation system

M. Wei, M. S F V. De Pondeca, Z. Tóth, D. Parrish

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

Despite the tremendous progress that has been made in data assimilation (DA) methodology, observing systems that reduce observation errors, and model improvements that reduce background errors, the analyses produced by the best available DA systems are still different from the truth. Analysis error and error covariance are important since they describe the accuracy of the analyses, and are directly related to the future forecast errors, i.e., the forecast quality. In addition, analysis error covariance is critically important in building an efficient ensemble forecast system (EFS). Estimating analysis error covariance in an ensemble-based Kalman filter DA is straightforward, but it is challenging in variational DA systems, which have been in operation at most NWP (Numerical Weather Prediction) centers. In this study, we use the Lanczos method in the NCEP (the National Centers for Environmental Prediction) Gridpoint Statistical Interpolation (GSI) DA system to look into other important aspects and properties of this method that were not exploited before. We apply this method to estimate the observation impact signals (OIS), which are directly related to the analysis error variances. It is found that the smallest eigenvalue of the transformed Hessian matrix converges to one as the number of minimization iterations increases. When more observations are assimilated, the convergence becomes slower and more eigenvectors are needed to retrieve the observation impacts. It is also found that the OIS over data-rich regions can be represented by the eigenvectors with dominant eigenvalues. Since only a limited number of eigenvectors can be computed due to computational expense, the OIS is severely underestimated, and the analysis error variance is consequently overestimated. It is found that the mean OIS values for temperature and wind components at typical model levels are increased by about 1.5 times when the number of eigenvectors is doubled. We have proposed four different calibration schemes to compensate for the missing trailing eigenvectors. Results show that the method with calibration for a small number of eigenvectors cannot pick up the observation impacts over the regions with fewer observations as well as a benchmark with a large number of eigenvectors, but proper calibrations do enhance and improve the impact signals over regions with more data. When compared with the observation locations, the method generally captures the OIS over regions with more observation data, including satellite data over the southern oceans. Over the tropics, some observation impacts may be missed due to the smaller background errors specified in the GSI, which is not related to the method. It is found that a large number of eigenvectors are needed to retrieve impact signals that resemble the banded structures from satellite observations, particularly over the tropics. Another benefit from the Lanczos method is that the dominant eigenvectors can be used in preconditioning the conjugate gradient algorithm in the GSI to speed up the convergence.

Original languageEnglish
Pages (from-to)541-557
Number of pages17
JournalNonlinear Processes in Geophysics
Volume19
Issue number5
DOIs
Publication statusPublished - 2012

Fingerprint

assimilation
Eigenvalues and eigenfunctions
data assimilation
error analysis
Calibration
calibration
eigenvectors
Error analysis
prediction
predictions
interpolation
eigenvalue
Tropics
Interpolation
forecasting
capture method
Satellites
tropical regions
Kalman filter
method

ASJC Scopus subject areas

  • Geochemistry and Petrology
  • Geophysics
  • Statistical and Nonlinear Physics

Cite this

Estimation and calibration of observation impact signals using the Lanczos method in NOAA/NCEP data assimilation system. / Wei, M.; V. De Pondeca, M. S F; Tóth, Z.; Parrish, D.

In: Nonlinear Processes in Geophysics, Vol. 19, No. 5, 2012, p. 541-557.

Research output: Contribution to journalArticle

@article{db6644aa174e4e1cbd1e664d875f9057,
title = "Estimation and calibration of observation impact signals using the Lanczos method in NOAA/NCEP data assimilation system",
abstract = "Despite the tremendous progress that has been made in data assimilation (DA) methodology, observing systems that reduce observation errors, and model improvements that reduce background errors, the analyses produced by the best available DA systems are still different from the truth. Analysis error and error covariance are important since they describe the accuracy of the analyses, and are directly related to the future forecast errors, i.e., the forecast quality. In addition, analysis error covariance is critically important in building an efficient ensemble forecast system (EFS). Estimating analysis error covariance in an ensemble-based Kalman filter DA is straightforward, but it is challenging in variational DA systems, which have been in operation at most NWP (Numerical Weather Prediction) centers. In this study, we use the Lanczos method in the NCEP (the National Centers for Environmental Prediction) Gridpoint Statistical Interpolation (GSI) DA system to look into other important aspects and properties of this method that were not exploited before. We apply this method to estimate the observation impact signals (OIS), which are directly related to the analysis error variances. It is found that the smallest eigenvalue of the transformed Hessian matrix converges to one as the number of minimization iterations increases. When more observations are assimilated, the convergence becomes slower and more eigenvectors are needed to retrieve the observation impacts. It is also found that the OIS over data-rich regions can be represented by the eigenvectors with dominant eigenvalues. Since only a limited number of eigenvectors can be computed due to computational expense, the OIS is severely underestimated, and the analysis error variance is consequently overestimated. It is found that the mean OIS values for temperature and wind components at typical model levels are increased by about 1.5 times when the number of eigenvectors is doubled. We have proposed four different calibration schemes to compensate for the missing trailing eigenvectors. Results show that the method with calibration for a small number of eigenvectors cannot pick up the observation impacts over the regions with fewer observations as well as a benchmark with a large number of eigenvectors, but proper calibrations do enhance and improve the impact signals over regions with more data. When compared with the observation locations, the method generally captures the OIS over regions with more observation data, including satellite data over the southern oceans. Over the tropics, some observation impacts may be missed due to the smaller background errors specified in the GSI, which is not related to the method. It is found that a large number of eigenvectors are needed to retrieve impact signals that resemble the banded structures from satellite observations, particularly over the tropics. Another benefit from the Lanczos method is that the dominant eigenvectors can be used in preconditioning the conjugate gradient algorithm in the GSI to speed up the convergence.",
author = "M. Wei and {V. De Pondeca}, {M. S F} and Z. T{\'o}th and D. Parrish",
year = "2012",
doi = "10.5194/npg-19-541-2012",
language = "English",
volume = "19",
pages = "541--557",
journal = "Nonlinear Processes in Geophysics",
issn = "1023-5809",
publisher = "European Geosciences Union",
number = "5",

}

TY - JOUR

T1 - Estimation and calibration of observation impact signals using the Lanczos method in NOAA/NCEP data assimilation system

AU - Wei, M.

AU - V. De Pondeca, M. S F

AU - Tóth, Z.

AU - Parrish, D.

PY - 2012

Y1 - 2012

N2 - Despite the tremendous progress that has been made in data assimilation (DA) methodology, observing systems that reduce observation errors, and model improvements that reduce background errors, the analyses produced by the best available DA systems are still different from the truth. Analysis error and error covariance are important since they describe the accuracy of the analyses, and are directly related to the future forecast errors, i.e., the forecast quality. In addition, analysis error covariance is critically important in building an efficient ensemble forecast system (EFS). Estimating analysis error covariance in an ensemble-based Kalman filter DA is straightforward, but it is challenging in variational DA systems, which have been in operation at most NWP (Numerical Weather Prediction) centers. In this study, we use the Lanczos method in the NCEP (the National Centers for Environmental Prediction) Gridpoint Statistical Interpolation (GSI) DA system to look into other important aspects and properties of this method that were not exploited before. We apply this method to estimate the observation impact signals (OIS), which are directly related to the analysis error variances. It is found that the smallest eigenvalue of the transformed Hessian matrix converges to one as the number of minimization iterations increases. When more observations are assimilated, the convergence becomes slower and more eigenvectors are needed to retrieve the observation impacts. It is also found that the OIS over data-rich regions can be represented by the eigenvectors with dominant eigenvalues. Since only a limited number of eigenvectors can be computed due to computational expense, the OIS is severely underestimated, and the analysis error variance is consequently overestimated. It is found that the mean OIS values for temperature and wind components at typical model levels are increased by about 1.5 times when the number of eigenvectors is doubled. We have proposed four different calibration schemes to compensate for the missing trailing eigenvectors. Results show that the method with calibration for a small number of eigenvectors cannot pick up the observation impacts over the regions with fewer observations as well as a benchmark with a large number of eigenvectors, but proper calibrations do enhance and improve the impact signals over regions with more data. When compared with the observation locations, the method generally captures the OIS over regions with more observation data, including satellite data over the southern oceans. Over the tropics, some observation impacts may be missed due to the smaller background errors specified in the GSI, which is not related to the method. It is found that a large number of eigenvectors are needed to retrieve impact signals that resemble the banded structures from satellite observations, particularly over the tropics. Another benefit from the Lanczos method is that the dominant eigenvectors can be used in preconditioning the conjugate gradient algorithm in the GSI to speed up the convergence.

AB - Despite the tremendous progress that has been made in data assimilation (DA) methodology, observing systems that reduce observation errors, and model improvements that reduce background errors, the analyses produced by the best available DA systems are still different from the truth. Analysis error and error covariance are important since they describe the accuracy of the analyses, and are directly related to the future forecast errors, i.e., the forecast quality. In addition, analysis error covariance is critically important in building an efficient ensemble forecast system (EFS). Estimating analysis error covariance in an ensemble-based Kalman filter DA is straightforward, but it is challenging in variational DA systems, which have been in operation at most NWP (Numerical Weather Prediction) centers. In this study, we use the Lanczos method in the NCEP (the National Centers for Environmental Prediction) Gridpoint Statistical Interpolation (GSI) DA system to look into other important aspects and properties of this method that were not exploited before. We apply this method to estimate the observation impact signals (OIS), which are directly related to the analysis error variances. It is found that the smallest eigenvalue of the transformed Hessian matrix converges to one as the number of minimization iterations increases. When more observations are assimilated, the convergence becomes slower and more eigenvectors are needed to retrieve the observation impacts. It is also found that the OIS over data-rich regions can be represented by the eigenvectors with dominant eigenvalues. Since only a limited number of eigenvectors can be computed due to computational expense, the OIS is severely underestimated, and the analysis error variance is consequently overestimated. It is found that the mean OIS values for temperature and wind components at typical model levels are increased by about 1.5 times when the number of eigenvectors is doubled. We have proposed four different calibration schemes to compensate for the missing trailing eigenvectors. Results show that the method with calibration for a small number of eigenvectors cannot pick up the observation impacts over the regions with fewer observations as well as a benchmark with a large number of eigenvectors, but proper calibrations do enhance and improve the impact signals over regions with more data. When compared with the observation locations, the method generally captures the OIS over regions with more observation data, including satellite data over the southern oceans. Over the tropics, some observation impacts may be missed due to the smaller background errors specified in the GSI, which is not related to the method. It is found that a large number of eigenvectors are needed to retrieve impact signals that resemble the banded structures from satellite observations, particularly over the tropics. Another benefit from the Lanczos method is that the dominant eigenvectors can be used in preconditioning the conjugate gradient algorithm in the GSI to speed up the convergence.

UR - http://www.scopus.com/inward/record.url?scp=84880560443&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84880560443&partnerID=8YFLogxK

U2 - 10.5194/npg-19-541-2012

DO - 10.5194/npg-19-541-2012

M3 - Article

AN - SCOPUS:84880560443

VL - 19

SP - 541

EP - 557

JO - Nonlinear Processes in Geophysics

JF - Nonlinear Processes in Geophysics

SN - 1023-5809

IS - 5

ER -