OptIC project: AN INTERCOMPARISION OF OPTIMITAZION TECHNIQUES FOR PARAMETER ESTIMATION IN TERRESTRIAL BIOGEOCHEMICAL MODELS
Cathy M. Trudinger Marine and Atmospheric Research, CSIRO, Aspendale, Victoria, Australia
Michael R. Raupach Marine and Atmospheric Research, CSIRO, Canberra, ACT, Australia
Peter J. Rayner Laboratoire CEA/CNRS/UVSQ, LSCE/IPSL, Paris, France
Jens Kattge Max Planck Institute for Biogeochemistry, Jena, Germany
Qing Liu School of Earth and Atmospheric Sciences, Georgia Institute of Technology, Atlanta, Georgia, USA
Bernard Pak Marine and Atmospheric Research, CSIRO, Aspendale, Victoria, Australia
Markus Reichstein Max Planck Institute for Biogeochemistry, Jena, Germany
Luigi Renzullo Land and Water, CSIRO, Canberra, ACT, Australia
Andrew D. Richardson Complex Systems Research Center, University of New Hampshire, Durham, New Hampshire, USA
Stephen H. Roxburgh School of Biological, Earth and Environmental Sciences and Bushfire Cooperative Research Center, University of New South Wales, Sydney, New South Wales, Australia
Julie Styles Oregon State University, Corvallis, Oregon, USA
Ying Ping Wang Marine and Atmospheric Research, CSIRO, Aspendale, Victoria, Australia
Peter Briggs Marine and Atmospheric Research, CSIRO, Canberra, ACT, Australia
Damian Barrett Land and Water, CSIRO, Canberra, ACT, Australia
Sonja Nikolova Marine and Atmospheric Research, CSIRO, Canberra, ACT, Australia
We describe results of a project known as OptIC (Optimisation InterComparison) for comparison of parameter estimation methods in terrestrial biogeochemical models. A highly simplified test model was used to generate pseudo-data to which noise with different characteristics was added. Participants in the OptIC project were asked to estimate the model parameters used to generate this data, and to predict model variables into the future. Ten participants contributed results using one of the following methods: Levenberg-Marquardt, adjoint, Kalman filter, Markov chain Monte Carlo and genetic algorithm. Methods differed in how they locate the minimum (gradient-descent or global search), how observations are processed (all at once sequentially), or the number of iterations used, or assumptions about the statistics (some methods assume Gaussian probability density functions; others do not). We found the different methods equally successful at estimating the parameters in our application. The biggest variation in parameter estimates arose from the choice of cost function, not the choice of optimization method. Relatively poor results were obtained when the model-data mismatch in the cost function included weights that were instantaneously dependent on noisy observations. This was the case even when the magnitude of residuals varied with the magnitude of observations. Missing data caused estimates to be more scattered, and the uncertainty of predictions increased correspondingly. All methods gave biased results when the noise was temporally correlated or non-Gaussian, or when incorrect model forcing was used. Our results highlight the need for care in choosing the error model in any optimization.
Received 10 November 2006; accepted 20 March 2007; published 1 June 2007.
Citation: Trudinger, C. M., et al. (2007), OptIC project: An intercomparison of optimization techniques for parameter estimation in terrestrial biogeochemical models, J. Geophys. Res., 112, G02027, doi:10.1029/2006JG000367.
