A simple model was found to be produce forecasts that are over seven times more accurate than forecasts from the procedures used by the United Nations Intergovernmental Panel on Climate Change (IPCC). This important finding, is reported in an article titled "Validity of climate change forecasting for public policy decision making" (PDF) in the latest issue of the International Journal of Forecasting. It is the result of a collaboration among forecasters J. Scott Armstrong of the Wharton School, Kesten C. Green of Monash University, and climate scientist Willie Soon of the Harvard-Smithsonian Center for Astrophysics.
In an earlier paper (PDF), Armstrong and Green found that the IPCC's approach to forecasting climate violated 72 principles of forecasting. To put this in context, would you put your children on a trans-Atlantic flight if you knew that the plane had failed engineering checks for 72 out of 127 relevant items on the checklist?
The IPCC violations of forecasting principles were partly due to their use of models that were too complex for the situation. Contrary to everyday thinking, complex models provide forecasts that are less accurate than forecasts from simple models when the situation is complex and uncertain.
Confident that a forecasting model that followed scientific forecasting principles would provide forecasts that were more accurate than those provided by the IPCC, Green, Armstrong and Soon used a model that was more consistent with forecasting principles and knowledge about climate. The forecasting model was the so-called "naive" model. It assumes that things will remain the same. It is such a simple model that people are generally not aware of its power. In contrast to the IPCC's central forecast that global mean temperatures will rise by 3 degrees C over a century, the naive model simply forecasts that temperatures next year and for each of 100 years into the future would remain the same as the last years'.
The naive model approach is confusing to non-forecasters who are aware that temperatures have always varied. Moreover, much has been made of the observation that the temperature series that the IPCC uses shows a broadly upward trend since 1850 and that this is coincident with increasing industrialization and associated increases in manmade carbon dioxide gas emissions.
In order to test the naive model, annual forecasts were made from one to 100 years in the future starting with 1850's global average temperature as the forecast for the years 1851 to 1950. This process was repeated by updating for each year up through 2007. This produced 10,750 annual average temperature forecasts for all horizons. It was the first time that the IPCC's forecasting procedures had been subject to a large-scale test of the accuracy of the forecasts that they produce.
Over all the forecasts, the IPCC error was 7.7 times larger than the error from the naive model. While the superiority of the naive model was modest for one- to ten-year-ahead forecasts (where the IPCC error was 1.5 times larger), its superiority was enormous for the 91- to 100-year-ahead forecasts, where the IPCC error was 12.6 times larger.
Is it proper to conduct validation tests? In many cases, such as the climate change situation, people claim that: "Things have changed! We cannot use the past to forecast." While they may think that their situation is unique, there is no logic to this argument. The only way to forecast the future is by learning from the past. In fact, the warmers claims are also based on their analyses of the past.
Could one improve upon the naive model? The naive model violates some principles. For example, it violates the principle that one should use as long a time series as possible, because it bases all forecasts on simply the global average temperature for the single year just prior to making the forecasts. It also fails to combine forecasts from different reasonable methods. The authors planned to start simple with this self-funded project and to then obtain funding to undertake a more ambitious forecasting effort to ensure that all principles were followed. This would no doubt improve accuracy. However, the forecasts from the naive model were very accuurate. For example, the mean absolute error for the 108 fifty-year ahead forecasts was only 0.24 degrees C. It is difficult to see any economic value to reducing such a small forecast error.
Above article received by email. Author: Scott Armstrong [firstname.lastname@example.org]
Posted by John Ray (M.A.; Ph.D.). For a daily critique of Leftist activities, see DISSECTING LEFTISM. To keep up with attacks on free speech see TONGUE-TIED. Also, don't forget your daily roundup of pro-environment but anti-Greenie news and commentary at GREENIE WATCH . Email me here