More on the Bates revelations about the NOAA paper by Tom Karl
The writer below says that the Bates revelations have one and one only important implication: That unvalidated data was used. In my career as a psychometrician, I too often railed against the unvalidated data often used by my fellow social scientists. So I agree that use of unvalidated data means that the conclusions of that particular study cannot be accepted.
I don't think the problem ends there, however. I think it unlikely that the data used CAN be validated. The revelation about the best measurements of sea surface temperature not being used do, I think, have that implication. They imply that the data body used was constructed to defraud
It is sometimes said science is all about data… observation, measurement, experiment, measurement… But that is NOT the whole story. To ensure data is reliable and understood, we’ve developed standard units of measure, and document procedures used to obtain and record measurements. The intention is to make sure BOTH the data AND collection methods can be reliably understood and used by others. The fleshed out version of this is the scientific method, and is integral to, and indispensable in the advance of science. It works because it helps eliminate bias and protect the integrity of both data and process. Any departure from rigorous adherence to these principles may or may not adversely affect data. But it increases the risk, and introduces doubt as to the overall integrity. And any subsequent reliance on this data must not assert confidence levels beyond the weakest preceding link. For example, it would be inaccurate or dishonest to claim 100% certainty on results that can only be replicated 50% of the time.
So let’s wind forward…
There has been much suck-and-blow blather in the aftermath of the David Rose column on the whistleblower allegations by former NOAA scientist John Bates. I won’t rehash the article, other than to say Rose does seem eager to sensationalize speculative results rather than the details, but that in no way negates the seriousness of the allegations stated. What I want to discuss is the allegations and impacts. Rose is not the story. Bates is not the story. The story is the circumvention of procedures put in place to protect the integrity of the data, and hence the reputation of the NOAA. From John Bates:
Predictably, both the “consensus” and skeptic camps largely missed the mark in jumping to defend or attack positions. There were a flurry of hastily written newspaper and blog reports on “bad data“, “data manipulation“, and “data tampering“. Bates’ report didn’t say data was deliberately compromised (he mentioned a “thumb on the scale” which he later seemed to walk back), but that the presentation may have been biased, and adherence to protocol was haphazard. These of course are different things. This opened the door for the usual suspects from the other side to rush out reports showing the NOAA data was largely in agreement with other datasets, directing the discussion away from the presentation and protocol questions to “The data checks out. See? No problem.” This was cleverly, cynically, and all too accurately highlighted by Gavin Schmidt:
Let there be NO mistake: Regardless of the best efforts of Schmidt and friends to paint this as just deniers denying, if NOAA followed THEIR OWN established protocols, there would be no story.
Now the hordes of hyperactive and secure-in-their-ignorance columnists, tweeters and bloggers from the periphery join in with escalations of character attacks, dishonest misdirections, and deliberately uncharitable interpretations of innocuous statements. The Guardian chipped in with a nastily biased bit:
Referring back to the Science Insider piece…
Just one little problem: They provide no evidence that Bates said anything about being wary of skeptics. He said “people”. And as both skeptic and consensus camps have seemingly derailed in their rush to the wrong conclusion, it could easily mean either, or more likely both.
I could go on at length about the ridiculous obfuscation and mean spirited BS thrown about during any attempted discussion of the allegations (most of which have not been denied, but rather downplayed) but I’ll save that for a separate post. That’s just another distraction from the real issue at hand.
No, the issues are as Bates outlined: “Ethical standards must be maintained”. There can be no confidence in data without confidence in the procedures surrounding collection and storage of data. And persons or organizations that place no value in these procedures further erode confidence. This happens repeatedly:
- publicly funded trustee of information gets “sloppy”
- concern is expressed
- those at fault are defended
- the ‘concerned’ are attacked
- conversation derails
- nothing is fixed
- rinse and repeat
Perhaps in this case no data was harmed. I hope not. But if we don’t take these matters seriously eventually there will be damages. And not just to a database.