Thursday, 26 November 2009

Lies, Damn Lies, and Statistics.

I've refrained from commenting on the leaked/hacked emails (etc) from the Hadley Climate Unit in East Anglia. At the very least, I wanted to ensure that it was not an elaborate hoax; on that score it seems that I need not worry.

Let me make on thing clear from the start; I don't think that the extracts which I have seen show any evidence of deliberate distortion or manipulation of the data*. Nor, in fact, do I think that the data has actually been deliberately distorted after collection; whilst that does happen in science from time to time, it is quite rare and is usually associated with scientists working alone or in very small groups.

You will note my careful qualification of the nature of the distortion, though. What the emails do show me is that the researchers working at the CRU (I am avoiding use of the term "scientists") had a very firm view on the subject of climate change, and saw their task as being to prove that climate change/global warming was real and likely to cause us serious harm. In that respect, they were evangelical in nature.

That does not make them bad people. They honestly and sincerely believe that the world faces a serious threat and they want to avert that. Their intentions are good and their motives are proper.

What is does make them, though, is advocates; not scientists. The two are incompatible. A scientist looks for the truth in nature, and takes the data as s/he finds it. Looking at the data, a possible truth may strike the scientist, at which point s/he will construct an experiment to test that. The purpose of the experiment is to construct a set of circumstances in which the outcome will be markedly different according to whether the scientist's idea is true or not.

To do this properly takes huge care. Bias in scientific work is extremely difficult to exclude, but it is of the utmost importance that it is. It can only be achieved if the scientist is either utterly disinterested, or wholly aware of the need to exclude bias and determined to do so.

The Hadley centre is clearly, unequivocally, and evidently neither of these. There was no need for them to distort the data; the means by which it was collected will have provided ample distortion.

--------------------------------------------------------------
*although I obviously cannot rule that out.

11 comments:

  1. I think one of the interesting aspects of this is what effect the whole climate debate will have on the popular perception of science.

    There has been so much invested in the rhetoric of "all serious scientists are agreed," what happens in the future if it turns out this alleged "scientific consensus" got it wrong? If that happens after green policies have resulted in costly damage to Western economies and restrictions on the development of poorer countries, it will not look good. This could be science's Galileo.

    But perhaps some of the email stuff will result in more scepticism at Copenhagen as politicians worry if it is all real, or if they can get painful policies passed their voters. If so, this email hacking may have done science (and the rest of us) a favour. If on the other hand, the "climate consensus" has got it right, a few poorly chosen words in some private emails may be used in the future to illustrate the butterfly-effect.

    ReplyDelete
  2. What is does make them, though, is advocates; not scientists. The two are incompatible.

    Actually, I suspect that a lot of people who are generally seen as scientists, including many of the great scientists of the past, are/were advocates as well as scientists.

    So I'm not sure that they are incompatible.

    (And for what it's worth, I have a degree in biology.)

    ReplyDelete
  3. YMB - Hmm. A scientist can validly act as an advocate for the results of his/her experiment. In that case, the advocacy follows the science; I think that is the type of advocacy you are discussing. Such a scientist would need to take great care in designing the next experiment to ensure that their opinion did not bias the experiment.

    What they cannot do is to start as an advocate of a specific view, design experiments accordingly, and call that science. That is what I perceive at the CRU, together with an inability to step back before designing the next experiment.

    (FWIW, my degree is Natural Sciences, including History & Philosophy of Science!)

    Albert, I have been worried for a long time about the impact of this affair in the public perception of science. We have long been told that "the science is settled" and "the debate is over", statements that are as unscientific as they are factually inaccurate. Those alone have already (in my opinion) communicated a misperception of the nature of science. The latest discovery could now do great harm to its reputation.

    ReplyDelete
  4. So what does that make the people who believe in subatomic particles and to prove themselves right have built a huge tunnel in Switzerland? What is the difference between trying to prove your hypothesis and advocating your hypothesis?

    ReplyDelete
  5. Perhaps the real problem with all this is not that scientists have been biased or "guilty" of advocacy - for which no one can be blamed. Perhaps the real problem is that the "peer review" checks appear to have been hijacked to make the statement "all serious scientists think X" more or less tautological.

    As a non-scientist, I don't take much notice of what an individual scientist tells me, it is the judgment of the wider scientific community in a particular field that counts. Both the emails and the increasing grumbles of unbelieving scientists (who complain that the consensus is made up not of proper climate scientists, but of computer modellers etc.) have made it hard for the scientific layman to know where the real scientific authority lies.

    ReplyDelete
  6. If I may rephrase my point, what I am saying is that scientists are human and hence fallible. They have prejudices, and preconceived notions. Some of them are better at keeping those prejudices out of their work than others, and some work in fields where those prejudices are not going to make any difference. That does not mean that those who allow their prejudices and preconceived notions to intrude into their work are not scientists. What it means is that they are flawed scientists, who may still contribute to the advancement of knowledge.

    To "start as an advocate of a specific view, design experiments accordingly" may not be science, but those who do these that sort of thing may still be termed "scientists".

    BTW, I heartily agree that to say "the science is settled" is completely unscientific. This needs to be said more often.

    ReplyDelete
  7. I heartily agree that to say "the science is settled" is completely unscientific.

    Perhaps someone can help me here. Surely there are things that are settled, the earth isn't the centre of the universe, smoking is likely to damage our health, complex species have evolved from simpler ones etc.

    ReplyDelete
  8. Blue - that one is easy to deal with. The distinction lies whether the experiment is designed to answer a question or prove a point. CERN believe the Higg's boson exists, so have built an experiment to find it. The evidence yielded by that experiment could point either way.

    But they do not yell at "Higgs deniers", prevent them from being published, announce that the Higgs boson must exist because the consensus is there and the debate is over, and use evidence pointing in any direction whatsoever to support the existence of the Higgs particle.

    If there were to be ten years of declining temperatures... (ooops, no, sorry, I mean an exhaustive search with no sign of the Higgs) then they would start thinking again. Physics has had plenty of dead ends and new directions.

    YMB - yes, I am more inclined to agree with you expressed in that manner. But such scientists are (objectively speaking) flawed, and if a field is dominated by flawed scientists who shout down and suppress dissent from less flawed scientists, then the field as a whole becomes insufficiently scientific to be described as such. I think that is my point!

    This needs to be said more often.

    Thanks - I keep yelling it (and have done for as long as I can remember!) but people seem resistant to the idea that science is a questioning art. Which brings me to:

    Albert - No, none of those subjects are (in truth) settled. Dealing with them in turn;

    - the earth may well be at the centre of the universe. The problem is that you would need to define the centre in order to answer that. Given that we do not know whether the universe is bounded or infinite, or what form its bounds might take, then on the current state of our knowledge we conclude that the question has no meaning. But we cannot say that we will never be able to determine a centre, or that the earth will not be located there. Indeed, in some frames of reference the universe seems to extend equally i.e. infinitely!) in all directions, which would put us at a centre, if not the centre!

    - On current evidence, smoking does harm your health. Can you rule out, though, the existence of any medical condition in which the stimulative effects of smoking would have a positive effect that was not outweighed by the collateral harm? Or, what if you were presented with a patient who was terminally ill with a non-smoking-related condition, but whose depression might be lifted by the pleasure obtainned from smoking? Would you assert that smoking was objectively bad for their health?

    - Equally, the theory of evolution fits the observed evidence exceptionally well - far better than any alternative. No other viable suggestions are on the table at present. However, the same was once true of Newtonian mechanics, phlogiston theory, and the early forms of the Copenhagen Interpretation.

    The point is that none of these (or any other) theories are necessarily right. If counter-evidence is found for any of them, then the theory is reconsidered. That does not happen with climate change, though; a declaration is issued that "the science is settled" and the debate is ended.

    ReplyDelete
  9. Personally I believe that there is a case for a police investigation at the University of East Anglia, not only into the hacking, but also into other possible wrongdoing.
    As has been mentioned elsewhere, there seem to be at least two possible offences worthy of examination.
    1. The misappropriation of public funds - It is alleged that data was falsified and that such data was subsequently used to justify further public funding.
    2. Failure to disclose information under the FOI - it is alleged that there was a deliberate attempt to prevent disclosure of information and urging others to destroy it (also incitement, presumably).
    This needs a police investigation, not a cosy internal enquiry.

    ReplyDelete
  10. P, thanks, so if I understand you, the issue is not so much that the climate change scientists think there is good or convincing evidence to support their conclusion, it is that they are closed to any evidence to the contrary. Right? or wrong?!

    ReplyDelete
  11. This needs a police investigation, not a cosy internal enquiry.

    Absolutely... the hacking is an offence (and should be investigated), but so are both the aspects you point out.

    Right? or wrong?!

    Getting there. That is one criticism of them.

    More serious, though, is the point that if you design an experiment to produce data that supports a specific predetermined answer, then it will (often) oblige. So the only way of producing reliable data is for the experimenter to have the correct mindset in advance. The CRU staff seem to have failed in that respect.

    That, sadly, means that not only are we unable to trust their statements and their arguments, we must also suspect their data.

    ReplyDelete