Daniel, you say: "At the minimum I would support the open and free release of all unaltered data."
If i may extract quote from the message you replied to: As you know, I have refused to send McIntyre the "derived" model data he requests, since all of the primary model data necessary to replicate our results are freely available to him. I will continue to refuse such data requests in the future. Nor will I provide McIntyre with computer programs, email correspondence, etc. I feel very strongly about these issues. We should not be coerced by the scientific equivalent of a playground bully. I will be consulting LLNL's Legal Affairs Office in order to determine how the DOE and LLNL should respond to any FOI requests that we receive from McIntyre." (emphasis added by me)
I do agree with you that transparency is the litmus test of good science. At the same time, catering to that can lay you open to abuses from, well, trolls.
As an analogy, suppose I had a bee in my bonnet about Linux security and I kept demanding that Linus send me copies of his binaries every time he did a new commit. He would rightly tell me to get the sources and build it myself and leave him alone. and in turn I could post lonely rants along the lines of 'why won't Linus Torvalds come clean about Linux security'. It would be quite meaningless, but it would sucker in some people. If I was sufficiently clever I could probably get quoted by someone at Microsoft or in the BSD camp.
The parallel I am drawing here is to the public datasets and methods laid out in published, peer-reviewed papers. If McIntyre is so sure that either the methods or the datasets are flawed, why not follow the established practice of writing to the journal and challenging the paper, or even submitting his own analysis of the public data and explaining how it improves on existing analyses? Instead he posts in some minor journal which is not part of the scientific corpus, not a hard science journal, and has been widely accused of lax publication criteria.
One of the things that really bothers me about the skeptical crowd (as opposed to individual skeptics) is that they employ a lot of same kind of arguments as the proponents of 'intelligent design' do, claiming there's an ivory-tower conspiracy that silences all dissenting views and shuts them out of publication. Any time you have a bunch of people going 'OMG teh conspiracy', it's time to whip out the old bullshit detector...and all too often, comparing their claims with the published literature sends the BS detector way into the red.
Nor will I provide McIntyre with computer programs,
Sad fact: most journal articles do not include all the details necessary to reproduce their results. There are usually all sorts of heuristics and tricks used which are not interesting enough to include in the paper. Typically, the experts in the field all know each other and share source code/tricks in private conversation.
As for the "ivory-tower conspiracy", it apparently isn't bullshit. As these emails reveal, there is a conspiracy to make it difficult for Steve McIntyre to examine published scientific results. There is also a conspiracy (revealed in other emails) to oust climate skeptics from scientific societies. It isn't paranoia if they are really out to get you.
why not follow the established practice of writing to the journal and challenging the paper, or even submitting his own analysis of the public data and explaining how it improves on existing analyses?
It looks like, from the emails, (and I hate to prejudge) that "submitting his own analysis" is not going to be an option for him.
Which leads me to the discriminating factor -- if publicly available data and publicly available processes allow an independent to process the data and reach different conclusions and be published that's fine. But that's not what is happening here.
There is no mystery about what happens between Linus' source code and the compiled executable. It's imminently transparent and reproducible. Science should be exactly the same way.
I also agree with having alarm bells ring whenever you hear "OMG the conspiracy!". I would humbly add, however, that groupthink is a real and prevalent problem -- not just among academics but everybody. If your groupthink reaches a point where you view that it's "us against them" and that your job in academia is to defend a pre-established position? I'm thinking you should be fired and never work in that field again. It's critical among scientists more than anybody else to have an open mind and a large dose of humility. The entire idea of science is that what you're working on will be replaced or fine-tuned later on down the road. Everybody is going to be wrong to some degree. It's part of the job. Big egos are not such a great thing.
I don't have a lot of tolerance for taking public money and playing politics with it, even if you are proven correct in the long run. One man's troll is another man's skeptic, and it's your job as an honest scientist to deal with them. Good scientists should first and foremost always be open about data and methods. Good grief, it's 2009, publishing all of this on a wiki somewhere is trivial.
"Why not ... [submit] his own analysis of the public data ... explaining how it improves on existing analyses?"
This is the meat of the matter. Scientists write letters to journals all the time saying that they think an author overstretched. Those letters, in and of themselves, do not negate the previous research. Scientists also write original papers showing that a previous hypothesis is invalid. Those papers, if they are well written, are easy to publish, especially if they make the journal more interesting.
If i may extract quote from the message you replied to: As you know, I have refused to send McIntyre the "derived" model data he requests, since all of the primary model data necessary to replicate our results are freely available to him. I will continue to refuse such data requests in the future. Nor will I provide McIntyre with computer programs, email correspondence, etc. I feel very strongly about these issues. We should not be coerced by the scientific equivalent of a playground bully. I will be consulting LLNL's Legal Affairs Office in order to determine how the DOE and LLNL should respond to any FOI requests that we receive from McIntyre." (emphasis added by me)
I do agree with you that transparency is the litmus test of good science. At the same time, catering to that can lay you open to abuses from, well, trolls.
As an analogy, suppose I had a bee in my bonnet about Linux security and I kept demanding that Linus send me copies of his binaries every time he did a new commit. He would rightly tell me to get the sources and build it myself and leave him alone. and in turn I could post lonely rants along the lines of 'why won't Linus Torvalds come clean about Linux security'. It would be quite meaningless, but it would sucker in some people. If I was sufficiently clever I could probably get quoted by someone at Microsoft or in the BSD camp.
The parallel I am drawing here is to the public datasets and methods laid out in published, peer-reviewed papers. If McIntyre is so sure that either the methods or the datasets are flawed, why not follow the established practice of writing to the journal and challenging the paper, or even submitting his own analysis of the public data and explaining how it improves on existing analyses? Instead he posts in some minor journal which is not part of the scientific corpus, not a hard science journal, and has been widely accused of lax publication criteria.
One of the things that really bothers me about the skeptical crowd (as opposed to individual skeptics) is that they employ a lot of same kind of arguments as the proponents of 'intelligent design' do, claiming there's an ivory-tower conspiracy that silences all dissenting views and shuts them out of publication. Any time you have a bunch of people going 'OMG teh conspiracy', it's time to whip out the old bullshit detector...and all too often, comparing their claims with the published literature sends the BS detector way into the red.