A message from the recovery room

Eli's immediate post below as reference. I haven't written anything yet because I've been meaning to first set aside several hours to recover where everything ended up. I think where my hubris messed things up was in that wasn't just those skepticalish abstracts I wanted to understand, but an attempt to get all discrepancies in any category resolved by forgetting the abstracts when reviewers disagreed and reading the source papers instead. That bridged too far for us volunteers and ended it.

Still could be a good thing just to read the papers for the skeptical abstracts. I think some of the implicit doubting might go away (not all of it). That info could feed into the other studies.

Eli's done a great job resurrecting the work we all did.

Meta-analysis in the middle

My general view both in general and as a policymaker in my small local-office pond is that a widely held (few expert dissenters) and strongly held (high confidence) consensus position might as well be a fact as far as the policymaking is concerned. A contradictory study is just that, a study. If it turns out to be accurate, then the consensus will eventually fracture soon enough and create problems for policymakers, but there's no need or even a rational way for policymaking to jump the gun.

In the gray zone between just one study and a consensus is the meta-analysis, and I'm still trying to figure out what policymakers should do with them. If the meta-analysis reinforces the consensus position - say, Oreskes' review of scientific abstracts on climate change - then it's just a helpful tool in examining the consensus. OTOH, if the meta-analysis seems to point in a different direction - say, Choi et al. on adverse neurological effects of high-fluoride exposure in China and Iran - that presents a bit of a problem.

My assumption, generally, is that the consensus should be handling this. The main Canadian health research arm, Health Canada, looked at a lot of the same studies and reached a different conclusion:

These studies were included in the review conducted by the Expert Panel on fluoride convened by Health Canada in 2007. Despite the consistency in the results from these studies, the panel agreed that the weight of evidence does not support a link between fluoride and IQ deficit. There are significant concerns regarding the available studies, including quality, credibility, and methodological weaknesses, such as the lack of control for confounding factors, the small number of subjects, and the dose of exposure (Health Canada, 2008). Most of these studies performed in China were also included in the reviews conducted by other organizations and/or committees, which also mentioned that the significance of these studies is uncertain (IPCS, 2002; ATSDR, 2003; NRC, 2006).

I think Health Canada is probably closer to showing what the fluoride consensus is than Choi, although I think there's reason to believe the consensus, that therapeutic levels of fluoridated water are generally safe, isn't as solid as the climate consensus. Worth noting one difference between meta-analysis and consensus is the meta-analysis is only as good as the meta-analyst, while consensus hopefully has a broader base.

Maybe the consensus will change, but this is what we've got.

The expert exception to consensus reliance

I've really enjoyed Chris Mooney's Point of Inquiry podcasts, so it's too bad the latest one with pro-GMO activist Mark Lynas failed to wrestle significantly with real arguments about GMOs (Chris, you talk about GMOs too much to say you don't want to delve into technical issues).

One interesting issue did come out of the podcast - at one point Lynas says the Union of Concerned Scientists' rejection of the National Academy of Sciences position on GMOs is a contradiction of UCS arguments that we should rely on the consensus opinion on climate change.  I've argued something similar before, that the expert consensus should basically just plug in as fact in policy discussion, but there's one exception - if you yourself are an expert in that field, I think you could conclude that you're right and the consensus is wrong.  Presumably UCS could make this conclusion.

It takes a lot of hubris to conclude you're right and the other 90% of the experts are wrong. You might carefully examine whether you're missing something little or something big, and withhold judgment for a while. In the end though if you feel like you really understand the issue, you don't have a choice but to conclude what you conclude.

So what should happen next depends on who you are.  If you're the dissenting expert, you should work to move the consensus in your direction.  If you're someone else, the dissenter is just a dissenter, and only if there's enough of them to disrupt the consensus do we need to worry about them.

So no change to the big picture, but I disagree with Lynas that experts are required to toe the line of an expert consensus when they disagree with that consensus.

"The decisions came in the context of accumulating scientific evidence"

Above is from George Will of all people, acknowledging that juvenile criminal defendants have different brains and different levels of culpability than adults:

This [Supreme Court] ruling extends two others, one holding that the Eighth Amendment bars capital punishment for children under 18, the other that it bars life without parole for a juvenile convicted of a non-homicide offense. These decisions held that regarding culpability, and hence sentencing, children are constitutionally unlike adults. The decisions came in the context of accumulating scientific evidence about increased impulsivity and diminished responsibility because of adolescent brain development.

I vacillate between wanting to react with snarkiness versus simply welcoming sanity when I find it coming from an unexpected person.  I sure wish he'd replicate his interest in accumulation of scientific evidence to the far more conclusive area of climate change.  Or better yet, I'd like to know why a person like Will can think reasonably in some regards (this isn't the first time for him) and fall so short in others.

Anyway, thanks for the sanity, George.  Consider expanding it.

A scientist is a feather, a lawyer is a sail

Some time ago I guest-lectured to some undergrads in a science curriculum track about environmental advocacy. I said I had read somewhere that a scientist is a feather and the evidence is the wind - the scientist makes no effort to control the evidence but just floats wherever it takes her. Obviously this is an incomplete construct that ignores hypothesis formation etc., but is supposed to represent the ideal of how a scientist reacts to evidence.

The advocate isn't a feather, neutral as to where the wind blow. I didn't have a good analogy then for an environmental advocate/lawyer, but now I think the advocate is a sail and the evidence is the wind - how and where it blows is critical, but you have a role as well in where you're going. I also like the sail analogy instead of a sailor, retaining the ideal that the lawyer is a tool of the client/sailor (ideally) and isn't in charge of the ship.

What the sail analogy doesn't capture is the idea that legal contests are pattern-fitting contests. One side says the present facts and law fit that side's represented pattern of facts and law, while the other side presents different patterns.

Still working it out.

John McPhee, pre-1960 geology, and the climate consensus

A previous post refers to my semi-fruitless quest for a precedent of science in any field being as incredibly 100% wrong as the denialists claim is the case for climate. Commenters there suggested geology before plate tectonics as the best shot. Someone mentioned McPhee's Assembling California for context examining the history of the science of plate tectonics, a book I've had sitting around and unread.

Off to the races!

[Wary of multiple theories by some geologists for what made mountains rise,] many more geologists would not venture further than than to say (indisputably) that "earth forces" or "orogenic forces" had lifted the geosynclines, and that these forces were "not well understood".

[regarding different California mountain geosynclines thrust on top of each other,] "that was the Golconda Thrust. No one knew how this 'orogeny' happened."

[on one side of a mountain range geosyncline] there were shallow-water sediments followed by deep-water material, but there was no other side. "That was never explained".

"the geosynclinal cycle was said to be about two hundred million years. In the Overthrust Belt in Montana, forty thousand feet of Precambrian sediment had been thrust over Cretaceous sediment. As students, we wondered why all that Precambrian was still there. What had the source geosyncline been doing sitting there for a billion years when the cycle was two hundred million? There was no answer."

Halls's idea [orogeny not from tectonics] was not preposterous. It was incomplete. There was, after all, marine rock in mountains. Between the geosyncline and the mountains, though, something was missing, and what was missing was plate tectonics.

(text excerpts pages 38-40).

I think the picture isn't of a scientific field that's confident in a wrong paradigm, but one that has many acknowledged, open questions and hadn't yet accepted a solution that was proven with the subsequent accumulation of evidence. This isn't a matter of overconfidence, the claim made by denialists against climatology.

There's also the issue of whether European geologists were more open to tectonics than Americans prior to 1960, something I don't know anything about.

Granted, this is a pop-sci book, but McPhee's pretty good, so I'll see what else he has to say on this subject.

UPDATE: some great comments below. Read them! In particular, I did wheel reinventing from a 2008 comment at Deltoid:

[tectonics is] a good illustration of one flavor of paradigm shift, in this case, where plausible hypotheses were identified early, but evidence just didn't get strong enough for a long time, but when new kinds of evidence popped up, the discipline pretty much changed views in a decade.

But indeed, the evidence for AGW is (by now) immensely stronger than the evidence for continental drift in 1920. After all, Arrhenius was talking about Greenhouse Effect over 100 years ago, and that wasn't accepted instantly either :-)

And also this:

For a proper comparison in your search for "a precedent of science in any field being as incredibly 100% wrong as the denialists claim is the case for climate.", you really need to consider the supposed "wrong-headed" theory in the light of the existing evidence base. In other words we want a theory that is "bone-headed" in the context of the knowledge-base pertaining at the time.

So Newtonian dynamics isn't a teribly good example since it was a theory that was entirely consistent with the existing evidence base).