Half of of three sigma detections are wrong.On one hand the quote is funny because three sigma is supposed to mean, roughly, that the result has a 99.7% chance of being correct.
However, there is truth to the quote, not because you can't trust statistics, but because so often scientists are not aware of everything that can be causing a false signal.
For example, and this is what provoked the quote, we heard a 9-sigma result saying there was a preferred direction to the universe. The statistics were done right, the team just wasn't aware that the along the eclipitic plane there is a preferred direction of photons due to our own galaxy, Once you account for these photons, the 9 sigma detection goes away.
So perhaps the quote should be:
Half of of three sigma detections are wrong because, all too often, physicists don't understand the background signals as well as they think they do.
The biggest threat in most cases is some sort of systematic bias. Statistics can take care of the devils you know in your data, but there is really no defense against the devils you don't even know about.
ReplyDeleteHere's an example of the possibilities for systematic errors that is sure the upset Joe and the rest of the cosmologists out there. Almost all of what we now know about dark energy comes from results using type Ia supernovae, which have to be corrected for reddening by dust. If it turns out that there is, for some unknown reason, more 10% dust around type Ia supernova per Delta z (change in redshift, a proxy for distance), then the expansion of the universe is not accelerating and we've simply been fooled by a systematic error.
ReplyDeleteI'm not saying that is the case, but merely that systematic errors can masquerade as physical effects. And the real problems come in when you run into systematic errors that you haven't even thought of yet, let alone tested or tried to account for them.
ReplyDeleteNick, I actually wouldn't be upset for 2 reasons:
ReplyDelete1. I know even more issues with the supernova data so I am well aware that there could be issues there.
2. My claim in past posts is that we should assume lambda exists since it is allowed for the symmetries in the Lagrangian. But, like E&M constants, there are no theoretical arguments for what exact values they take on without doing an experiment. This puts lambda as being something that should be as suspected as the constants in E&M are. I don't care that the value is wrong, only that it should be there.
And if lambda is zero, that deserves a huge explanation since we have to have a new symmetry forcing it to be zero not currently employed in GR.
Let's just say it like this: being around real cosmologists I know of several issues with lots of data from many experiments, even WMAP. Yes, it may be half of the 3-sigma signals are false.
ReplyDeleteYet at the same time, on the theory side, you don't randomly put constants to zero that are consistant with symmetries. This rule has served particle physicists very well and so there is no reason cosmologists should violate it with lambda.