I'm a little late in posting this, but I thought this ought to be discussed. We've talked a lot about the report of superluminal neutrinos and how the erroneous measurement was apparently caused by a loose cable and some clock drift. Well, apparently Dario Auterio and Antonio Ereditato both resigned from their positions as OPERA team leaders after the final investigations concluded and OPERA's collaboration board gave them a vote of "no confidence." I was personally just surprised at the direction all of this took.
On one hand, I think this underscores the need to be careful in science, particularly when making a big claim. But I also don't think this whole business has really been bad for physics. It got a lot of people thinking about physics that otherwise wouldn't, and it publicly demonstrated how scientific research is actually done. It must be embarrassing for the OPERA folks, but I think they might be over-reacting a bit here.
ReplyDeleteI agree. Personally I thought this whole scenario was exactly how physics is supposed to be done. They ran just about every experimental check they could think of, and
ReplyDeletewhen none of the errors they could find could affect the experimental
result, they had some level of obligation to publish the experimental
results. At that point the entire scientific community weighed in on
either finding a theoretical explanation or some unseen experimental
error. There was no real touting this as the experiment to disprove
Einstein, they simply explained what they did and what they saw. When,
in the end, the result was shown to be invalid (still an interesting
result, and if it hadn't been for the undue hype it would still be an
immensely interesting experiment and paper), scientists pretty quickly
reached a consensus and moved on. I think this reaction betrays a real
social obsession with finding a "scapegoat" for anything that goes
counter to what is expected or desired (similar to what happened with
the geologists in Italy). I really hope this doesn't have a chilling
effect on people hoping to move up in scientific research in these
areas. (It would be a real tragedy if this caused other scientists to
ignore results simply because they might be too risky or unwelcome in
the public eye.)
I think we are truly missing the key rationale: why was the experiment conducted? What were the objectives of the experiment and what science and hypothesis led to the formulation of the objectives as well as the design of the experiment?
ReplyDeleteIf one were to measure speed of neutrinos through rocks, I would think there would be some kind of modeling and analysis that would suggest why an experiment would be worthwhile.
Now, so many papers are published where theories are cryptic and obfuscated with math just to show off the learned blockheadedness (Ben Franklin’s phrase) while experiment designs and set-ups are left as block diagrams. Basically, you need a PhD in their su-sub-sub specialty and we are reduced to believe, trust and other non-quantifiable emotional metrics. This is not just in physics but most sciences, and the worst offenders are in drug developments, philosophy and social sciences.
I have no problem in trying something because such an experiment could be done and the results are confined within the context. They proclaimed faster than light travel and that is where they are wrong.
Let me cite a couple of parallels: BP claimed there was no serious ecological damage by that deep water well blow-up, only to now find sick fish, blind shrimps and general collapse of microbial life. And that sudden acceleration of cars that was blamed on human error, now possibly being attributed to solder whiskers in the electronics. Everything was a buzz with faster than light neutrinos until loose wires and clock drift came to light, somehow, at a speed less than lightspeed.
In this case the experiment was being conducted to study neutrino oscillations. One of the big open questions in particle physics is the rest-mass of neutrinos since the current standard model says it should be zero while observations say it must be greater than zero. Along the way they stumbled into data that showed the neutrinos were moving faster than light. Serendipitous discovery is how many if not most major advances in science come about (e.g., penicillin, genetics, the cosmic microwave background, etc.). I'm not sure how those cases are any different than this one except they turned out to be right while this one turned out to be wrong. As Bill was saying, the danger here is that scientists refrain from publishing or pursuing research because it might turn out to be wrong. Half of what I work on is probably wrong, but I keep working on all of it because I don't yet know which half is the wrong half.
ReplyDeleteLet me also say that the examples you mentioned are cases of clear conflicts of interest on the part of the scientists. The OPERA folks were not trying to shield anyone from lawsuits or make anybody money. They were doing good-faith science exactly how science should be done.
ReplyDeleteNick, you observed: Along the way they stumbled into data that showed the neutrinos were moving faster than light.
ReplyDeleteThat is my point: they did not question their data, their experimental set up, their standard (clock), but proceeded to claim a well established and mostly proven theory incorrect. I do not accuse them of wrong doing intentionally, but I do raise serious concern that we do get blinded by beliefs and even possibly by desire to be famous.
I have no problem that they published their results, it is their claim of faster than speed of light that is bothersome. It is like we have over a trillion events in search of Higgs, and four more trillions this year, and all we can say that we have not seen Higgs, except that stat analysis indicates that we may have seen a Higg or two like the black swan of Nasib Taleb (author: Fooled by Randomness).
These two scientists would have made "money" in the form of fame, lecture circuits, and other means of making money like tenured professorships etc etc. At minimum, their group would be funded much more easily than other groups.
They most certainly did question their data and their set-up and when they could not find any problem with it they published the data. Their paper was very clearly skeptical of their own result. They put forward their data and pointed out that if correct it implied that the neutrinos were traveling very slightly faster than the speed of light, which I think was the responsible thing to do at that point, so that others could test their results. Independent testing cannot happen if somebody keeps possible controversial data to themselves because it might be wrong.
ReplyDeleteThe original paper on the topic concludes with this:
ReplyDelete"...despite the large significance of the measurement reported here and the
robustness of the analysis, the potentially great impact of the result motivates the continuation of
our studies in order to investigate possible still unknown systematic effects that could explain the
observed anomaly. We deliberately do not attempt any theoretical or phenomenological
interpretation of the results."
I liked what you said about "Half of what I work on is probably wrong, but I keep working on all of
ReplyDeleteit because I don't yet know which half is the wrong half." That seems to be the case more often than we would really like to admit.
That reminds me of a quote,
"An experimentalist is a scientist, where, after giving a talk about his research, everyone understands what he did and everyone believes his results, except the person who gave the talk.
"A theorist is a scientist, where, after giving a talk about his research, no one
understands what he did and no one believes his results, except the
person who gave the talk."
NIck,
ReplyDeleteSo they were saying that this is false positive in lots of weasel words while creating a extraordinary buzz on faster than light travel. I know of no major experiment where tons of data, especially initial data, seem contradictory to what was expected. Most do not go publishing these data but work their behind off to assure integrity of the data.
We now live in the world where data explosion is so large that we can basically find data to support any observation we want to make.
I do not think they intended to mislead but I would suspect that they were delusioned by their own smartness.
Well one major and famous experiment of that sort is the Michelson & Morley experiment:
ReplyDeletehttp://en.wikipedia.org/wiki/Michelson%E2%80%93Morley_experiment
Also, given the magnitude of complexity in modern high energy physics experiments the term loose cable does not accurately describe the elusiveness of experimental systematic errors.
Reading the statements Nick quoted, I don't see any "weasel words." They assumed that this was a false positive, but after they had already "work[ed] their behind[s] off to assure integrity of the data", and could not find any error, they essentially had to report the data. It was the only scientifically responsible thing to do. To not report experimental data that has been checked and double-checked and subjected to the most rigorous statistical and experimental analysis that can be considered reasonable, just because the data contradicts a physical theory, no matter how well established, is scientifically unethical, one step below falsifying experimental data.
ReplyDeleteThe current case was not one of "jumping the gun," reporting experimental data prematurely nor before it had been subjected to rigorous analysis, nor was it a case of sensationalizing the report just to make a big splash. They did good faith science, found a very surprising result, double-checked everything, but when the result persisted, they simply reported what they saw. In science it is important to report unexpected results, especially suspected false results, so that they can be analyzed by the larger scientific community. If this measurement had simply been buried, no one would have been able to check their results and look for possible errors that they themselves could not find (Remember, this was a ridiculously difficult error to find!), nor run independent tests of the result.
There are lots of times in the history of science where an unexpected result that flies in the face of conventional wisdom, both on the part of experiment and theory. Modern, more complex experiments make checking for experimental errors that much more difficult, but the principles are the same. Modern science would have never developed if every unexpected result had been buried while the researchers looked for every possible quantum-sized "needle in the haystack" that might be an error. To expect such of every unexpected result is simply not reasonable. Rigorously performed experimental and theoretical results, if the scientist performing the experiment or theory cannot find an error after all reasonable checking and double-checking has been done, ethically must be reported.
Excellent example!
ReplyDelete