But a question arises: should we feel comfortable accepting one theory over another merely because it is more likely than the alternative? Technically the other may still be correct, just less likely.
And furthermore: what should be the threshold for when we say a theory is unlikely enough that it is ruled out? The particle physics community has agreed at the 5σ level which is a fancy pants way of saying essentially the theory has a 99.9999426697% chance of being wrong. Is this too high, too low or just right?
The Inverse Problem: For an example lets assume that supersymmetry (SUSY) is correct and several SUSY particles are observed at the LHC. Now, it seems like there are 5 bajillion SUSY models that can explain the same set of data. For example, I coauthored a paper on SUSY where we showed that for a certain SUSY model, a wide variety of next-to-lightest SUSY particles are possible. (See plot above). Furthermore, other SUSY models can allow for these same particles.
So, how do we decide between this plethora of models given many of them can find a way to account for the same data? I am calling this the inverse problem: the problem where many theories allow for the same data so given that data how can you know what theory is correct?
So again I will ask: should we really be choosing between two theories that can reproduce the same data but one has an easier time doing it than another? Is this just a sophisticated application of Ockham's razor? Should we as scientists say "The LHC has shown this theory X to be true" when in reality theory Y could technically also be true but is just far less likely?
What do you think?
I think what we have here is the quantum mechanical equivalent of theories. If there is no clear distinction between two theories, but we only have a probability that one may be correct, then I don't think that it makes sense to talk about one being "true" in the same sense that the heliocentric model, or GR, or the photoelectric effect, was shown to be "true". It would be like talking about the exact location of a quantum particle, rather than its wave function.
ReplyDeleteIf we are going to start talking about certain theories being "true" or "correct" with the condition that we have only shown that there is a 2 sigma probability that a particular theory has more predictive power than another, then we will have to change the way we talk about it. Yes this will cause confusion and it will add yet another layer of technobabble on top of an already complex field, but it just might help dispel some misconceptions about what has been "proven true" by the experiments.
In general, complex statistical tests are only really relevant when whatever you are trying to prove isn't in a special class of results that are quite simply obvious. That's what makes a result like this image of the bullet cluster so amazing. Nobody needs to compute the chi-squared value or do a Kolmogorov-Smirnov test to confirm it, you just look at the image and see the result. My test for whether complex statistics are needed is simply showing it to a group of experts and seeing if they can immediately pull-out the result.
ReplyDeleteGreat question. I actually took a course on Inverse Problem theory. It was very enlightening. To answer your question, no, I don't think it's fair to talk about likely scenarios as being "true" or "false." Rather, I think we should be honest, and say we believe something to be the case.
ReplyDeleteHowever, our world is less interested in such a thing. It applies in religion too. People are happy to convert faith into knowledge if they can up the "testimony" ante just a bit.
I think a sometimes more meaningful to look at the problem is to think of uncertainty as it's inverse - information. There is information contained in events, experiments, observations. Our job is to say what is most likely given the information in the system. I think we're fooling ourselves if we don't admit that virtually ALL of what we believe is based on bayesian statistics. Science itself is based on the highly probable assumption that the future will behave like the past. Newton's laws held 200 years ago, and they hold now. Clearly an inductive argument, but, I think, a valid one.
Bayesian inference, in and of itself, is nothing more than a valid mechanism of inductive reasoning. Can it be flawed, and produce erroneous results? Absolutely.
I suppose for me, I don't really think of ANY scientific theory as absolute. More like an extremely likely approximation. This leaves room for new theories to be adopted as we learn more.
I think Bayesian inference is quite a reliable mechanism. I think we just need to remember that it does not produce a solution, or answer, merely a best guess.
Great post. What exactly do you mean when you use the word "true" in this context? I ask because it seems obvious that we ought to *believe* the most likely theory -- in the sense that if you had to place a single bet on one theory (and you couldn't lay off bets at different odds on the range of theories), you ought to bet on the most likely theory. But what more could you want from a theory than that you believe it? I usually think of "is true" as expressing the conviction that a belief in the proposition (theory) being described would survive sustained, honest inquiry. But I don't see how that helps us decide whether to say right now whether a theory is true. Or at least, I don't see how to separate it from saying that the theory is to be believed.
ReplyDeleteI really hate bringing this up (for various reasons), but how do you see the priors working in these contexts? When theories predict the same evidence in a range of cases that we've actually tested, it seems that the priors are going to have a lot of influence on the posterior probability one assigns to the theories. Or am I missing something?
Finally, I'm pretty sure the preferred spelling for the name of the medieval nominalist philosopher is "Ockham"; I've also seen "Occam" but not "occum." ;)
Thanks everyone.
ReplyDeleteJonathan,
1.) I think the last thing I want to do is try to define what it means to be true for a philosophy expert. :) And I think your suggesting as it being a conviction is an interesting one.
Usually when I think of true is that it is a real property of nature but then I guess I have to start defining those worlds are so lets just say true means to me "How nature really is". (Admitting that there are issues with this definition.)
2.) Yes priors are an issue and what happens in practice is everyone who is a coauthor of a science paper scrutinizes over the priors they are using and often referees will demand the paper addresses the issue. So yes, priors cause issues but in practice scientists are forced to face up to those issues when they want to publish and sometimes they have to admit things change with priors.
3.) Sorry about the misspelling. I will fix that now. Thanks for the heads up.