Me in ’06: You can get 80% of the improvement that prediction markets offer by using a much simpler solution: collect track records. … When people make forecast-like-statements, write them down in a clear standardized form, and then check back later to see who was more accurate.
Farhi hopes that something (anything, please!) will be found that is not currently predicted by the Standard Model. Otherwise High-energy particle physics in its current form, accurate to 10 decimal places, will be almost completely explained.
What's wrong with the Standard Model being correct? It would appear that 10 decimal places accuracy is sufficient to indicate a theory correct and complete.
High-energy particle physicists are worried that, should the LHC prove to be a $6B boondoggle contributing little or nothing to knowledge, they will be unable to get future funding and will face future unemployment. Farhi admits he is hedging his bets by doing side research in quantum computing (that's why he's at Google).
I hope the LHC finds nothing new. Then research money could be redirected from high-energy particle physics to more productive areas of physics and the other sciences.
I think you misunderstand the nature of dialogue in physics. Certainly certain physicists research predominantly certain ideas, and many if not most view the ideas they research more favorably than those who don't research those ideas. However, physicists appreciate that it is Nature that decides which theories are correct. If person X's theory is confirmed, X will NOT claim to have been "right" or to have "known all along," as if he or she had some special access to truth.
There is some historical precedent for this. For example, long ago Coleman developed a unified theory of all (non-gravitational) forces, and its simplicity and elegance made it quite compelling. It made a testable prediction about proton decay, which subsequently was shown to be incorrect. This history is described among physicists as "unfortunate" -- in the sense that Coleman's theory could have been correct but it simply wasn't the way of things -- and not as if Coleman somehow lacked insight for making an incorrect prediction.
Perhaps it helps to clarify that physicists are hoping that the LHC will discern between different *theories*, not the predictions of a given theory. Perhaps at times, a given theory was seen as complicated or not fully understood, and different people decided the theory should predict different outcomes for the same experiment. Then, if the experiment were performed and one were right, the community might view that physicist as the "more insightful" one. But that's because physicists view the theory as making just one prediction, and understanding that prediction is a matter of understanding the theory. In the present case, we have competing theories, and there is no understanding that can reveal which theory is correct -- all you can do is check the results of experiment and see.
It should be noted that Mike Kenny made an excellent series of comments in this thread.Mike, I don't see why Robin would be skeptical about your proposals, they sound modestly achievable, sound, and useful in filling an information gap.
Garrett, was great to meet you and to realize you actually knew who I was! :) Good to hear you make predictions, but yours aren't quite scoreable yet, as you neglected to assign them probabilities.
Hi Robin, it was good to meet you at SciFoo. To generalize your post, I think it would be very cool to have a technical futures market, where anyone could bet money on scientifically testable predictions. But, barring that, Sean's probabilities seem about right, though mine differ a little. In particular, my horse in this race is consistent with the appearance of a Higgs, a W' and Z', and some colored bosons. If some of these are seen (which Sean guestimates at less than a 2% chance) I will be extremely happy, and probably crow a bit. If anything else appears at the LHC, my horse and crow are probably cooked.
I don't think that's what I said. "If X, then Y" is a prediction. But we have no means of favouring that prediction over "If A, then B". If we did, we wouldn't need the experiment. So, nobody is going to come out and say "A is true, so we will see B". There's a difference between believing "If A then B" and believing that A is more likely than X.
Well, I think this is sort of the point - physics doesn't have any information to choose one model over another. If we did, we wouldn't have to do the experiment. It's all of the form "if X, then Y", with nothing to tell you how to choose between the X other than seeing which Y you get. Nobody's going to stake any reputation on a gut feel.
onymous, if people want to crow about being vindicated, then people must make forecasts. It is not enough that there were 1000 papers that each made a forecast, and then crowing in retrospect about the one paper that got closest - bigshot physicists should choose now which of those paper forecasts to endorse to what degree.
FAMOUS SCIENTIST TO ROBIN HIGH IQ HANSON: Science, which is a very long-term endeavor, does not need your stickin idea about scoreable predictions and track records. Please, go back to minding economic issues in your Ivory Tower, and let us run science ou
Overcoming Whatever:I dont really think the comparison with sports/business/weather forecasters really holds up, for a prosaic reason in particle physics, the timescale for experiments is years and decades, not days. There is no way to ...
sean carroll, who's funding the lhc? $10bln for a hunch seems pretty unlikely to me when they shoot down hubble because it's expensive. there must be something else there.
I don't understand what you're asking for. There are zillions of papers that explain what the LHC would see given a particular model. Those are making predictions. But of course for most of those models no one expects them to be exactly right, not even the authors. I don't see how to attach any meaningful numerical "confidence" to any prediction.
Sean, I don't understand the relevance of the timescale to the efficient grading of predictions. Given enough forecasts we can see a signal of accuracy above the noise of luck in individual forecasts. I agree that the longer the timescale the weaker are incentives from any given reward tied to scoring. But I'm not really focused on incentives in this post - I'm focused on whether it is reasonable for folks to crow about being vindicated when they weren't willing to make scoreable forecasts.
Is this more along the lines of what you wanted - a prediction on finding the Higgs?
"CERN is losing ground rapidly in the race to discover the elusive Higgs boson, its American rival claims.
Fermilab say the odds of their Tevatron accelerator finding it first are now 50-50 at worst, and up to 96% at best."
Just finished watching Google Tech Talks presentation by Edward Farhi at
"Why Physicists Need the LHC"http://www.youtube.com/watc...
Farhi hopes that something (anything, please!) will be found that is not currently predicted by the Standard Model. Otherwise High-energy particle physics in its current form, accurate to 10 decimal places, will be almost completely explained.
What's wrong with the Standard Model being correct? It would appear that 10 decimal places accuracy is sufficient to indicate a theory correct and complete.
High-energy particle physicists are worried that, should the LHC prove to be a $6B boondoggle contributing little or nothing to knowledge, they will be unable to get future funding and will face future unemployment. Farhi admits he is hedging his bets by doing side research in quantum computing (that's why he's at Google).
I hope the LHC finds nothing new. Then research money could be redirected from high-energy particle physics to more productive areas of physics and the other sciences.
I think you misunderstand the nature of dialogue in physics. Certainly certain physicists research predominantly certain ideas, and many if not most view the ideas they research more favorably than those who don't research those ideas. However, physicists appreciate that it is Nature that decides which theories are correct. If person X's theory is confirmed, X will NOT claim to have been "right" or to have "known all along," as if he or she had some special access to truth.
There is some historical precedent for this. For example, long ago Coleman developed a unified theory of all (non-gravitational) forces, and its simplicity and elegance made it quite compelling. It made a testable prediction about proton decay, which subsequently was shown to be incorrect. This history is described among physicists as "unfortunate" -- in the sense that Coleman's theory could have been correct but it simply wasn't the way of things -- and not as if Coleman somehow lacked insight for making an incorrect prediction.
Perhaps it helps to clarify that physicists are hoping that the LHC will discern between different *theories*, not the predictions of a given theory. Perhaps at times, a given theory was seen as complicated or not fully understood, and different people decided the theory should predict different outcomes for the same experiment. Then, if the experiment were performed and one were right, the community might view that physicist as the "more insightful" one. But that's because physicists view the theory as making just one prediction, and understanding that prediction is a matter of understanding the theory. In the present case, we have competing theories, and there is no understanding that can reveal which theory is correct -- all you can do is check the results of experiment and see.
It should be noted that Mike Kenny made an excellent series of comments in this thread.Mike, I don't see why Robin would be skeptical about your proposals, they sound modestly achievable, sound, and useful in filling an information gap.
Garrett, was great to meet you and to realize you actually knew who I was! :) Good to hear you make predictions, but yours aren't quite scoreable yet, as you neglected to assign them probabilities.
Hi Robin, it was good to meet you at SciFoo. To generalize your post, I think it would be very cool to have a technical futures market, where anyone could bet money on scientifically testable predictions. But, barring that, Sean's probabilities seem about right, though mine differ a little. In particular, my horse in this race is consistent with the appearance of a Higgs, a W' and Z', and some colored bosons. If some of these are seen (which Sean guestimates at less than a 2% chance) I will be extremely happy, and probably crow a bit. If anything else appears at the LHC, my horse and crow are probably cooked.
I don't think that's what I said. "If X, then Y" is a prediction. But we have no means of favouring that prediction over "If A, then B". If we did, we wouldn't need the experiment. So, nobody is going to come out and say "A is true, so we will see B". There's a difference between believing "If A then B" and believing that A is more likely than X.
If you can't say which theories predict which experimental results now, how dare you say so after peeking at the LHC results?
Well, I think this is sort of the point - physics doesn't have any information to choose one model over another. If we did, we wouldn't have to do the experiment. It's all of the form "if X, then Y", with nothing to tell you how to choose between the X other than seeing which Y you get. Nobody's going to stake any reputation on a gut feel.
onymous, if people want to crow about being vindicated, then people must make forecasts. It is not enough that there were 1000 papers that each made a forecast, and then crowing in retrospect about the one paper that got closest - bigshot physicists should choose now which of those paper forecasts to endorse to what degree.
FAMOUS SCIENTIST TO ROBIN HIGH IQ HANSON: Science, which is a very long-term endeavor, does not need your stickin idea about scoreable predictions and track records. Please, go back to minding economic issues in your Ivory Tower, and let us run science ou
Overcoming Whatever:I dont really think the comparison with sports/business/weather forecasters really holds up, for a prosaic reason in particle physics, the timescale for experiments is years and decades, not days. There is no way to ...
thanks for the tip robin. i'll check out david brin. are there good places to look for criticisms of such approaches off the top of your head?
sean carroll, who's funding the lhc? $10bln for a hunch seems pretty unlikely to me when they shoot down hubble because it's expensive. there must be something else there.
There are mass estimates of the Higgs boson by non-string theorists with probability bars.
Try digging a little deeper.
I don't understand what you're asking for. There are zillions of papers that explain what the LHC would see given a particular model. Those are making predictions. But of course for most of those models no one expects them to be exactly right, not even the authors. I don't see how to attach any meaningful numerical "confidence" to any prediction.
Sean, I don't understand the relevance of the timescale to the efficient grading of predictions. Given enough forecasts we can see a signal of accuracy above the noise of luck in individual forecasts. I agree that the longer the timescale the weaker are incentives from any given reward tied to scoring. But I'm not really focused on incentives in this post - I'm focused on whether it is reasonable for folks to crow about being vindicated when they weren't willing to make scoreable forecasts.