Category Archives: Bayesian

Probability is Subjectively Objective

Followup toProbability is in the Mind

"Reality is that which, when you stop believing in it, doesn’t go away."
        — Philip K. Dick

There are two kinds of Bayesians, allegedly.  Subjective Bayesians believe that "probabilities" are degrees of uncertainty existing in our minds; if you are uncertain about a phenomenon, that is a fact about your state of mind, not a property of the phenomenon itself; probability theory constrains the logical coherence of uncertain beliefs.  Then there are objective Bayesians, who… I’m not quite sure what it means to be an "objective Bayesian"; there are multiple definitions out there.  As best I can tell, an "objective Bayesian" is anyone who uses Bayesian methods and isn’t a subjective Bayesian.

If I recall correctly, E. T. Jaynes, master of the art, once described himself as a subjective-objective Bayesian.  Jaynes certainly believed very firmly that probability was in the mind; Jaynes was the one who coined the term Mind Projection Fallacy.  But Jaynes also didn’t think that this implied a license to make up whatever priors you liked.  There was only one correct prior distribution to use, given your state of partial information at the start of the problem.

How can something be in the mind, yet still be objective?

Continue reading "Probability is Subjectively Objective" »

GD Star Rating

Where Recursive Justification Hits Bottom

Followup toNo Universally Compelling Arguments, Passing the Recursive Buck, Wrong Questions, A Priori

Why do I believe that the Sun will rise tomorrow?

Because I’ve seen the Sun rise on thousands of previous days.

Ah… but why do I believe the future will be like the past?

Even if I go past the mere surface observation of the Sun rising, to the apparently universal and exceptionless laws of gravitation and nuclear physics, then I am still left with the question:  "Why do I believe this will also be true tomorrow?"

I could appeal to Occam’s Razor, the principle of using the simplest theory that fits the facts… but why believe in Occam’s Razor?  Because it’s been successful on past problems?  But who says that this means Occam’s Razor will work tomorrow?

And lo, the one said:

"Science also depends on unjustified assumptions.  Thus science is ultimately based on faith, so don’t you criticize me for believing in [silly-belief-#238721]."

Continue reading "Where Recursive Justification Hits Bottom" »

GD Star Rating

All Hail Info Theory

Hard questions are often hard because different ways to think about them conflict.  When each way seems to have strong support, we are reluctant to choose.  But if we cannot avoid the conflict, choose we must.  For example, last October I wrote:

Our standard ("Bayesian") formal theories of information and probability … are by far the main formal approaches to such issues in physics, economics, computer science, statistics, and philosophy.  … There are, however, a number of claimed exceptions, cases where many people think certain beliefs are justified even though they seem contrary to this standard framework. … I am … tempted to reject all claimed exceptions, but that wouldn’t be fair.  So I’m instead raising the issue and offering a quick survey of claimed exceptions. … The following do not seem to be exceptions: Indexicals … Logical Implications … Here are possible exceptions:  Math and Concept Axioms Basic Moral ClaimsConsciousnessThe Real World Real Stuff … [I could have added religious beliefs to this list.]

Actually, in all these cases it seems it is standard info theory (i.e., info is whatever excludes possibilities) alone that seems to conflict with something else – probability theory is irrelevant.  And it seems to me that: Nothing that seems to conflict with standard info theory is remotely as well established as it is.  So when there is a conflict, info theory must just win.  (More are willing to challenge standard "Bayesian" probability theory – e.g., see Andrew Gelman, Scott Aaronson.) 

Continue reading "All Hail Info Theory" »

GD Star Rating
Tagged as:

Timeless Causality

This post is part of the Quantum Physics Sequence.
Followup toTimeless Physics

Julian Barbour believes that each configuration, each individual point in configuration space, corresponds individually to an experienced Now – that each instantaneous time-slice of a brain is the carrier of a subjective experience.

On this point, I take it upon myself to disagree with Barbour.

There is a timeless formulation of causality, known to Bayesians, which may glue configurations together even in a timeless universe.  Barbour may not have studied this; it is not widely studied.

Such causal links could be required for “computation” and “consciousness” – whatever those are.  If so, we would not be forced to conclude that a single configuration, encoding a brain frozen in time, can be the bearer of an instantaneous experience.  We could throw out time, and keep the concept of causal computation.

Continue reading "Timeless Causality" »

GD Star Rating

That Alien Message

Followup toEinstein's Speed

Imagine a world much like this one, in which, thanks to gene-selection technologies, the average IQ is 140 (on our scale).  Potential Einsteins are one-in-a-thousand, not one-in-a-million; and they grow up in a school system suited, if not to them personally, then at least to bright kids.  Calculus is routinely taught in sixth grade.  Albert Einstein, himself, still lived and still made approximately the same discoveries, but his work no longer seems exceptional.  Several modern top-flight physicists have made equivalent breakthroughs, and are still around to talk.

(No, this is not the world Brennan lives in.)

One day, the stars in the night sky begin to change.

Some grow brighter.  Some grow dimmer.  Most remain the same.  Astronomical telescopes capture it all, moment by moment.  The stars that change, change their luminosity one at a time, distinctly so; the luminosity change occurs over the course of a microsecond, but a whole second separates each change.

It is clear, from the first instant anyone realizes that more than one star is changing, that the process seems to center around Earth particularly. The arrival of the light from the events, at many stars scattered around the galaxy, has been precisely timed to Earth in its orbit.  Soon, confirmation comes in from high-orbiting telescopes (they have those) that the astronomical miracles do not seem as synchronized from outside Earth.  Only Earth's telescopes see one star changing every second (1005 milliseconds, actually).

Almost the entire combined brainpower of Earth turns to analysis.

Continue reading "That Alien Message" »

GD Star Rating

Einstein’s Speed

Followup toFaster Than Science

Yesterday I argued that the Powers Beyond Science are actually a standard and necessary part of the social process of science.  In particular, scientists must call upon their powers of individual rationality to decide what ideas to test, in advance of the sort of definite experiments that Science demands to bless an idea as confirmed.  The ideal of Science does not try to specify this process – we don’t suppose that any public authority knows how individual scientists should think – but this doesn’t mean the process is unimportant.

A readily understandable, non-disturbing example:

A scientist identifies a strong mathematical regularity in the cumulative data of previous experiments.  But the corresponding hypothesis has not yet made and confirmed a novel experimental prediction – which his academic field demands; this is one of those fields where you can perform controlled experiments without too much trouble.  Thus the individual scientist has readily understandable, rational reasons to believe (though not with probability 1) something not yet blessed by Science as public knowledge of humankind.

Noticing a regularity in a huge mass of experimental data, doesn’t seem all that unscientific.  You’re still data-driven, right?

But that’s because I deliberately chose a non-disturbing example.  When Einstein invented General Relativity, he had almost no experimental data to go on, except the precession of Mercury’s perihelion.  And (AFAIK) Einstein did not use that data, except at the end.

Continue reading "Einstein’s Speed" »

GD Star Rating

Faster Than Science

Followup toScience Doesn’t Trust Your Rationality, Einstein’s Arrogance

I sometimes say that the method of science is to amass such an enormous mountain of evidence that even scientists cannot ignore it; and that this is the distinguishing characteristic of a scientist, a non-scientist will ignore it anyway.

Max Planck was even less optimistic:

"A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it."

I am much tickled by this notion, because it implies that the power of science to distinguish truth from falsehood ultimately rests on the good taste of grad students.

The gradual increase in acceptance of many-worlds in academic physics, suggests that there are physicists who will only accept a new idea given some combination of epistemic justification, and a sufficiently large academic pack in whose company they can be comfortable.  As more physicists accept, the pack grows larger, and hence more people go over their individual thresholds for conversion – with the epistemic justification remaining essentially the same.

But Science still gets there eventually, and this is sufficient for the ratchet of Science to move forward, and raise up a technological civilization.

Scientists can be moved by groundless prejudices, by undermined intuitions, by raw herd behavior – the panoply of human flaws.  Each time a scientist shifts belief for epistemically unjustifiable reasons, it requires more evidence, or new arguments, to cancel out the noise.

Continue reading "Faster Than Science" »

GD Star Rating

Changing the Definition of Science

New Scientist on changing the definition of science, ungated here:

Others believe such criticism is based on a misunderstanding. "Some people say that the multiverse concept isn’t falsifiable because it’s unobservable – but that’s a fallacy," says cosmologist Max Tegmark of the Massachusetts Institute of Technology. He argues that the multiverse is a natural consequence of such eminently falsifiable theories as quantum theory and general relativity. As such, the multiverse theory stands or fails according to how well these other theories stand up to observational tests.
So if the simplicity of falsification is misleading, what should scientists be doing instead? Howson believes it is time to ditch Popper’s notion of capturing the scientific process using deductive logic. Instead, the focus should be on reflecting what scientists actually do: gathering the weight of evidence for rival theories and assessing their relative plausibility.

Continue reading "Changing the Definition of Science" »

GD Star Rating

Science Isn’t Strict Enough

Followup toWhen Science Can’t Help

Once upon a time, a younger Eliezer had a stupid theory.  Eliezer18 was careful to follow the precepts of Traditional Rationality that he had been taught; he made sure his stupid theory had experimental consequences.  Eliezer18 professed, in accordance with the virtues of a scientist he had been taught, that he wished to test his stupid theory.

This was all that was required to be virtuous, according to what Eliezer18  had been taught was virtue in the way of science.

It was not even remotely the order of effort that would have been required to get it right.

The traditional ideals of Science too readily give out gold stars. Negative experimental results are also knowledge, so everyone who plays gets an award.  So long as you can think of some kind of experiment that tests your theory, and you do the experiment, and you accept the results, you’ve played by the rules; you’re a good scientist.

You didn’t necessarily get it right, but you’re a nice science-abiding citizen.

Continue reading "Science Isn’t Strict Enough" »

GD Star Rating

The Dilemma: Science or Bayes?

Followup toIf Many-Worlds Had Come First, The Failures of Eld Science

"Eli: You are writing a lot about physics recently.  Why?"
        — Shane Legg (and several other people)

"In light of your QM explanation, which to me sounds perfectly logical, it seems obvious and normal that many worlds is overwhelmingly likely. It just seems almost too good to be true that I now get what plenty of genius quantum physicists still can’t. […] Sure I can explain all that away, and I still think you’re right, I’m just suspicious of myself for believing the first believable explanation I met."
        — Recovering irrationalist

RI, you’ve got no idea how glad I was to see you post that comment.

Of course I had more than just one reason for spending all that time posting about quantum physics.  I like having lots of hidden motives, it’s the closest I can ethically get to being a supervillain.

But to give an example of a purpose I could only accomplish by discussing quantum physics…

In physics, you can get absolutely clear-cut issues.  Not in the sense that the issues are trivial to explain.  But if you try to apply Bayes to healthcare, or economics, you may not be able to formally lay out what is the simplest hypothesis, or what the evidence supports.  But when I say "macroscopic decoherence is simpler than collapse" it is actually strict simplicity; you could write the two hypotheses out as computer programs and count the lines of code. Nor is the evidence itself in dispute.

I wanted a very clear example – Bayes says "zig", this is a zag – when it came time to break your allegiance to Science.

Continue reading "The Dilemma: Science or Bayes?" »

GD Star Rating