Category Archives: Epistemology

Followup to:  Disguised Queries, Superexponential Conceptspace

If a tree falls in the forest, and no one hears it, does it make a sound?

"Tell me why you want to know," says the rationalist, "and I’ll tell you the answer."  If you want to know whether your seismograph, located nearby, will register an acoustic wave, then the experimental prediction is "Yes"; so, for seismographic purposes, the tree should be considered to make a sound.  If instead you’re asking some question about firing patterns in a human auditory cortex – for whatever reason – then the answer is that no such patterns will be changed when the tree falls.

What is a poison?  Hemlock is a "poison"; so is cyanide; so is viper venom.  Carrots, water, and oxygen are "not poison".  But what determines this classification?  You would be hard pressed, just by looking at hemlock and cyanide and carrots and water, to tell what sort of difference is at work.  You would have to administer the substances to a human – preferably one signed up for cryonics – and see which ones proved fatal.  (And at that, the definition is still subtler than it appears: a ton of carrots, dropped on someone’s head, will also prove fatal. You’re really asking about fatality from metabolic disruption, after administering doses small enough to avoid mechanical damage and blockage, at room temperature, at low velocity.)

Where poison-ness is concerned, you are not classifying via a strictly local property of the substance.  You are asking about the consequence when a dose of that substance is applied to a human metabolism.  The local difference between a human who gasps and keels over, versus a human alive and healthy, is more compactly discriminated, than any local difference between poison and non-poison.

Continue reading "Unnatural Categories" »

Followup to:  Passing the Recursive Buck, No License To Be Human

Roko has mentioned his "Universal Instrumental Values" several times in his comments.  Roughly, Roko proposes that we ought to adopt as terminal values those things that a supermajority of agents would do instrumentally.  On Roko’s blog he writes:

I’m suggesting that UIV provides the cornerstone for a rather new approach to goal system design. Instead of having a fixed utility function/supergoal, you periodically promote certain instrumental values to terminal values i.e. you promote the UIVs.

Roko thinks his morality is more objective than mine:

It also worries me quite a lot that eliezer’s post is entirely symmetric under the action of replacing his chosen notions with the pebble-sorter’s notions. This property qualifies as "moral relativism" in my book, though there is no point in arguing about the meanings of words.

My posts on universal instrumental values are not symmetric under replacing UIVs with some other set of goals that an agent might have. UIVs are the unique set of values X such that in order to achieve any other value Y, you first have to do X.

Continue reading "Invisible Frameworks" »

Followup to:  You Provably Can’t Trust Yourself

Yesterday I discussed the difference between:

• A system that believes – is moved by – any specific chain of deductions from the axioms of Peano Arithmetic.  (PA, Type 1 calculator)
• A system that believes PA, plus explicitly asserts the general proposition that PA is sound.  (PA+1, meta-1-calculator that calculates the output of Type 1 calculator)
• A system that believes PA, plus explicitly asserts its own soundness.  (Self-PA, Type 2 calculator)

These systems are formally distinct.  PA+1 can prove things that PA cannot.  Self-PA is inconsistent, and can prove anything via Löb’s Theorem.

With these distinctions in mind, I hope my intent will be clearer, when I say that although I am human and have a human-ish moral framework, I do not think that the fact of acting in a human-ish way licenses anything.

I am a self-renormalizing moral system, but I do not think there is any general license to be a self-renormalizing moral system.

And while we’re on the subject, I am an epistemologically incoherent creature, trying to modify his ways of thinking in accordance with his current conclusions; but I do not think that reflective coherence implies correctness.

Continue reading "No License To Be Human" »

Followup to:  Where Recursive Justification Hits Bottom, Löb’s Theorem

Peano Arithmetic seems pretty trustworthy.  We’ve never found a case where Peano Arithmetic proves a theorem T, and yet T is false in the natural numbers.  That is, we know of no case where []T ("T is provable in PA") and yet ~T ("not T").

We also know of no case where first order logic is invalid:  We know of no case where first-order logic produces false conclusions from true premises. (Whenever first-order statements H are true of a model, and we can syntactically deduce C from H, checking C against the model shows that C is also true.)

Combining these two observations, it seems like we should be able to get away with adding a rule to Peano Arithmetic that says:

All T:  ([]T -> T)

But Löb’s Theorem seems to show that as soon as we do that, everything becomes provable.  What went wrong?  How can we do worse by adding a true premise to a trustworthy theory?  Is the premise not true – does PA prove some theorems that are false?  Is first-order logic not valid – does it sometimes prove false conclusions from true premises?

Continue reading "You Provably Can’t Trust Yourself" »