Phil: the more I think about it, the more I’m convinced we have to reject any formulation of the paradoxes in which the god or intelligence has certainty* about our decisions or reasons.

If we imagine that we and the god are both in a simulation, then doesn’t the claim the god can be certain about whether our repentance was planned or ‘genuine’ imply that they (one program) can prove a nontrivial property about another (arbitrary) program? Which would seem to run afoul of Rice’s Theorem/the Halting Problem.

There are a couple ways out of this that I see:

1) We can assume the god is or has access to, an oracle. This is explicitly unrealistic though, and I regard such an out as being as bad as saying ‘imagine a universe where 1!=1; Wouldn’t rationality really suck there?’.
2) We can assume that sentient programs are sufficiently restricted that Rice’s theorem doesn’t apply. That is, any sentient program is by definition restricted enough to be amenable to analysis. I don’t see any a priori reason to believe this, though.
3) We can suggest that maybe we are provable-about by construction: the god or intelligence created us in such a way that they can prove our unfaithfulness or insincerity, that yes there are beings which the god couldn’t be sure about, but tough luck – we aren’t them and the god knows that. (This is similar to Eliezer’s defense of proofs for FAIness – ‘yeah, we can’t prove friendliness over arbitrary AIs, but we’re not interested in creating arbitrary AIs.’) This doesn’t work for the Newcomb paradox, but could for Augustine’s god.

* And not just extremely high confidence

Perhaps you can clarify. Do you, now, believe as you earlier wrote, that:

“All conservative variants of Christianity teach, in one way or another, that your eternal fate depends on your state in the last moment of your life. If you live a nearly-flawless Christian life, but have a sinful thought ten minutes before dying and the priest has already left, you go to Hell.”

Try to answer yes or no.

No, I do not believe that entire sentence. Nor did I originally write that. Eliezer removed some qualifications that I had originally written to that sentence, perhaps because he thought them too wordy, and asked me to check if it was okay. I said that I thought this simplified version would cause people to raise irrelevant objections about purgatory and protestantism, but to go ahead with it anyway.

I repent of ever writing the original post. The point I wanted to make was not worth the time we have spent on it, even supposing that it got across.

Let us suppose that God decides who goes to heaven and who goes to hell using the following rule: you go to heaven if and only if you have never been rational with respect to heaven or hell, otherwise you go to hell.

Obviously, a rational agent always loses this game, and non-rational agents always win.

I think that what scenarios like this show is that you can’t say just “rationality always wins”, or more carefully, “no strategy has a better expected outcome than rationality”. Instead, you have to say that “no rationally chosen strategy has a better expected outcome than rationality”, or “no strategy has a better expected outcome than rationality unless the game includes an a priori penalty against rationality”.

This is consistent with the simple scenario above and Phil’s scenario, both of which are forms of “you lose if you act rationally”, but there is still a meaningful sense in which rationality always wins. Rationality can be beaten, but only when the game is rigged beforehand so that rationality cannot win.

- Intuitively insightful fools. Agents with no conscious attempt at rationality, but possessing an intuitive, meta-level understanding of the behaviors of other agents in their social environment, sufficient to work within the environment to obtain more optimal results.

Do you meant to suggest here that rationality requires consciousness or that an intuitive process that was provably optimal in all circumstances would not be a rational process? It sounds like it. If so, this seems problematic, as I hope the following thought experiment shows.

Consider two people whose brains are working on a complex problem (that doesn’t require and isn’t aided by consciousness of any form), with the only relevant difference between what the 2 brains are doing being that one has conscious awareness of (some of) what it is doing as it works on the problem. (If you think this is impossible in principle, please say why.) If they both reliably solve the problem optimally — there is no other strategy of any kind that is superior to either one — via an analogous sequence of steps and calculations, on what basis can you say that one is rational and the other is not?

I don’t think intuition ever gets this reliable, and so it is not rational for that reason, but if it could be so reliable (which seems possible in principle for domains that don’t involve consciousness [e.g., anything a computer could do optimally], although even then, perhaps consciousness could be modeled ‘intuitively’ in a non-conscious manner), then it would be a form of rationality.

What would you call a state in which you have internalized the principles and rules for a domain of problems that you can now solve instantly without any conscious effort whatsoever? It sounds like a form of intuition to me. And yet, it’s hard to say that the process was a form of rationality for the entire time that it required effort and was conscious, but that after you became so skilled at the intellectual tasks that they just began to happen automatically and effortlessly, the process suddenly became non-rational.

sternhammer:Phil::Phil:Eliezer

Phil Goetz,
Lots of people responded before your hysterical outburst. I would have, too, if sternhammer hadn’t hijacked the thread. Eliezer, in the very first comment, lays out his position quite clearly. In particular, he hints that he does believe that “A rational agent should get the maximum payoff in the PD.” The absolute max is impossible from symmetry, but he has said elsewhere on OB that two transparent rational agents cooperate in a one-shot PD.

You seem to be upset that he didn’t address the meaning of the slogan “rational agents should win.” This is stupid to argue about. I endorse Nick Tarleton’s version.

I disagreed with one particular sentence you wrote, quoting it several times. At no point were you able to admit that you were wrong about that. I think you are hinting that you now know it is wrong. But I am not sure. Your post seems very oblique.

Perhaps you can clarify. Do you, now, believe as you earlier wrote, that:

“All conservative variants of Christianity teach, in one way or another, that your eternal fate depends on your state in the last moment of your life. If you live a nearly-flawless Christian life, but have a sinful thought ten minutes before dying and the priest has already left, you go to Hell.”

Try to answer yes or no.

If no, then we have no argument.

I have read your last paragraph many times without understanding what you are saying. Please edit it for clarity. Are you talking about something I have claimed or something you have claimed?

And when you reject my proposed test as biased, despite my offering you the exact language I would use and your alleging no bias in that, perhaps you would consider offering a test of your own.

But not offering a test, and not offering supporting evidence that can be checked, would be very consistent with an effort to avoid detection of errors in your claims. I can’t read your mind. But I spent years as a lawyer, and when a witness acts like that, it speaks louder than anything he says.

No, I don’t want you bothering my ex-pastors. And I know perfectly well that you can get different answers from them depending on how you ask the question. I will not bet money on their rationality and logical consistence.

I already wrote, “The second part, that you go to heaven if you sincerely repent (and meet other conditions that may include confession, baptism, etc.) before dying, is taught by almost every Christian church, conservative, Catholic, Protestant, or otherwise. (The major exception is forms of predestination that teach that you can never know whether you are saved or not.) That is the only part that matters for this discussion.”

I also explained some of the reasons behind my opinions on particular Catholic beliefs. And I referred, twice, to the episode of the thief on the cross as the explanation of why all Christians believe you can be saved at the end of your life regardless of how you have lived your life before that.

You seem unaware that what you are claiming, in light of what I have said, is that most Christian churches teach that you can still go to Hell even after you have sincerely repented. You keep repeating my first statement and ignoring all of my later clarifications and qualifications. You keep asking for responses I have already made. I can only conclude you have written a long impassioned response to me without reading what I said.