Yeah... 3 felt like a bit of a side note that Eliezer inserted becuase it fit with 1 and 2 in his initial thoughts, but he explained it less comprehensively.
His social institutions argument seems to at least somewhat rest on problems from inherited problems from older versions, as well as deliberately planning things around the problems.
Other than that, you make excellent points regarding a few areas where Eliezer seems to lack sufficient justification to make assertions about meta-rationality, though it ocassionally feels like 'that is the topic of another series' rather than something that he should nesscarily have adressed there.
EY does keep quoting " a tautology that for every loaf of bread bought there must be a loaf of bread sold, and therefore supply is always and everywhere equal to demand", even though it doesn't demontrate all markets are efficient, and he doesn't believe that all markets are efficient. Why? Is this some kind of standing joke amongst economists that the rest of us are not in on?
1. E.Y. has a belief, unsupported by empirical evidence, that he's a member of an "elect" better-able to divine the truth of the world than the average Joe.2. E.Y. comes from a Rabbinical line, all of whom had a belief set similar to (1).3a. E.Y. attributes HIS membership in the elect to _mind tools_, and various commitments to something he calls "rationality."3b. E.Y.'s Rabinnical forbears presumably attributed THEIR membership in the elect to being the chosen people, and their course of Talmudic studies.4. E.Y. rejects his forbears' claim to being in the elect, because they've not studied the right things; yet his Rabinnical forbears would likely have a similar view of E.Y.5a. Given E.Y. hypothesis of ascendance through rationality training and meta-commitments isn't based on empirical evidence, and reasonable priors, it's at least as likely both the belief, and any sustaining 'evidence' for it, are based on genetic propensities mildly mediated by culture, and not rationality training or anything of the sort.5b. The fact E.Y. comes from a Rabbinical line with similar beliefs, in broad outline, supports this hypothesis.
The doctor is trying to win in a broken system, which often induces him to give inaccurate advise. So you can be justified in disagreeing with him, if you think your incentives are less broken.
When you write "Similarly, official medical advisors tend to advise medical treatment too often relative to doing nothing", you seem to be saying that if a doctor says you should get medical treatment, then your confidence in this claim should be less (ceteris paribus) than what is expressed by the doctor's statement. But doesn't this violate Aumann's theorem? How do you reconcile this? Are you saying that the doctor has a high chance of being dishonest because of his incentives? Is that enough to explain the disagreement?
To put it another way, a doctor seems to be a person rather than an institution, so saying that you should disagree with a doctor does seem to "imply that it is actually reasonable to disagree with [someone] about [something]."
Would I be accurate to summarize your objection as, "You can expect to outperform the experts with respect to your own life, because you have better incentives to be right; but you don't have better incentives with respect to questions not about your own life"?
You are right that Eliezer's statement was more nuanced.
I do see in the rationality community very often the meme of perfectly efficient markets and my comment was intended as an antidote to that, as for example if people took your statement literally.
> competitive speculative financial markets, where it is kind of crazy to expect your beliefs to be much more accurate than are market prices.
It is very common in the rationality world to overstate the efficient markets hypothesis. I think it is important to understand its limits.
Efficient markets depend on the ability of smarter players to use arbitrage to correct wrong prices. As an example, if IBM's versus MSFT's prospects are worse in the short term than market thinks, one can easily 'short' IBM and go 'long' MSFT. So you should not expect a naive retail investor to be able to win by exploiting relative mispricings in large, highly liquid stocks.
But arbitrage is often not possible and so prices are often far from rational values. For an excellent academic discussion see "The Limits of Arbitrage" Andrei Shleifer; Robert W. VishnyThe Journal of Finance, Vol. 52, No. 1. (Mar., 1997), pp. 35-55, readily available online.
Some areas where arbitrage fails
1. Real estate and housing markets because shorting is not possible and there are too many properties and not enough experts. Note that a high proportion of all investable wealth is in the form of real estate.
2. Small illiquid stocks because shorting is too expensive and risky (stocks borrowed to put on the short position can be taken away, usually at the worst possible moment). Also because of high trading costs, especially for large fund managers. Note that the vast majority of stocks are small stocks.
3. Long term mispricings. Because financial intermediaries are judged by short term performance, long term mispricings are hard to fix. Mispricings can get worse in the short term and the fund manager who bets against them can get fired. Many did in the lead up to the dot.com crash.
So markets as a whole can go far from fundamental values and the market cannot correct it in the short term.
Keynes put it roughly this way: markets can stay crazy longer than you can stay solvent.
I am not suggesting it is easy to beat the market but one should not overstate the rationality of financial markets.
"I should expect a priori to be below average at half of things, and be 50% likely to be of below average talent overall"
Perhaps if these 'things' are things that require knowledge. But that assumes that someone who is above average in IQ would not attempt to learn anything about a subject- and be able to catch up faster than most- before engaging on it. Obviously this is the case for most Hollyweird celebrities and TV scientists. But I would hope not, for Yudkowsky.
My recollection is the opposite: most ideologies identify the True and the Good. (Don't get me wrong. In a dispute between the True and the Good, my conscious values tell me to side with the True. But their nonidentity is inconvenient - not convenient - for ideologies.)
Yeah... 3 felt like a bit of a side note that Eliezer inserted becuase it fit with 1 and 2 in his initial thoughts, but he explained it less comprehensively.
His social institutions argument seems to at least somewhat rest on problems from inherited problems from older versions, as well as deliberately planning things around the problems.
Other than that, you make excellent points regarding a few areas where Eliezer seems to lack sufficient justification to make assertions about meta-rationality, though it ocassionally feels like 'that is the topic of another series' rather than something that he should nesscarily have adressed there.
EY does keep quoting " a tautology that for every loaf of bread bought there must be a loaf of bread sold, and therefore supply is always and everywhere equal to demand", even though it doesn't demontrate all markets are efficient, and he doesn't believe that all markets are efficient. Why? Is this some kind of standing joke amongst economists that the rest of us are not in on?
1. E.Y. has a belief, unsupported by empirical evidence, that he's a member of an "elect" better-able to divine the truth of the world than the average Joe.2. E.Y. comes from a Rabbinical line, all of whom had a belief set similar to (1).3a. E.Y. attributes HIS membership in the elect to _mind tools_, and various commitments to something he calls "rationality."3b. E.Y.'s Rabinnical forbears presumably attributed THEIR membership in the elect to being the chosen people, and their course of Talmudic studies.4. E.Y. rejects his forbears' claim to being in the elect, because they've not studied the right things; yet his Rabinnical forbears would likely have a similar view of E.Y.5a. Given E.Y. hypothesis of ascendance through rationality training and meta-commitments isn't based on empirical evidence, and reasonable priors, it's at least as likely both the belief, and any sustaining 'evidence' for it, are based on genetic propensities mildly mediated by culture, and not rationality training or anything of the sort.5b. The fact E.Y. comes from a Rabbinical line with similar beliefs, in broad outline, supports this hypothesis.
Appreciated—I took the statement literally and benefited from your caveating of it.
Truth != Good. If you can't distinguish them, it's a failure of *your* imagination.
Close. If you don't obviously have better incentives, I'll wonder what reason you have to say your opinion is more reliable.
The doctor is trying to win in a broken system, which often induces him to give inaccurate advise. So you can be justified in disagreeing with him, if you think your incentives are less broken.
When you write "Similarly, official medical advisors tend to advise medical treatment too often relative to doing nothing", you seem to be saying that if a doctor says you should get medical treatment, then your confidence in this claim should be less (ceteris paribus) than what is expressed by the doctor's statement. But doesn't this violate Aumann's theorem? How do you reconcile this? Are you saying that the doctor has a high chance of being dishonest because of his incentives? Is that enough to explain the disagreement?
To put it another way, a doctor seems to be a person rather than an institution, so saying that you should disagree with a doctor does seem to "imply that it is actually reasonable to disagree with [someone] about [something]."
Would I be accurate to summarize your objection as, "You can expect to outperform the experts with respect to your own life, because you have better incentives to be right; but you don't have better incentives with respect to questions not about your own life"?
Yeah, right.
You are right that Eliezer's statement was more nuanced.
I do see in the rationality community very often the meme of perfectly efficient markets and my comment was intended as an antidote to that, as for example if people took your statement literally.
Probably a misunderstanding on your part regarding what the word 'ideology' means.
Eliezer doesn't overstate this at all. And my short comment couldn't (and shouldn't) make so many disclaimers.
> competitive speculative financial markets, where it is kind of crazy to expect your beliefs to be much more accurate than are market prices.
It is very common in the rationality world to overstate the efficient markets hypothesis. I think it is important to understand its limits.
Efficient markets depend on the ability of smarter players to use arbitrage to correct wrong prices. As an example, if IBM's versus MSFT's prospects are worse in the short term than market thinks, one can easily 'short' IBM and go 'long' MSFT. So you should not expect a naive retail investor to be able to win by exploiting relative mispricings in large, highly liquid stocks.
But arbitrage is often not possible and so prices are often far from rational values. For an excellent academic discussion see "The Limits of Arbitrage" Andrei Shleifer; Robert W. VishnyThe Journal of Finance, Vol. 52, No. 1. (Mar., 1997), pp. 35-55, readily available online.
Some areas where arbitrage fails
1. Real estate and housing markets because shorting is not possible and there are too many properties and not enough experts. Note that a high proportion of all investable wealth is in the form of real estate.
2. Small illiquid stocks because shorting is too expensive and risky (stocks borrowed to put on the short position can be taken away, usually at the worst possible moment). Also because of high trading costs, especially for large fund managers. Note that the vast majority of stocks are small stocks.
3. Long term mispricings. Because financial intermediaries are judged by short term performance, long term mispricings are hard to fix. Mispricings can get worse in the short term and the fund manager who bets against them can get fired. Many did in the lead up to the dot.com crash.
So markets as a whole can go far from fundamental values and the market cannot correct it in the short term.
Keynes put it roughly this way: markets can stay crazy longer than you can stay solvent.
I am not suggesting it is easy to beat the market but one should not overstate the rationality of financial markets.
"I should expect a priori to be below average at half of things, and be 50% likely to be of below average talent overall"
Perhaps if these 'things' are things that require knowledge. But that assumes that someone who is above average in IQ would not attempt to learn anything about a subject- and be able to catch up faster than most- before engaging on it. Obviously this is the case for most Hollyweird celebrities and TV scientists. But I would hope not, for Yudkowsky.
My recollection is the opposite: most ideologies identify the True and the Good. (Don't get me wrong. In a dispute between the True and the Good, my conscious values tell me to side with the True. But their nonidentity is inconvenient - not convenient - for ideologies.)