It seems to me that self-identified smart people are biased towards complex or counter-intuitive answers to problems. The reason is simple: complex or counter-intuitive answers allow one to show off intelligence. So let’s call this bias “show off bias.”
I think the bias is value-adding bias, and showing off is just a secondary matter.
As H.L. Mencken wisely observed, every complex problem has an answer that is simple, plausible, and wrong. We can pretty much count on ordinary minds not only to think of these solutions, but to offer them. So what is the genius to do but "specialize" in solutions that others do not think of? That's what makes genius worth having. Yes, the need to specialize in things other don't think of is a bias - no matter how smart you are, if you have a hammer, things tend to look like nails - but I wouldn't attribute it to something as small-minded as showing off, as it is serves a wholly rational purpose of using rare resources for rare purposes.
I disagree in part (for there will alway be showoffs). Complexity in explanations are at times a way to "blanket" a problem and fill all the nooks and crannies with a theory before simplifying that theory via a common explanation.
I liken it to writing and to quote Pascal ... "My Reverend Fathers, my letters have not usually followed so closely, nor been so long. The small amount of time that I have is the cause of both. I would not have made this so long except that I do not have the leisure to make it shorter."
Perhaps the reason for complexity is lack of time to make it simpler?
What you say is true if indexes were a collection of stocks picked by really smart people who evaluate what businesses have wonderful prospects etc.
In reality all they look to do is follow a simple algorithm that needs no smarts.
You just start off with the CAPM style reasoning which leads you to conclude that the optimal portfolio to hold (from the point of view of minimum variance 'risk' for a particular expected return), is simply buying the 'market'. This in other words means that you should put some fraction of your money in all stocks and other risky assets in the world, with the weightage being proportional to the market capitalization of the stock or financial instrument and the remaining money in a "risk-free" instrument like US treasury bills.
All index funds do is to approximate at least a country's market, by picking a sufficiently large number of stocks with high market cap. And thereby avoiding the hassle for the individual investor to actually go and buy 500-10000 odd stocks in a market cap weighted fashion.
So for all practical purposes, the index is fairly "zero knowledge", since the algorithm is quite precise and public, which is "create a large basket of stocks that approximates the whole market to the extent possible".
Index funds are not that much different, fundamentally speaking, than actively managed funds. After all, someone has to choose what stocks get put in the index; an index fund simply copies the algorithm used by the makers of the index. It is not a "zero intelligence" strategy of picking stocks at random.
In the case of the S&P 500, the stocks in the index are chosen by a committee of humans, using their own judgment. (They attempt to select stocks that form a representative sample of the economy of the United States as a whole.) If index funds that follow the S&P 500 beat actively managed funds, then wouldn't that make the members of that committee abnormally good at stock picking? )
Interestingly, because index funds exist, whether or not a stock is listed in an index affects the price of the stock. If a stock is taken off the S&P 500, all the index funds that follow the S&P 500 have to sell it, depressing the price of the stock. Similarly, a stock that gets added to an index experiences a surge of demand, causing its price to rise.
Robin wrote: "I've noticed that a mark of academics is that they expect the answers of important questions to be more complex and subtler than others expect. It would be great to collect data to see who is more biased here, academics or non-academics."
Important questions don't have answers. They have increasingly accurate and complex approximations. Academics are more willing to consider more complex approximations. That doesn't mean they have a lower opinion of the simple answers.
Another explanation for this phenomenon is a complexity is fun bias. If you realize how good tit-for-tat is, you'll stop searching, which means the fun is over.
Finding subtle differences is fun because of the pleasure of grokking. I always get a kick out of understanding something on a deeper level than it's being explained to me.
As for stocks: People also have come up with complicated strategies which are supposed to help them make money while gambling. If there is the potential for easy money with very little work, people will overanalyze.
The show-off bias is what comes in at the end when you share your discoveries, even though they are useless.
So how come we don't see counter-showoff bias, where people in communities of people who adopt, e.g., atheism, adopt things like theism to be contextually counterintuitive?
Corollary: Because a simple answer to a complex problem is always counter-intuitive, the bias of a self-identified smart person toward complex and counter-intuitive answers is not a detractor.
Therefore, smart people should only be given complex problems. Let those with less intelligence (or at least more humility) handle the simple ones, because they will perform more accurately (i.e. with less bias).
Therefore, smart people should have assistants. Or, to quote Don Lapre, "never do anything yourself when someone else is ready, willing, and able to do it for eight bucks an hour". (Actually, in my experience, you can't expect competence for less than twelve.)
Index funds struggle to fully catch on because they are a bit of a catch-22.
Index funds work great if enough market makers are doing their job correctly for the market to function efficiently (i.e. people not using index funds).
The second everyone uses index funds the second the market complete stops being efficient. By definition index funds cannot filter or select the best companies. All they do is piggy back on others' work.
The problem with your hypothesis is that it is inconceivable that nobody who entered subsequent competitions didn't know that Tit-for-Tat was a candidate for optimal strategy. So what was the point of subsequent competitions? To test the hypothesis that Tit-for-Tat is the optimal strategy. It's not clear that those who entered the first competition didn't know this as well.
Everyone on Wall Street has an incentive to say bad things about index funds. This is another reason index funds aren't popular, which might be difficult to disentangle from show-off bias.
"i don't refer to "self-identified" again because it's awkward and made the sentences harder to read."
did it make the sentences harder to understand or just degrade the prose?
self-identified smart people often sacrifice clarity and precision for style. instead of making ideas easy to comprehend they present them in the most impressive, stylized way possible. i think this is far and away the most common and insidious way the "Show Off Bias" manifests itself.
Yes, it is an important distinction. You can be smart without self-identifying as smart. And you can stupid and still think you are smart. Show-off bias should only apply to people who think they are smart. Whether they actually are smart is less important.
I don't refer to "self-identified" again because it's awkward and made the sentences harder to read.
Show-Off Bias
I think the bias is value-adding bias, and showing off is just a secondary matter.
As H.L. Mencken wisely observed, every complex problem has an answer that is simple, plausible, and wrong. We can pretty much count on ordinary minds not only to think of these solutions, but to offer them. So what is the genius to do but "specialize" in solutions that others do not think of? That's what makes genius worth having. Yes, the need to specialize in things other don't think of is a bias - no matter how smart you are, if you have a hammer, things tend to look like nails - but I wouldn't attribute it to something as small-minded as showing off, as it is serves a wholly rational purpose of using rare resources for rare purposes.
I disagree in part (for there will alway be showoffs). Complexity in explanations are at times a way to "blanket" a problem and fill all the nooks and crannies with a theory before simplifying that theory via a common explanation.
I liken it to writing and to quote Pascal ... "My Reverend Fathers, my letters have not usually followed so closely, nor been so long. The small amount of time that I have is the cause of both. I would not have made this so long except that I do not have the leisure to make it shorter."
Perhaps the reason for complexity is lack of time to make it simpler?
Doug:
What you say is true if indexes were a collection of stocks picked by really smart people who evaluate what businesses have wonderful prospects etc.
In reality all they look to do is follow a simple algorithm that needs no smarts.
You just start off with the CAPM style reasoning which leads you to conclude that the optimal portfolio to hold (from the point of view of minimum variance 'risk' for a particular expected return), is simply buying the 'market'. This in other words means that you should put some fraction of your money in all stocks and other risky assets in the world, with the weightage being proportional to the market capitalization of the stock or financial instrument and the remaining money in a "risk-free" instrument like US treasury bills.
All index funds do is to approximate at least a country's market, by picking a sufficiently large number of stocks with high market cap. And thereby avoiding the hassle for the individual investor to actually go and buy 500-10000 odd stocks in a market cap weighted fashion.
So for all practical purposes, the index is fairly "zero knowledge", since the algorithm is quite precise and public, which is "create a large basket of stocks that approximates the whole market to the extent possible".
Philip, yes more precisely academics expect the most interesting or useful answers to be more complex and subtle.
Index funds are not that much different, fundamentally speaking, than actively managed funds. After all, someone has to choose what stocks get put in the index; an index fund simply copies the algorithm used by the makers of the index. It is not a "zero intelligence" strategy of picking stocks at random.
In the case of the S&P 500, the stocks in the index are chosen by a committee of humans, using their own judgment. (They attempt to select stocks that form a representative sample of the economy of the United States as a whole.) If index funds that follow the S&P 500 beat actively managed funds, then wouldn't that make the members of that committee abnormally good at stock picking? )
Interestingly, because index funds exist, whether or not a stock is listed in an index affects the price of the stock. If a stock is taken off the S&P 500, all the index funds that follow the S&P 500 have to sell it, depressing the price of the stock. Similarly, a stock that gets added to an index experiences a surge of demand, causing its price to rise.
Robin wrote: "I've noticed that a mark of academics is that they expect the answers of important questions to be more complex and subtler than others expect. It would be great to collect data to see who is more biased here, academics or non-academics."
Important questions don't have answers. They have increasingly accurate and complex approximations. Academics are more willing to consider more complex approximations. That doesn't mean they have a lower opinion of the simple answers.
(Though it wouldn't surprise me if they did.)
Another explanation for this phenomenon is a complexity is fun bias. If you realize how good tit-for-tat is, you'll stop searching, which means the fun is over.
Finding subtle differences is fun because of the pleasure of grokking. I always get a kick out of understanding something on a deeper level than it's being explained to me.
As for stocks: People also have come up with complicated strategies which are supposed to help them make money while gambling. If there is the potential for easy money with very little work, people will overanalyze.
The show-off bias is what comes in at the end when you share your discoveries, even though they are useless.
So how come we don't see counter-showoff bias, where people in communities of people who adopt, e.g., atheism, adopt things like theism to be contextually counterintuitive?
Not always.
See this Ted Talk by Murray Gell-Man about how he looks for "simplicity" as an indication of truth:
http://www.youtube.com/watc...
Corollary: Because a simple answer to a complex problem is always counter-intuitive, the bias of a self-identified smart person toward complex and counter-intuitive answers is not a detractor.
Therefore, smart people should only be given complex problems. Let those with less intelligence (or at least more humility) handle the simple ones, because they will perform more accurately (i.e. with less bias).
Therefore, smart people should have assistants. Or, to quote Don Lapre, "never do anything yourself when someone else is ready, willing, and able to do it for eight bucks an hour". (Actually, in my experience, you can't expect competence for less than twelve.)
Index funds struggle to fully catch on because they are a bit of a catch-22.
Index funds work great if enough market makers are doing their job correctly for the market to function efficiently (i.e. people not using index funds).
The second everyone uses index funds the second the market complete stops being efficient. By definition index funds cannot filter or select the best companies. All they do is piggy back on others' work.
This hypothesis seems... hm...
...true :-)
The problem with your hypothesis is that it is inconceivable that nobody who entered subsequent competitions didn't know that Tit-for-Tat was a candidate for optimal strategy. So what was the point of subsequent competitions? To test the hypothesis that Tit-for-Tat is the optimal strategy. It's not clear that those who entered the first competition didn't know this as well.
Everyone on Wall Street has an incentive to say bad things about index funds. This is another reason index funds aren't popular, which might be difficult to disentangle from show-off bias.
"i don't refer to "self-identified" again because it's awkward and made the sentences harder to read."
did it make the sentences harder to understand or just degrade the prose?
self-identified smart people often sacrifice clarity and precision for style. instead of making ideas easy to comprehend they present them in the most impressive, stylized way possible. i think this is far and away the most common and insidious way the "Show Off Bias" manifests itself.
Yes, it is an important distinction. You can be smart without self-identifying as smart. And you can stupid and still think you are smart. Show-off bias should only apply to people who think they are smart. Whether they actually are smart is less important.
I don't refer to "self-identified" again because it's awkward and made the sentences harder to read.