Irrational Investment Disagreement
The Spring Journal of Economic Perspectives reviews how many investment puzzles can be explained by irrational disagreement:
One should not be able to forecast a stock’s return with anything other than … riskiness … Yet … a large catalog of variables with no apparent connection to risk have been shown to forecast stock returns, …. stocks that have had unusually high past returns or good earnings news to continue to deliver relatively strong returns over the subsequent six to twelve months … "glamour" stocks with high ratios of market value to earnings, cashflows or book value to deliver weak returns over the subsequent several years … many of the most interesting patterns in prices and returns are tightly linked to movements in volume …
We … argue in favor of… "disagreement" models. … encompassing … the following underlying mechanisms: i) gradual information flow; ii) limited attention; and iii) heterogeneous priors. … this class of models is at its heart about the importance of differences in the beliefs of investors. …
Gradual information flow by itself can be entirely consistent with a rational model … What is also required… is that, … investors do not fully take into account the fact that they may be at an informational disadvantage, …
limited attention needs to be combined with the assumption that … when trading with others, they do not adjust for the fact that they are basing their valuations on only a subset of the relevant information. … one needs to combine heterogeneous priors with an assumption that the investors do not fully update their beliefs based on each other.
Gradual information flow makes intuitive sense to me even for fully rational investors who are aware that they are at an informational disadvantage.
The investors who are at an informational disadvantage are more likely to be long term holders of the securities that they purchase than market participants as a whole. Because of that, it's rational for them to model many of the short term events which cause major (but transitory) market swings as if they were noise. This means that their process for updating their assumptions has to heavily discount the information content of short term market movements because they don't have the understanding of short term movements necessary to recognize their causes (and therefore whether they reflect long term changes in fundamentals).
The main testable prediction from the way that I'm looking at this is that investors who know they are at an informational disadvantage (at least in their understanding of short term movements) should trade less often -- either by owning index funds or by trading only on long term fundamental changes where they don't consider themselves at an informational disadvantage.
One reason (I'd say) for irrationality in markets is that the information flow also includes what other players in the market are doing. Investors are concerned that others have more information than they do; hence, when the market goes down (up), the investor infers that others have more information, and then sells (buys) accordingly. Also, investors suffer mightily from recency bias, weighting recent information more heavily than what is already known.