Discover more from Overcoming Bias
I’d like to try to make a point here that I’ve made before, but hopefully make it more clearly this time. My point is: trend tracking and policy analysis have little relevance for each other.
You can discuss education policy, or you can discuss education trends. You can discuss medical policy or you can discuss medical trends. You can discuss immigration policy, or you can discuss immigration trends. And you can discuss redistribution and inequality trends, or you can discuss redistribution and inequality policy. But in all of these cases, and many more, the trend and policy topics have little relevance for each other.
On trends, we collect a lot of data, usually on parameters that are relatively close to what we can easily measure, and also close to summary outcomes that we care about, like income, mortality, or employment. Many are interested in explaining past trends, and in forecasting future trends. Such trend tracking supports the familiar human need for news to discuss and fret about. And when a trend looks worrisome, that naturally leads people to want to discuss what oh what we might do about it.
On policy, we have lots of thoughtful theoretical analysis of policies, which try to judge which policies are better. And we have lots of relevant data analysis, that tries to distinguish relevant theories. Such analysis usually ends up identifying a few key parameters on which policy decisions should depend. But those tend to be abstract parameters, close to theoretical fundamentals. They usually have only a distant relation to the parameters which are tracked so eagerly as trends.
To repeat for emphasis: the easy to measure parameters where trends are most eagerly tracked are rarely close to the key theoretical parameters that determine which policies are best. They are in fact usually so far away that it is hard to judge the sign of the relation between them. This makes it unlikely that a change in one of these policies is a reasonable response to noticing some tracked-parameter trend.
For example which policies are best in medicine depends on key theoretical parameters like risk-aversion, asymmetric info on risks, meddling preferences, market power of hospitals, customer irrationality, and where learning happens, etc. But the trends we usually track are things like mortality, rates of new drug introduction, and amounts, fractions, and variance of spending. These later parameters are just not very relevant for inferring the former. People may find it fascinating to track trends in doctor salaries, cancer deaths, or how many are signed up for Obamacare. But those are pretty irrelevant to which policies are best.
As another example, debates on immigration refer to many relevant theoretical parameters, including meddling preferences, demand elasticity for low wage workers, and the intelligence, cultural norms, and cultural plasticity of immigrants. In contrast, trend trackers talk about trends in immigration, low-skill wages, wage inequality, labor share of income, voter participation, etc. Which might be fascinating topics, but they are just not very relevant for whether immigration is a good or bad idea. So it just doesn’t make sense to suggest changing immigration policy in response to noticing particular trends in these tracked parameters.
Alas, most people are a lot more interested in tracking trends than in analyzing policies. So well meaning people with smart things to say about policy often try to make their points seem more newsworthy by suggesting those policies as answers to the problems posed by troublesome trends. But, in doing so they usually mislead their audiences, and often themselves. Trends just aren’t very relevant for policy. If you want to talk policy, talk policy, and skip the trends.
Trends Rarely Inform Policy
Is not the reason for a policy to change a trend? If there are no trends in need of changing, what use is policy? Sure, arguing about the trend implications of policy might be hard and rife with many of the sociological facts that you often note. But even so, without trend implications, what use is policy? Another way of putting it might be that trend measurement is a way of making policy beliefs pay rent. Of course folks might enforce that rent payment badly, or with biases, or for status gains, but there is still a kernel of purpose behind it.
Another point you mention is that policies are changed rarely and so we should not expect trends to be much affected by policy changes, at least not after controlling for many other trend changers. But there is something normative in your use of "rarely." One might say that while the number of policy changes that have had suitably large trend impacts is low, the trend impact of the few trend-impacting policy changes was extremely high. I am not making such a claim but I could envision folks making such a claim with regard to e.g. civil rights, gender wage gap, measure of freedom for affected minorities, etc. If someone derived utils from the improvement in the trends of these topics, and the degree to which they derived utils was super huge, then even that small amount of trend-changing policy would have been vastly worth all the cost of noisy, unproductive trend measurement and trend debate on a host of many other less possibly efficacious policies.
Another way of reading your post is that people simply have lottery-like preferences when it comes to how they attach utils to policy-induced trend changes. Spend lots of small costs analyzing many trends; always analyze policy in terms of the trends it might impact (even if there is weak evidence that it can actually impact the trend); and for the few lightning-strike policies that really do change trends dramatically this way, celebrate them lots; and for the others, oh well. Just another scratch off ticket that, while it didn't pay, was fun to scratch.
Trends usually are used to persuade outsiders that action should be taken. But the particularly form of action is always what the insiders already favored. Even if the trends later change 180 degrees, it's still the same set.
Theoretical policy discussions are almost completely irrelevant except maybe 1 generation later.