The Accuracy of Authorities

“WHO treads a difficult line, & tends to be quite conservative in its recommendations to avoid putting out info that later proves to be incorrect. ‘You can’t be backtracking’ … because ‘then you lose complete credibility’.” (More)

There is something important to learn from this example. The best estimates of a maximally accurate source would be very frequently updated and follow a random walk, which implies a large amount of backtracking. And authoritative sources like WHO are often said to be our most accurate sources. Even so, such sources do not tend to act this way. They instead update their estimates rarely, and are especially reluctant to issue estimates that seem to backtrack. Why?

First, authoritative sources serve as a coordination point for the behavior of others, and it is easier to coordinate when estimates change less often. Second, authoritative sources need to signal that they have power; they influence others far more than others influence them. Both of these pressures push them toward making infrequent changes. Ideally only one change, from “we don’t know”, to “here is the answer”. But if so, why do they feel pressures to issue estimates more often than this?

First, sometimes there are big decisions that need to be made, and then authorities are called upon to issue estimates in time to help with those decisions. For example, WHO was often called upon to issue estimates to help with a rapidly changing covid epidemic.

Second, sometimes a big source of relevant info appears, and it seems obvious to all that it must be taken into account. For example, no matter how confident we were to win a battle, we should expect to get news about how that battle actually went, and update accordingly. In this case, the authority is more pressed to update its estimate, but also more forgiven for changing its estimate. So during covid, authorities were expected to update on changing case and death counts, and that didn’t count so much as “backtracking”.

Third, sometimes rivals compete for authority. And then sources might be compared regarding their accuracy track record. This would push them toward the frequently updated random walk scenario, which can degrade the appearance of authority for all such competitors. (The other two pressures to update more often may also degrade authority; e.g., WHO’s authority seems to have degraded during covid.)

Due to the first of these pressures, the need to inform decisions, authoritative sources prefer that dependent decisions be made infrequently and opaquely. Such as by central inflexible organizations, who decide by opaque political processes. E.g., masking, distancing, and vaccine policies for covid. There can thus form a natural alliance between central powers and authoritative sources.

Due to the second of these pressures, authoritative sources prefer a strong consensus on what are the big sources of info that force them to update. This pushes for making very simple, stable, and clear distinctions between “scientific” info sources, on which one must update, and “unscientific” sources, on which it is in considered inappropriate for authors to update. Those latter sources must be declared not just less informative, but un-informative, and slandered in enough ways so that few who aspire to authority are tempted to rely on them.

Due to the third of these pressures, authoritative sources will work hard to prevent challengers competing on track record accuracy. Authorities will issue vague estimates that are hard to compare, prevent the collection of data that would support comparisons, and accuse challengers of crimes (e.g., moral positions) to make them seem ineligible for authority. And other kinds of powers, who prefer a single authority source they can defer to in order to avoid responsibility for their decisions, will help to suppress such competitors.

This story seems to explain why ordinary people take backtracking as a sign of inaccuracy. They have a hidden motive to follow authorities, but give accuracy as their excuse for following such sources. This forces them to see backtracking as a general sign of inaccuracy.

This all seems to be bad news for efforts to gain credibility, funding, and legal permission for alternative estimate sources, such as those based on prediction markets or forecasting competitions. This helps explain why individual org managers are reluctant to support such alternate sources, and why larger polities create barriers to them, such as via censorship, professional licensing, and financial regulation.

This all points to another risk of our increasingly integrated world community of elites. They may form central sources of authoritative estimates, which coordinate with other authorities to suppress alternate sources. Previously, world wide competition made it easier to defy and challenge such estimate authorities.

Added: As pointed out by @TheZvi, a 4th pressure on authorities to update more often is to stay consistent with other authorities. This encourages authorities to coordinate to update together at the same time, by talking first behind the scenes.

Added 11Apr: Seem many comments on this over at Marginal Revolution.