Discover more from Overcoming Bias
For econ topics where info is relevant, including key areas of mechanism design, and law & econ, we often make use of a key distinction: verifiable versus unverifiable info. For example, we might say that whether it rains in your city tomorrow is verifiable, but whether you feel discouraged tomorrow is not verifiable.
Verifiable info can much more easily be the basis of a contract or a legal decision. You can insure yourself against rain, but not discouragement, because insurance contracts can refer to the rain, and courts can enforce those contract terms. And as courts can also enforce bets about rain, prediction markets can incentivize accurate forecasts on rain. Without that, you have to resort to the sort of mechanisms I discussed in my last post.
Often, traffic police can officially pull over a car only if they have a verifiable reason to think some wrong has been done, but not if they just have a hunch. In the blockchain world, things that are directly visible on the blockchain are seen as verifiable, and thus can be included in smart contracts. However, blockchain folks struggle to make “oracles” that might allow other info to be verifiable, including most info that ordinary courts now consider to be verifiable.
Wikipedia is a powerful source of organized info, but only info that is pretty directly verifiable, via cites to other sources. The larger world of media and academia can say many more things, via its looser and more inclusive concepts of “verifiable”. Of course once something is said in those worlds, it can then be said on Wikipedia via citing those other sources.
I’m eager to reform many social institutions more in the direction of paying for results. But these efforts are limited by the kinds of results that can be verified, and thus become the basis of pay-for-results contracts. In mechanism design, it is well known that it is much easier to design mechanisms that get people to reveal and act on verifiable info. So the long term potential for dramatic institution gains may depend crucially on how much info can be made verifiable. The coming hypocralypse may result from the potential to make widely available info into verifiable info. More direct mind-reading tech might have a similar effect.
Given all this reliance on the concept of verifiability, it is worth noting that verifiability seems to be a social construct. Info exists in the universe, and the universe may even be made out of info, but this concept of verifiability seems to be more about when you can get people to agree on a piece of info. When you can reliably ask many difference sources and they will all confidently tell you the same answer, we tend to treat that as verifiable. (Verifiability is related to whether info is “common knowledge” or “common belief”, but the concepts don’t seem to be quite the same.)
It is a deep and difficult question what actually makes info verifiable. Sometimes when we ask the same question to many people, they will coordinate to tell us the answer that we or someone wants to hear, or will punish them for contradicting. But at other times when we ask many people the same question, it seems like their best strategy is just to look directly at the “truth” and report that. Perhaps because they find it too hard to coordinate, or because implicit threats are weak or ambiguous.
The question of what is verifiable opens an important meta question: how can can we verify claims of verifiability? For example, a totalitarian regime might well insist not only that everyone agree that the regime is fair and kind, a force for good, but that they agree that these facts are clear and verifiable. Most any community with a dogma may be tempted to claim not only that their dogma is true, but also that it is verifiable. This can allow such dogma to be the basis for settling contract disputes or other court rulings, such as re crimes of sedition or treason.
I don’t have a clear theory or hypothesis to offer here, but while this was in my head I wanted to highlight the importance of this topic, and its apparent openness to investigation. While I have no current plans to study this, it seems quite amenable to study now, at least by folks who understand enough of both game theory and a wide range of social phenomena.
Added 3Dec: Here is a recent paper on how easy mechanisms get when info is verifiable.
What Info Is Verifiable?
I have found my calling.
Agreed, it's a deep and difficult question.
I think that to really look for solutions the concept would probably need to be broken down more. Verification is a single word for what is likely a complex process involving various numerical methods combined with multiple complex social phenomena.
For example, the very concept of verification requires a reference to some authority or authorities; it's impossible to verify something unless you have something to verify it against. But the question of who decides what counts as a valid authority is very different from the question of whether some information matches the information approved by the authority; matching information to the source is likely much easier than agreeing who the valid authorities are.
And even if we assume we could all could agree what the valid authorities were (not realistic, but let's assume) - how do we then handle it when multiple recognized authorities disagree? This is crucial, since contradictions happen all the time even within science, to say nothing of the contrasting views of science, religion, and various other ideologies.
So at a bare minimum, I think determining what's verifiable probably requires answering at least 3 questions:
1. Which sources count as authorities?2. In cases of conflict, how much weight should be given to each of these sources?3. How well does the information to be verified match each of these sources?
Of course almost every single person will differ in their answers to these. And it's likely that the vast majority of people aren't even consciously aware of what their answers are, and will have varied and likely inconsistent answers for different information.
The value of game theory or similar advanced approaches is that they might be able to consider verifiability without having to break things down like this. The challenge, though, is that without more clarity on these, I'm not sure verifiability rises above being more than just an aggregation of opinion, however it may be performed. And opinion, even among highly experienced and aware people, can still be swayed to reflect agendas rather than facts; in fact, with enough media influence, opinion can probably be swayed significantly on almost any issue (as in the case of the authoritarian regime).
So without a deeper breakdown, I'm not sure verifiability can become more than a fancy opinion/propaganda meter.
Last note: Verifiability is not really a matter of saying something is or is not verified (black-or-white). For all but pure dogmatists it's necessarily a probability function, where we're trying to gain some measure of confidence that the information is correct (e.g. matches the authoritative sources). But I don't think this changes the minimum questions that need to be addressed to attempt to improve our understanding of it.