Open Policy Evaluation

Hypocrisy is a tribute vice pays to virtue. La Rochefoucauld, Maximes

In some areas of life, you need connections to do anything. Invitations to parties, jobs, housing, purchases, business deals, etc. are all gained via private personal connections. In other areas of life, in contrast, invitations are made open to everyone. Posted for all to see are openings for jobs, housing, products to buy, business investment, calls for proposals for contracts and grants, etc. The connection-only world is often suspected of nepotism and corruption, and “reforms” often take the form of requiring openings to be posted so that anyone can apply.

In academia, we post openings for jobs, school attendance, conference attendance, journal publications, and grant applications for all to see. Even though most people know that you’ll actually need personal connections to have much of a chance for many of these things. People seems to want to appear willing to consider an application from anyone. They allow some invitation-only conferences, talk series, etc., but usually insist that such things are incidental, not central to their profession.

This preference for at least an appearance of openness suggests a general strategy of reform: find things that are now only gained via personal connections, and create an alternate open process whereby anyone can officially apply. In this post, I apply this idea to: policy proposals.

Imagine that you have a proposal for a better policy, to be used by governments, businesses, or other organizations. How can you get people to listen to your proposal, and perhaps endorse it or apply it? You might try to use personal connections to get an audience with someone at a government agency, political interest group, think tank, foundation, or business. But that’s stuck in the private connection world. You might wait for an agency or foundation to put out an open call for proposals, seeking a solution to exactly the problem your proposal solves. But for any one proposal idea, you might wait a very long time.

You might submit an article to an open conference or journal, or submit a book to a publisher. But if they accept your submission, that mostly won’t be an endorsement of whether your proposal is good policy by some metric. Publishers are mostly looking at other criteria, such as whether you have an impressive study using difficult methods, or whether you have a book thesis and writing style that will attract many readers.

So I propose that we consider creating an open process for submitting policy proposals to be evaluated, in the hope of gaining some level of endorsement and perhaps further action. This process won’t judge your submission on wit, popularity, impressiveness, or analytical rigor. Their key question is: is this promising as a policy proposal to actually adopt, for the purpose of making a better world? If they endorse your proposal, then other actors can use that as a quality signal regarding what policy proposals to consider.

Of course how you judge a policy proposal depends on your values. So there might be different open policy evaluators (OPE) based on different sets of values. Each OPE needs to have some consistent standards by which they evaluate proposals. For example, economists might ask whether a proposal improves economic efficiency, libertarians might ask if it increases liberty, and progressives might ask whether it reduces inequality.

Should the evaluation of a proposal consider whether there’s a snowball chance in hell of a proposal being actually adopted, or even officially considered? That is, whether it is in the “Overton window”? Should they consider whether you have so far gained sufficient celebrity endorsements to make people pay attention to your proposal? Well, those are choices of evaluation criteria. I’m personally more interested in evaluating proposals regardless of who has supported them, and regardless of their near-term political feasibility. Like how academics say we do today with journal article submissions. But that’s just me.

An OPE seems valid and useful as long as its actual choices of which policies it endorses match its declared evaluation criteria. Then it can serve as a useful filter, between people with innovative policy ideas and policy customers seeking useful ideas to consider and perhaps implement. If you can find OPEs who share your evaluation criteria, you can consider the policies they endorse. And of course if we ever end up having many of them, you could focus first on the most prestigious ones.

Ideally an OPE would have funding from some source to pay for its evaluations. But I could also imagine applicants having to pay a fee to have their proposals considered.