17 Comments

I'm thinking of something much like review today of journal articles and grant proposals.

Expand full comment

Different OPE could have different standards of evaluation, some focusing on close experts, and others on more general experts.

Expand full comment

People already generate policy proposals and write detailed descriptions of them. I can't see how OPEs would make that less likely.

Expand full comment

What is the incentive for submitter to go the expense and trouble of generating submissions?

Many people have policy ideas they think would be beneficial for society. (Usually they're either mistaken about that, or the ideas are well-known but opposed by entrenched interests).

But generating a formal proposal is a lot of work, while the personal benefit to the submitter, even if the policy is implemented, is likely to be tiny (if spread across all of society).

The moreso if the submitter is expected to fund the evaluation.

Expand full comment

I think Robin's intent is that evaluations are by subject matter experts, not via "open voting'. Selling votes at $1 each is a reasonable way of measuring depth of approval by the voters, but if voters aren't subject matter experts their approval may have little to do with the intrinsic quality of the proposal.

And if votes actually influence policy, then they're an invitation to corruption at $1 each - the benefit of the policy to particular voters may exceed $1/vote, even if the benefit to society is negative.

On the other hand, subject matter experts are notorious for rejecting ideas "not invented here" or that "have already been tried" (despite changes in critical details or circumstances).

Expand full comment

With Tyler already going for small scale moonshot philanthropy, what is stopping YOU, Robin, from "organizing" a draft OPE "council" for 2018 -- and request proposals to evaluate as well as people, professors and especially NON-professors, who agree to evaluate some subset of the highest quality of proposals submitted.

Of course, looking at 15 years of Weblog Awards, ended 2015, indicates a "work of love" won't be enough -- but we already know that.

Unpaid quality entrepreneurship ability is not so hard to get a little of, especially low cost quick brainstorming, but much harder for the admin/ executive work of getting something done, on time as per prior commitments.

In "open voting", a key issue should be to collect $1 or so from all who vote (voting often IS allowed!). Starting with easy mini-transactions should allow it to reach sustainable cost recovery fairly soon.

Expand full comment

Then I do not understand key parts of the system you have in mind. Some kind of platform for establishing structured communication between proposer and OPE is needed, right? Or for recording evaluations.

Or do you just intend for a OPE market platform where the evaluations are traded?

Expand full comment

That is the sort of policy proposal that might be evaluated, but it is not the process of evaluation I have in mind.

Expand full comment

No the OPE isn't the customer for the policy, it is an independent evaluator of policies.

Expand full comment

Would you say that LiquidFeedback goes into the direction of the system you have in mind?

https://en.wikipedia.org/wi...

Expand full comment

So, then, is the OPE expected to be part of the organization at which policy proposals are targeted?

And even so, wouldn't that naturally select against proposals that don't have any reasonable hope of implementation, since quality proposals that can't be implemented don't really have much utility (except, perhaps, in the very long term)?

Expand full comment

Quality proposals hopefully.

Expand full comment

What is it that is common to all OPEs? Is it that the submission process itself (and feedback) has been standardized?

It seems that the OPE is bringing to the table:- Subject matter expertise (potentially useful to the submitter as feedback to refine their proposal)- A quality stamp carrying the weight of the ODE's reputation (a kind of credentialing of the proposal)

Is there anything the applicants are offering the OPEs other than the price of evaluation?

Expand full comment

There are also public comments and hearings at the legislative level though it takes insiders to investigate, propose, and invite speakers, and commissions tasked with the same. It is what writing your congressman is all about. It is important to do in advance to allow data gathering, analysis, and review because once the time is ripe to act there is little time to delay. I wouldn't say it is always reactive since many of these are investigatory or considering wide alternatives to desired ends.

Expand full comment

Yes, we should be able to do better.

Expand full comment

You can publically comment on policies on the U.S. Federal Register when agencies are deciding on how to implement and ammend rules. If you can get some high status academics or stake holders to sign your comment, and you make actually logical arguments: especially those that imply their could be a lawsuit if the agency acts otherwise, then you have some chance of influence as an outsider since the agencies have an obligation to process the comments. Of course it would be the government agency doing the evaluation though, and this is not proactive, only reactive in terms of policy proposal.

Expand full comment