Inspiring content. Thanks that you holds some great ideas about the good impacts of social innovations to our education. These are great things and information that we have to remember because it ideally helps us to stabilized our good learning skills.
"by now academics have accumulated a large pile of promising institution ideas, many of which have supporting theory, lab experiments, and even simplified field trials. In addition, academics have even larger literatures that measure and theorize about existing social institutions."
I know someone who's starting a new institution--is there a guide to this literature you can recommend? (If so, perhaps the creation of such a user-friendly guide would itself represent low-hanging fruit?)
Fair enough, but you provided no quantities (relative or absolute) in your post about compression ratios at each stage in the pipeline you described.
Here's a number: there have been (extremely conservatively) >1,000 well-structured RCTs of institutional alternatives that I know of personally that have been executed by large companies since 2000.
Here's another number: ~90% of these tested institutional alternatives failed to create statistically-significant improvement in the test group versus the control group with incremental benefits greater than incremental costs.
Unless you think that companies (in general, not just some specific company) are extremely bad at deciding what to test, and there is this incredible backlog of great institutional alternatives out there that nobody is willing to try, then why are they willing to try thousands of others ideas with a 90% failure rate?
Isn't a more realistic interpretation that coming up with new ideas that create improvements is incredibly hard, i.e., the real bottleneck?
To say something is a bottleneck is not to say that nothing gets through it. It is to say that is the most restrictive limiting point in the flow. I'm happy you know of some field trials of some things, but there are a great many other things that aren't being tried.
Sorry but you're wrong on the facts in part of this. The company that I started provides the technology used to execute field trials for many of the largest consumer companies in the world. There have been thousands of field trials of institutional alternatives run by large companies over the past 15 years that I know about personally on topics ranging from changing salesforce compensation / incentive structures to reorganizing jobs and roles.
As an aside, academic research has very rarely (note, not never) been a direct source of those ideas that create positive gains net of costs in the test group versus the randomized control group.
How can someone graduate from a modern MBA program without easily understanding that more accurate probabilities enable more effective decisions? I can more believe that they fear prediction markets will “run amok of existing power structures and interests.”
I'm curious what would ideas you (or others) would rate as the most promising social innovations are at (or near) the point of prediction markets the social innovation pathway?
In the corporate world, I would argue that the source prestige and funding related to adapting academic ideas to the complex corporate world changes at that step. The funding comes from the organizations who are piloting or implementing new processes or innovations not for research purposes, but for their own benefit. Prestige is derived from being seen as the expert in implementation - or at least, helping others implement.
Management consultants are constantly vying to find new ideas they can take to clients. Academically validated ideas are a potential rich source here. The challenge is driving adoption - a sales and marketing problem (as Doug mentions), and in some cases a design and engineering problem. McKinsey (plenty of money and prestige) is has discussed prediction markets in its publications, obviously with a goal to facilitate conversations with clients.
I find that in practice (I'm a management consultant), the robustness of academic support behind the effectiveness of a given methodology is less important to getting an idea opted versus:a) How subjectively believable is the idea. Obviously this is influenced by prior research and case-studies, but a key input is having someone senior grok itb) How feasible is the implementation is in practice? Is it be cheaply tried and tested or it it expensive and will it run amok of existing power structures and interests?c) The magnitude of benefit you might reap, if the idea worked in practice as intended.
So, in the corporate world, I suspect that prediction markets suffer from primarily b) and c). It's probably seen as a pain in the ass, and in most businesses its tough to articulate the value of the marginal value from an increase in decision quality arising from the prediction market.
In contrast, take a commonly used new-ish management practice, the one-to-one meeting. (a weekly or bi-weekly meeting between manager and direct report that is discusses objectives, obstacles to success, how the manager can help, reviews priorities, etc.). Now as far as I have been able to discern, this was first advocated for in the management press by Andy Grove in High Output Management (1983). He, and many others, have made compelling arguments for their practice. I have looked and failed to find any academic research on the subject (I will freely admit this could result from my poor research skills). Managers can grasp why a one-to-one makes sense, you can try the idea pretty easily and can construct a narrative how it might make one's organization more effective.
Yes. But additionally, as I understand it, many of these prediction market ideas are for government functions. Governments are particularly bad at buying new types of services, usually worse than doing them themselves.
I sell services to governments, and they are effectively excuse finding machines when it comes to change. They can draw the purchasing cycle out for a full election cycle, and then it starts again without anything being purchased. I know of cases in which governments have been in the process of instgating new services from the private sector for over ten years without any contract being signed.
The resistance to change is why so many improvements have potential.
Sales and marketing is almost the same as conning people. The difference iis only known after the fact - if you feel conned, you were, if not, it was sales and marketing. There is a significant grey area in between.
Where does one find academic information/suggestions on social innovation options? Any particularly regarding small non-profit organizations? Student Unions?
Does this approach not call for imagination, which is a function of individuality; and thus require an end to the recession and suppression of individuality?
Marketing and salespeople are usually dismissed by abstract thinkers and academics. They're basically viewed as glorified conmen trying who spew spin bullshit to get people to make bad decisions. And that may describe a segment of that field. But in most orgs sales and marketing serves a very important function.
They serve as an interface between the engineers and customers. There's very few products that work like "if you build it they will come". Marketing help identify which potential customers are most likely to benefit from the product. Salespeople help determine the specific risks and identify the best method and version of the product that works with the clients' particular needs. Often times they go back and forth to work with the engineers and figure out what kind of design changes would be most in demand in real world usage.
Here's what I'm saying. Prediction markets are probably at the point where it needs marketing experts and salespeople. The academic research and foundational design is well-past the minimal viable product stage. And you probably understand the theory and research better than anyone. But you don't have much expertise or experience in marketing and sales. I would say the best way to move prediction markets as a product forward, would be to identify a way to onboard that expertise.
When a social/institutional innovation would harm the short term interest of an existing political tribe, it is dead from the start. Look at all the field trials on education innovation which achieved better results at lower labor costs, which the teachers union killed in favor of unproven higher labor cost changes.
First of all, from a "rationalist" perspective, if there's anything they think they're right about - e.g. that futarchy produces, in expectation, superior results than alternative decisionmaking algorithms -- they should try to capitalize on that edge. The downside is that there could be negative signalling effects if their anti-consensus tactics are perceived by high-status community members (VCs, etc) as counterproductive.
Second, the fact that they are a new company (therefore probably already betting big on some other orthogonal issue) might be precisely what enables them to experiment with a new idea about corporate governance.
So I see where you're coming from, but I think the argument is flawed. Is it not the case that Silicon Valley startups and their investors were the first to experiment with ideas like compensating employees with stock options, justifying a company valuation based on userbase growth rather than revenue growth, open office plans, compensation packages skewed towards amenities rather than traditional benefits, etc? How is this any different?
Startups are usually making a big bet on something else, often several other things, and they are wise to not try to bet big on two many innovative changes at once.
Ask what questions you'd like the answer to, where you will eventually know the answer. Or ask what specific decision options will lead to more of particular measurable outcomes.
Inspiring content. Thanks that you holds some great ideas about the good impacts of social innovations to our education. These are great things and information that we have to remember because it ideally helps us to stabilized our good learning skills.
"by now academics have accumulated a large pile of promising institution ideas, many of which have supporting theory, lab experiments, and even simplified field trials. In addition, academics have even larger literatures that measure and theorize about existing social institutions."
I know someone who's starting a new institution--is there a guide to this literature you can recommend? (If so, perhaps the creation of such a user-friendly guide would itself represent low-hanging fruit?)
Fair enough, but you provided no quantities (relative or absolute) in your post about compression ratios at each stage in the pipeline you described.
Here's a number: there have been (extremely conservatively) >1,000 well-structured RCTs of institutional alternatives that I know of personally that have been executed by large companies since 2000.
Here's another number: ~90% of these tested institutional alternatives failed to create statistically-significant improvement in the test group versus the control group with incremental benefits greater than incremental costs.
Unless you think that companies (in general, not just some specific company) are extremely bad at deciding what to test, and there is this incredible backlog of great institutional alternatives out there that nobody is willing to try, then why are they willing to try thousands of others ideas with a 90% failure rate?
Isn't a more realistic interpretation that coming up with new ideas that create improvements is incredibly hard, i.e., the real bottleneck?
To say something is a bottleneck is not to say that nothing gets through it. It is to say that is the most restrictive limiting point in the flow. I'm happy you know of some field trials of some things, but there are a great many other things that aren't being tried.
Robin,
Sorry but you're wrong on the facts in part of this. The company that I started provides the technology used to execute field trials for many of the largest consumer companies in the world. There have been thousands of field trials of institutional alternatives run by large companies over the past 15 years that I know about personally on topics ranging from changing salesforce compensation / incentive structures to reorganizing jobs and roles.
As an aside, academic research has very rarely (note, not never) been a direct source of those ideas that create positive gains net of costs in the test group versus the randomized control group.
How can someone graduate from a modern MBA program without easily understanding that more accurate probabilities enable more effective decisions? I can more believe that they fear prediction markets will “run amok of existing power structures and interests.”
I'm curious what would ideas you (or others) would rate as the most promising social innovations are at (or near) the point of prediction markets the social innovation pathway?
In the corporate world, I would argue that the source prestige and funding related to adapting academic ideas to the complex corporate world changes at that step. The funding comes from the organizations who are piloting or implementing new processes or innovations not for research purposes, but for their own benefit. Prestige is derived from being seen as the expert in implementation - or at least, helping others implement.
Management consultants are constantly vying to find new ideas they can take to clients. Academically validated ideas are a potential rich source here. The challenge is driving adoption - a sales and marketing problem (as Doug mentions), and in some cases a design and engineering problem. McKinsey (plenty of money and prestige) is has discussed prediction markets in its publications, obviously with a goal to facilitate conversations with clients.
I find that in practice (I'm a management consultant), the robustness of academic support behind the effectiveness of a given methodology is less important to getting an idea opted versus:a) How subjectively believable is the idea. Obviously this is influenced by prior research and case-studies, but a key input is having someone senior grok itb) How feasible is the implementation is in practice? Is it be cheaply tried and tested or it it expensive and will it run amok of existing power structures and interests?c) The magnitude of benefit you might reap, if the idea worked in practice as intended.
So, in the corporate world, I suspect that prediction markets suffer from primarily b) and c). It's probably seen as a pain in the ass, and in most businesses its tough to articulate the value of the marginal value from an increase in decision quality arising from the prediction market.
In contrast, take a commonly used new-ish management practice, the one-to-one meeting. (a weekly or bi-weekly meeting between manager and direct report that is discusses objectives, obstacles to success, how the manager can help, reviews priorities, etc.). Now as far as I have been able to discern, this was first advocated for in the management press by Andy Grove in High Output Management (1983). He, and many others, have made compelling arguments for their practice. I have looked and failed to find any academic research on the subject (I will freely admit this could result from my poor research skills). Managers can grasp why a one-to-one makes sense, you can try the idea pretty easily and can construct a narrative how it might make one's organization more effective.
What's the lower bound on group size to test prediction markets?
Yes. But additionally, as I understand it, many of these prediction market ideas are for government functions. Governments are particularly bad at buying new types of services, usually worse than doing them themselves.
I sell services to governments, and they are effectively excuse finding machines when it comes to change. They can draw the purchasing cycle out for a full election cycle, and then it starts again without anything being purchased. I know of cases in which governments have been in the process of instgating new services from the private sector for over ten years without any contract being signed.
The resistance to change is why so many improvements have potential.
Sales and marketing is almost the same as conning people. The difference iis only known after the fact - if you feel conned, you were, if not, it was sales and marketing. There is a significant grey area in between.
Where does one find academic information/suggestions on social innovation options? Any particularly regarding small non-profit organizations? Student Unions?
Does this approach not call for imagination, which is a function of individuality; and thus require an end to the recession and suppression of individuality?
Marketing and salespeople are usually dismissed by abstract thinkers and academics. They're basically viewed as glorified conmen trying who spew spin bullshit to get people to make bad decisions. And that may describe a segment of that field. But in most orgs sales and marketing serves a very important function.
They serve as an interface between the engineers and customers. There's very few products that work like "if you build it they will come". Marketing help identify which potential customers are most likely to benefit from the product. Salespeople help determine the specific risks and identify the best method and version of the product that works with the clients' particular needs. Often times they go back and forth to work with the engineers and figure out what kind of design changes would be most in demand in real world usage.
Here's what I'm saying. Prediction markets are probably at the point where it needs marketing experts and salespeople. The academic research and foundational design is well-past the minimal viable product stage. And you probably understand the theory and research better than anyone. But you don't have much expertise or experience in marketing and sales. I would say the best way to move prediction markets as a product forward, would be to identify a way to onboard that expertise.
When a social/institutional innovation would harm the short term interest of an existing political tribe, it is dead from the start. Look at all the field trials on education innovation which achieved better results at lower labor costs, which the teachers union killed in favor of unproven higher labor cost changes.
I'm not sure that's true.
First of all, from a "rationalist" perspective, if there's anything they think they're right about - e.g. that futarchy produces, in expectation, superior results than alternative decisionmaking algorithms -- they should try to capitalize on that edge. The downside is that there could be negative signalling effects if their anti-consensus tactics are perceived by high-status community members (VCs, etc) as counterproductive.
Second, the fact that they are a new company (therefore probably already betting big on some other orthogonal issue) might be precisely what enables them to experiment with a new idea about corporate governance.
So I see where you're coming from, but I think the argument is flawed. Is it not the case that Silicon Valley startups and their investors were the first to experiment with ideas like compensating employees with stock options, justifying a company valuation based on userbase growth rather than revenue growth, open office plans, compensation packages skewed towards amenities rather than traditional benefits, etc? How is this any different?
Startups are usually making a big bet on something else, often several other things, and they are wise to not try to bet big on two many innovative changes at once.
Ask what questions you'd like the answer to, where you will eventually know the answer. Or ask what specific decision options will lead to more of particular measurable outcomes.