Social Innovation Disinterest Puzzle

Back in 1977, I started out college in engineering, then switched to physics, where I got a BS and MS. After that I spent nine years in computer research, at Lockheed and NASA. In physics, engineering, and software I saw that people are quite eager to find better designs, and that the world often pays a lot for them. As a result, it is usually quite hard to find even modesty better designs, at least for devices and mechanisms with modest switching costs.

Over time, I came to notice that many of our most important problems had cores causes in social arrangements. So I started to study economics, and found many simple proposed social innovations that could plausibly lead to large gains. And trying my own hand at looking for innovations, I found more apparently plausible gains. So in 1993 I switched to social science, and started a PhD program at the late age of 34, then having two kids age 0 and 2. (For over a decade after, I didn’t have much free time.)

I naively assumed that the world was just as eager for better social designs. But in fact, the world shows far less interest in better designs for social arrangements. Which, I should have realized, is a better explanation than my unusual genius for why it seemed so easy to find better social designs. But that raises a fundamental puzzle: why does the world seem so much less interested in social innovation, relative to innovation in physical and software devices and systems?

I’ve proposed the thesis of our new book as one explanation. But as many other explanations often come to people’s minds, I thought I might go over why I find them insufficient. Here goes:

Innovation is Hard – Some point to the general fact that innovation is hard. It is hard to know what will work, it is hard to judge effectiveness in a complex world, there are costs to failure, and those who push innovations can seem non-conformist and threatening to status quo powers. But of course these are also obstacles to innovation in physical and software arrangements. We are trying to explain a different attitude toward social innovation.

Social Science Doesn’t Exist – This is basically what I was told by physics profs as an undergrad. Those people in that other building, they just make stuff up, they don’t know anything. And this view is just wrong. We know a whole lot about the social world. We are of course ignorant about a lot of society, but we also retain great ignorance about software and the physical world.

Few Social Gains Possible – Many implicitly assume that social innovation might have been a good idea centuries ago, but today we know nearly all the best ways to arrange firms, employee compensation, rental agreements, marriage contracts, voting systems, and so on. They just can’t imagine future changes in such things remotely as big as past changes, and so assume only small gains are possible. But that’s just crazy; the space of possible social arrangements is vast, and we’ve only explored a few small corners of it.

Humans Have Free Will – Some say that while physics and software can be deterministic, free will means that humans can never be fully predicted. Whatever our best theories predict that humans will do, they can choose to do the opposite, and they often will choose this just to show who’s boss. But in fact we can and do design physical and software systems that accept a lot of uncertainty about the behavior of parts. And we see a lot of pretty predictable behavior in existing social institutions.

Can’t Do Social Experiments – Some say that we can’t test interesting social innovations because it is immoral to do experiments on people if some of them might get worse treatment. But of course we do many medical experiments where some get worse treatment. We tolerate the real world continually producing worse outcomes for many. Paying people to participate in experiments seems to me sufficient to compensate subjects for worse treatment. And in fact experiments are done quite often in the social and human sciences.

Trial Costs Are Too High – Some say the cost of social experiment failure is too high, and point to huge changes like the French Revolution. But just as in physical and software innovation, there’s no excuse for not starting with small trials and tests, and only following with larger trials after smaller-scale success. Big revolutions are mainly a kind of war, not a kind of innovation. Yes war can induce innovation, in both physical and social devices, but surely one can favor innovation without favoring war.

Social Change Can’t Be Designed – Some say that while social systems can improve greatly over time, this can’t or shouldn’t result from conscious deign choices. Design attempts only show human arrogance to think that we can understand the vastly complex social world. And they just encourage those damn regulators to think they can useful interfere in cities and nations. But of course a great many useful social changes have resulted from conscious choices. Including laws and regulations.

Society Is More Complex – Some admit that we know enough about the social world to design useful changes, but claim that in practice we have far fewer occasions to do useful social design because social worlds are far more complex, compared to physical and software worlds. But if we focus on proposed social innovation that are quite simple, with solid arguments for their plausibility, we still find that people are much less interested in exploring or adopting those innovations. In fact, most social mechanisms are far simpler than many widely used physical and software devices.

High Social Switching Costs – Some say that disinterest in social innovation results from it being very expensive to make big changes to our fundamental social institutions. And yes, it is expensive to change things like constitutions or fundamental features of contract law. But we have many proposed social innovations that can be explored at far less cost. We can start with simple math models, simulations, and lab experiments, then try modest field experiments, and finally larger scale field trials. Furthermore, we have often explored and adopted physical and software innovations that were quite expensive to test. Comparing innovations that can be tested at a similar cost, we see far less interest in social innovations.

Value Conflicts – Some say that while we usually agree on which are better physical and software innovations, we have strong value conflicts regarding which are better social arrangements. This makes it hard to trust any advisors on changes, as they can have agendas. But most of our physical and software systems are part of our social systems, and as a result we do usually have differing interests in which physical and software systems are adopted. And advisors regarding physical and software systems can also have strong agendas. Yes, engineers can often agree on technical criteria for evaluating such systems, so that if we make the right transfers between people then adopting the most technically efficient system can be better for everyone. But economist also have constructed similar concepts of efficiency for which similar everyone-can-gain claims can be made.

Misleading Motivations – This is my favored explanation. We are less honest about our motives regarding many familiar social arrangements, relative to physical or software arrangements. We usually point to particular motives as the most important, but those are much less important than we admit. We reward our policy advisors for pretending with us to pursue these usual motives. But we know at some level that we are not that interested in them, and thus also in policy reforms designed to better achieve them. Such reforms don’t solve our real problem. We may show more interest in reforms designed to help us with our real motives, at least if they can let us keep pretending to pursue our pretend motives.

Added 15Jan: The same innovations arouse more interest when they are framed as tech than as social innovations. We see this especially sharply in the blockchain stuff now. Also, the same people are more likely to be hired to work on such innovations if they are presented as tech experts than as social experts. Hedge funds prefer physics over finance PhDs, for example.

GD Star Rating
Tagged as: , ,
Trackback URL:
  • rcousine

    I think social change has frequently been attempted and is still attempted to this day. But are you sure there are big gains to be had? Are you really sure?

    When I think of big social changes, I tend to think of Year Zero ideas ranging from the French Revolution to Pol Pot, and of course that’s a tendentious set of social revolutionaries to single out, but the point is self-evident: take a social revolution that seems self-evidently good to its fans, and the result is often blood…so much blood…

    So the cost of change and experiments is potentially grave.

    • The cost of experiments can also be large in physical and software devices. Costs are smaller if you start with small trials, then only do larger trials after small success.

  • A nice list of problems with social innovation with some more significant than others. While gains may or even are likely possible, so are losses, so the distribution of these become very important. I do think many believe there are good reasons for existing arrangements even if they don’t know or can’t imagine what they are. They also know many changes are proposed, not for social gain, but for zero sum or even negative sum advantage, so there is much more inertia and skepticism than in other areas. Being essentially political, the inclination is to assume they are zero sum to first order. Even when there are gains, the distribution of those gains matter and the losers know they won’t be compensated and will fight them. While social mechanisms aren’t that complex, there are far more numerous and operate in many different ways. People are used to approaches of gaming systems, are creative, and even if they don’t see how it can be gamed, they expect such gaming to occur. Not that it doesn’t already occur, but that we already tend to know how they are gamed and already have some checks and balances in place against them, even though this is part of the cost. Physical innovation offers more options, competition, and marketplaces to compare and contrast and arrive at more and less successful ones. Social innovation offer fewer in these regards as more leads to more complexity and more demands on those using them, dwelling at the network level. The disparity of values is more significant because the results must be shared rather than exist individually. Many also see existing social obstacles as opportunities to exploit than problems to be solved and consider innovation around them will remove most of the inefficiency associated with them. It may people assume ulterior motives as much as lack the presumption of hidden ones.

    • I just added a section on “Trial Costs Are Too High”

  • Robert Koslover

    Per your realization that you “…naively assumed that the world was just as eager for better social
    designs. But in fact, the world shows far less interest in better
    designs for social arrangements.” Yup, that sounds about right to me, although I also think you’ve done pretty darn well, all things considered, in shaking up the social sciences side of academia. So… you’ll now be returning to physics, engineering, and/or computers, right? There are actually still many interesting problems left in these fields to solve. Come on back and solve a few. You know you want to. 🙂

  • You might find some of the late Hal Linstone’s (founder of the Systems Science program at Portland State) interesting. His “multiple perspectives” approach looks at the Technical, Organizational, and Personal aspects of problems.

  • Of interest for effective altruists, since if social innovations are neglected they could be high impact.

    Not quite sure how you delineate “social innovation”, but these are plausible candidates for EA social innovations.

    The effective altruism community itself
    Career advice (80,000 Hours)
    Donation advice (eg, GiveWell)

    By contrast, I think EA software innovations haven’t been very succesful, in line with your claim.

    • Actually I’d reach just the opposite conclusion. This analysis tells us

      * Social innovations are hard to implement/make happen.

      * We don’t really have a good grip on the factors that make apparently desirable social innovations so hard to effect.

      * It’s easy to misidentify the value society derives from a given social innovation, e.g., if the utility people derive from college is largely social and psychological not educational than attempts to reform college to be more educationally effective may be quite harmful.

      * There are systematic factors that prevent EA’s from both having a firm grip on the true social good/values a social structure serves and effectively working to reform that organization.

      For instance, if you wanted to reform college to better serve the social (making friends), psychological (gentle launch into independent living) and value modification (college causes people to support things like research and education in cases where it is the true benefit as well) you won’t be effective if you go around explaining that we need to focus less on good teaching and academic rigor and more on socialization and the pretense of learning.

      The Gates foundation can probably handle this kind of dissonance since Bill and Melinda can chat privately about their real goals and sculpt their public rhetoric and purchases/donations to achieve their real ends. But once you start trying to get the EA community working on something like this you end up with a constant flux of people noticing your actions aren’t very good at effecting your stated goals so either donations dry up and people work at cross purposes or you need a widespread whisper campaign saying ‘Shh…we really don’t believe in all our rhetoric we’re really trying to …’ Such hypocrisy will surely be noticed and you’ll be even less effective than if you had simply honestly admitted your unpopular motives.

    • In fact one could frame this whole post as why doesn’t EA work for social innovation:

      There are lots of people who (in an EAish style) try and innovate to make the world better. This doesn’t happen in the social context. Why is the standard EA sort of style of notice a problem find a solution based on what has worked in the past and publicize it not effective?

  • But if we focus on proposed social innovation that are quite simple, with solid arguments for their plausibility, we still find that people are much less interested in exploring or adopting those innovations.

    For example?

    • How about prediction markets? Adopting company internal prediction markets is a pretty simple modification compared to the overall complexity of the corporate organization and the whole idea of jobs and careers that go with it with plenty of sound arguments and even data backing them up.

    • How about prediction markets inside companies?

  • I don’t think it’s so much that we we know at some level we aren’t interested in the reforms proposed by policy designers. For instance, I think people would very much like systems which increased government effectiveness, decreased busy work etc.. etc.. Indeed, we would value these systems more than the costs to other values that adopting things like prediction markets would entail.

    I’d argue the problem is more that *given* the current structure incentives disfavor expressing support for such changes. If people were really given a deciding vote about whether to make changes I think one might see far more such changes. However, i fact generally people only have very limited influence (e.g. voting) and therefore have incentives that highly favor using their answer to signal different values or allegiances or are the beneficiaries of the current structures (e.g. current CEOs, successful politicians and consultants have much to lose and little to gain by very disruptive changes like substantial use of prediction markets).

    There are some interesting results about how people’s views change based on whether or not they think their vote is decisive or not.

    • If we focus on small scale social arrangements, why don’t you have just as much incentive to push to improve them as to push to adopt small scale physical and software arrangements?

      • You may have as much incentive to push to improve them but you often have a very serious cost to pushing to improve them. For instance, being seen as weird or challenging the status quo is a big deal even inside a group of friends.

        If by small you mean things like being poly amorous or changes in friend group dynamics there are huge costs to being seen as weird or non-conformist. But maybe I misunderstand what you mean by small scale social arrangements…and I do think we see more innovation there than elsewhere.

      • I made no claim about what people would accept if moved to a new culture.

        Why don’t proposals to change physical or software devices create the same impression of b3ing weird or challenging the status quo?

      • I took you to be saying that people actually *liked* the way that current organization worked but in regard to unstated (and unwilling to state) values other than the ones they were willing to publicly endorse. The claim about if moved to another culture was an attempt to pry apart the question of whether they would be willing to consider/endorse a change to social organization from the question of whether they actually liked the outcomes produced because of values they weren’t willing to acknowledge. Indeed, I was somewhat surprised that you took this view since I thought you believed that even according to people’s own true preferences our social institutions were underperforming.

        As for why proposals for physical of software devices don’t create the same impression of being weird or challenging the status quo is that they don’t directly implicate the kind of allegiance/social conformity aspects that primates are particularly evolved to be sensitive to for the purposes of alliance etc.. At least in our present society.

        I suspect that if you went back in history in a great many societies there was this kind of resistance to doing things differently in terms of physical devices or innovations as well. Hence a huge reason the rate of productivity increase went through a huge leap at the time of the industrial revolution. Though I do suspect there may be hardwired reasons we would never become as comfortable with social change to the extent we are with technical change and thus a greater difficulty transitioning to a society in which we also don’t view social innovation as weird or disruptive in a bad way.

      • Maybe I misunderstood you but from the way I read your post it sounded like you were saying not only is the innovation in such areas difficult to accomplish but people actually wouldn’t favor such innovation if they were suddenly permanently transported to a culture where it had already been achieved and I’m skeptical of that.

  • Ronfar

    Why do people still use QWERTY keyboards?

  • davidfriedman

    Isn’t the obvious explanation that improvements in social institutions are mostly public goods, improvements in product design mostly private goods, and that for familiar reasons it’s much easier to get private goods produced?

    • Social innovations are often embodied in private goods. Firms, marketplaces, social media apps, and many social meeting places are all often privately owned.

      • Philon

        It is easy to gain acceptance of a new device for doing better what people already want to do: people as individuals can adopt the new device, without persuading lots of other people to join them. But “new social arrangements” usually require network acceptance, and the habits and opinions of people in general change only slowly and with difficulty. If you wanted to introduce successful innovations you should have stayed in engineering and software. (This is not to deny that your new book accurately presents part of the story.)

      • Most physical and software systems also require networks of people to adopt them together.

      • davidfriedman

        FB, WoW, dating programs, all seem like social innovations that were embedded in private goods and got produced.

      • Yes, my claim is a lower level of interest in such things, all else equal, not a zero level of interest.

      • citizen15

        What would you cite as examples of *private* social innovations that don’t seem to attract as much interest as they should?

  • Andrew Swift

    People prefer stability to change, and don’t trust anyone (least of all politicians) to be able to plan at large scale with any level of predictability.

    I might even argue that most of the large changes that have happened socially were possible because people didn’t know how big they would be.

    Better the devil you know.

    • Can you see that everything you said applies just as well to physical and software changes? We don’t like change, especially changes chosen by politicians. That doesn’t explain why we are less tolerant of social changes.

  • Matt Kruza

    been reading your blog for a few months, finally thought I would chime in

    One thing I don’t think you really touch on is the resistance to change is greatest from those who usually are in power / benefit from the exisiting system. Take education k-12. The best public schools are usually in very expensive (compared to other local schools) real estate areas. A useful social innovation would be to let kids go to any school that their parents could transport them to (or which the school would arrange transportation to). So for example in my midwest town a single mother of two could live in a $600 a month apartment and send to any of the best public schools, instead of having to buy a 300-500 thousand dollar house.

    Shockingly (i mock with sarcasm), those who paid 300-500k for a home don’t want the “others” in the poor apartment going to their kids schools. not only do they think it will make their kids education worse, it will probably knock 20-50% off their biggest asset (their home value which is inflated due to education tied to exclusive real estate).

    This is just the most obvious example to me, but I guess my point is that the 10% can exclude the bottom 40-80% and have no desire to change even before all the barriers to switching costs, inertia, routine etc. Explicitly they dont want the social change, but it fits in line with your other research / thoughts that they wont be as blatant and up front as i just explained with their true desire.

    Hope its a fruitful first comment… love your overall contrarian style (although not totally on board with your em future yet… lol)

    • But resistance from those in power should also be an obstacle to physical and software changes. The issue is to explain the different resistance to social changes.

  • Silent Cal

    We still have the question of why there isn’t more interest in social innovations that address people’s real motivations. Do you think the constraint that the innovation must address real motives while being colorable as addressing stated motives, reduces the space of possible social innovations enough to explain the observed lack of interest?

    • Silent Cal

      Also worth noting that subcultures have sometimes emerged with a great interest in social innovations (e.g. hippies, rationalists)

    • Yes, it is harder to design to more constraints, and harder to market such designs.

  • John Wentworth

    The coordination hypothesis: adoption of most physical/software innovation can be done unilaterally. I can buy the latest gadget/app, and benefit from it myself, without anyone else needing to adopt it. This is rarely the case with social innovations – as the word “social” suggests, adoption by many parties is usually necessary. Most social innovations begin with a coordination problem.

    Where social innovations do not face a steep coordination problem, or where some mechanism handles the coordination problem, people do adopt the innovations (e.g. options/futures contracts or online markets).

    • Most real innovation in physical and software devices requires the coordination of many parties.

      • EMP

        Most real innovation in physical and software devices requires the voluntary cooperation and coordination of many parties.

        From where I sit (North America), centralized democracy over large territories is the biggest mountain standing between social innovation and stagnation. The illusion that “we are the government” leads historically ignorant voters to conclusions which favor the status quo.

        That said, most humans are programmed by nature to avoid upsetting the existing order, for fear of what may come.

      • A lot of social innovation can proceed needing only voluntary cooperation. But I claim there is less interest in such things.

      • brianholtz

        This! _Voluntary_ social innovation often garners huge interest: just look at social networking, or the fact that 30% of marriages now start online. Robin’s complaint should be reframed in terms of “political innovation” — i.e. innovation directly involving coercive social institutions. Libertarians should be careful when wishing for political innovation. Beware any political innovation not firmly grounded on principles of choice/competition/exit. The road to serfdom is paved with political innovations.

  • Marius Catalin

    But what is social innovation? Society changes only as a consequence of culture, politics and ideology. We cannot work directly on the object, innovations happen in culture and some of them find a place on the political agendas

  • Riothamus

    I suspect questions of honesty in motivation are subordinate to questions of ignorance. People do not habitually examine the value of social processes the same way they value technological artifacts.

    Even if they did, I expect it would be much more difficult to clearly identify the value added by a social innovation vis-a-vis a technical one. I think this would be sufficient to explain why social innovations embedded in technical innovations garner more interest – the technical value is concrete, and therefore the social value is surplus (or harm – not all social impacts of technology are good, and few were predicted correctly beforehand).

    These two things would compound together badly, since social innovation would be mediated through existing social processes. With a lossy understanding of the innovation, which we understand through the lossy current process, in general I would expect very little value to be communicated. By contrast, technology like a computer is essentially Markovian.

    What about time? I can try a technical innovation and determine whether it will be good for me very quickly – say a 10 minute VR demonstration in the mall. By contrast, even trying a different version of an existing social process to the one I am currently in is much more demanding. Going to a new church or a new club takes hours. A whole new way value would probably be a pretty high minimum commitment.

    • I suspect that you personally know a lot more about tech than about social science. Yes in your world people are more ignorant about social stuff, but there really are other places where people know a lot more about social things, and can identify the value of innovations. The problem is that others show little interest in their achievements.
      Both social and tech systems vary a lot in size. There really are small social innovations, and less interest is shown in these than similar size innovations in tech.

      • Riothamus

        You are correct in your suspicion, though my expertise in technology is driven by my interest in the embedded social changes.

        It is the others to which I refer, not the experts. Even if we stipulate equivalent expertise in technical and social fields among experts, I claim that the consumers of innovations (non-experts in either area) have a harder time seeing the value of a social innovation. That there are people who can identify the value of social innovations does not imply that the people who benefit from it will be able to. Indeed we see similar factors at work with technical innovations all the time, despite the higher interest in tech.

        A consumer of tech knows what their current tech cost, and can evaluate the cost of new tech because it is clear. A consumer of a social process probably does not know the current cost, and the cost of a new social process is probably unclear. I therefore expect lower interest in social innovation. I would also expect that where non-experts have a better grasp of the cost of their present social process, interest in social innovation in that area would be higher relative to other areas.

      • Most costumers know little about the value of most things they buy. I don’t see them having much more ignorance about social than tech innovation value.