Social Design Debt

Technical debt .. reflects the extra development work that arises when code that is easy to implement in the short run is used instead of applying the best overall solution. (more)

In the design of complex systems, we have long observed a robust phenomenon: when people only consider local costs and benefits when making design changes, they miss the many costs that changes impose elsewhere. Such costs accumulate, and reducing them requires periodic redesign that considers larger scales of interactions. These sort of costs are naturally limited when systems frequently die to be replaced to other systems started recently from scratch. But long lasting complex systems can accumulate large costs of this sort.

For example, in contrast to most nations, apparently the US has *two federal agencies responsible for collecting economic data. Their authorizing legislation has been interpreted to mean that they can’t share details of this data with each other. A more accurate and consistent picture could be drawn about the economy from the data if such integration were allowed, but its not. Everyone in these agencies knows about this problem, but no one has bothered to try to change the authorizing legislation for a more rational outcome. New nations know to avoid this problem, but in old nations like the U.S. such problems just accumulate.

This seems to me an important and neglected issue for our longest lived social systems, such as in law and governance. In The Rise and Decline of Nations (1982), Mancur Olson famously argued that nations tend to decline via accumulating organized interest groups who lobby for changes in their local interest, and veto larger changes to more efficient arrangements. This seems a closely related point, but not quite the same point.

GD Star Rating
Tagged as: , ,
Trackback URL:
  • Gunnar

    This seems to show a limit to the benefit of Schelling Fences.
    In a way groups within a larger system can start to exploit the presence of these fences. And I wonder whether it is possible to counter that.

  • Ari

    I have personally observed the same phenomenon. The more important a system is, and the more likely it is going to last, the more important it is to get it right at the first time. This requires usually for the designers to have both broad and deep knowledge.

    Take for example Windows, Linux and Mac. Windows is suffering from many design problems now. It doesn’t have proper package manager, software version control is a mess, API problems. Linux is suffering from many same problems. Lack of standards, useless competition among libraries and basic software. No control over supply chain, API’s change, lack of QA etc.

    Not to mention programming languages. Think of C++ and even many of its successors tend to lack understanding of what has made recent other programming languages great because the developers have focused on some specific problems of the past. They tend to think of programming language features but not aesthetics, ecosystem, people involved or culture or whatever.

    I think a lot of these suffer from same kind of design problems. You can in many ways tell it from their personality too. People who know depth tend to lack breadth and vice versa. People who design things should aim for both, read and travel but specialise too. They should study competing designs. Maybe Elon Musk is an outlier.

    This is quite an interesting topic but I feel I’m too tired to continue. I think I had an old and more detailed rant about this in software world. Many of these problems are not a problem of manpower but lack of coordination and will to change beliefs and habits.

    • Doug

      What you’re describing is frequently referred to in computer science as “New Jersey Style”. When a problem is heavily loaded on network externalities, like programming languages or communication systems, “done” usually beats “perfect”. Design flaws become obvious in retrospect, but systems early to market have significant advantages.

      Good design takes time. Sloppy, but workable, products get delivered first. That gives them time to become entrenched. When it comes to commercial success in these types of problems, elegance is a bug not a feature.

  • AG

    Perhaps having redundant data sets leads to better self-policing. If one knows that others can easily vet one’s data by comparison to an alternate source, then one may be less likely to fudge the numbers. I’m uncertain how policing would be effected if two data gatherers* work for the same boss. Unintended consequences that assumes a level of insight on the party of each body of actors. Perhaps the assumption is misplaced.

    *or are they hunters? 🙂

  • Lord

    Not always though. Sometimes urgency intervenes, like wartime, sometimes there is deregulation, like airfares, and sometimes change supercedes, like mobile relative to fixed. It is important for the knowledgeable to have plans in place for these rare opportunities though.

  • Joe

    Seems odd to describe the US as an old nation.

    Do you think this phenomenon is noticeably more significant in much older nations?

    • Peter David Jones

      I though that.. are Iceland, Switzerland and the UK particularly terrible?

  • Doug

    One of the most effective ways to prevent technical debt is to minimize the contact surface between sub-components. Software architects describe this as “modularity”. Components present very simple external facing interfaces that are highly abstracted away from internal implementations. Larger-scale components are often composed of modular sub-components, which may in term have modular sub-sub-components, etc.

    The smaller the external surface, the less likely that any proposed change changes external behavior. Simple interfaces can accommodate a wider space of implementations. Re-designs of the module are less likely to have any non-localized effects. This comes at the cost of efficiency. Modularity tends to require a lot of “glue” to convert complex and optimized internal representations to simple external representations. Harmonizing the internal representations of two modules can make communication faster and easier, at the cost of coupling the design between the two.

    Back to the example given in the post. De-coupling the two economics agencies may not necessarily be a bad design. Even if it is less efficient. Allowing the two agencies to work in tandem would blur the lines between them. That makes it harder to reason about any one agency in isolation. Cutting the budget or changing the mission of a single agency would have spillover effects on the other. Cost-benefit analysis or performance would become likely be less accurate and require more effort.

  • Michael

    I think you can extend this idea even further. National institutions (democracy, economic policy, etc) are almost never designed from the ground up. Rather, they evolve, based on political momentum and feasibility. Even when a radical overhaul is made, it is done in the context of the status quo, and may only be made possible by recent events. For example, the glorious revolution in Britain, when the parliament effectively seized power from the king, was the culmination of a long process of the parliament gradually gaining more and more power. Software is easier in that you can re-design things from scratch. When it comes to national institutions, we’re stuck with an evolutionary process.

  • arch1

    I don’t see important differences between the “technical debt due to local optimization” problem and Olson’s “interest groups” problem.

    In addition to pursuing more obvious paths (expanding off-planet, learning from existing examples) it seems prudent to have some people investigating apparently wacky solutions to such problems.

    We *do* want to pass the Great Filter, after all. And it’s beginning to look pretty quiet out there.


    “For example, in contrast to most nations, apparently the US has *two federal agencies responsible for collecting economic data.”

    Actually, it has more than two. I think you can apply the social design debt even beyond nations: in many cases it’s the reason for incredibly similar neighboring nations not to formally merge into a larger nation.