Consider a few things we know about task specialization and innovation: Larger cities and larger firms both have both more specialization and more (i.e., faster) innovation. More global industries also have both more specialization and innovation. And across the great eras of human history (animal, forager, farmer, industry), each era has brought more specialization, and also faster rates of innovation.
Well can you please share the data as this? As this "fact" is both counter intuitive and exactly the opposite of my experience (and it's seems many other people are sceptical too).
A way to say this is that ideas are easier to find than opportunities to exploit them. So cross-fertilization is fine but it's easy; the hard thing is having a niche where you have fewer design constraints and fewer competitors. I wonder whether there's already an equivalent argument for biological diversity and the relative numbers of specialist vs. generalist species.
There are a lot more small firms than large firms. It may be that the most innovative firms are small firms, but that is very different from saying that small firms are generally innovative. You don't think that large firms should innovate by buying random small firms, do you?
Also, if a small firm is more specialized, a single innovation could be very visible.
Thanks. Could you please provide your go to source on this? (It's credible to me, but a citation would come in handy in the debates I sometimes have with opponents of big cities).
This suggests to me the slowdown in innovation might be due to a lack of connecting specialists.
The trouble is that innovation is a high-dimensional space. It seems like a lot of what specialization buys you is shrinking the number of dimensions under consideration to something tractable. Yet this leaves us with the problem of many specialists exploring different narrow chunks of the relevant space.
I frequently see arguments about a need for generalists, but they are almost never fruitful; generalist as a term seems to mean "has soft skills" or "anti-specialist" which provides some mysterious advantage. A better way to think about this is a need for specialists in the task of relating different specialized tasks. Math has a lot of kind of thing already - there are multiple whole fields dedicated to connecting algebra and geometry, for example.
Since the dimensionality of innovation is very high, there should be a huge number of possible connecting specialist tasks. Currently this sort of seems to get lumped into general-ish positions like manager or executive, as sort of an afterthought to their people and administrative tasks. Perhaps we are accidentally constraining ourselves to the dimensions of our orgs, like corporations or government labs. The famous success of places like Xerox PARC or Bell Labs could then be reinterpreted as the freedom to specialize differently than their parent orgs.
One thing I have wanted to ask you - big cities being more innovative, does this adjust for them also tending to attract a higher percentage of elite level human capital?
"Larger cities and larger firms both have both more specialization and more (i.e., faster) innovation." ??
I am skeptical that larger firms exhibit more/faster innovation. In my experience, the main way that larger firms innovate is acquiring smaller, more innovative firms.
But innovation is also much easier when processes are more vertically integrated, enabling the same party to control product market fit from both sides. Based on what I have seen working in manufacturing, I would expect that effect to dominate the benefit from specialization.
Very interesting. I've thought a little about this problem in the context of software design, where specialization is especially hard partly because code looks the same everywhere so we tend to jump all over the place. One obvious thing is to try to better separate people who create specifications from those who write implementations. We can try to write/adopt better languages or libraries for specs, particularly for sharing and combining specs.
There sorta was a movement in the 90s for massive Enterprise software systems that was all about giant plans about how everything was going to work, but those largely failed and the "scrappy" methodologies won out. So the "formal plan making" part somehow needs to be much more distributed and adaptable and competitive. Maybe there's an opportunity for distributed project governance models, where anyone can create plans/revisions/pivots and you bet on which ones are likely to succeed.
One down side of specialization is in some ways it hinders very high performers. I remember Gabe Newell was talking about how at Valve their best developer would write 4000 lines a day, as opposed to a highly specialized company like Intel where the average developer writes 1000 lines a year. And you can rest assured that the Valve dev's code was much better. Not quite sure what the trade-off is there.
On the other hand: https://www.overcomingbias...., https://www.overcomingbias....
Well can you please share the data as this? As this "fact" is both counter intuitive and exactly the opposite of my experience (and it's seems many other people are sceptical too).
A way to say this is that ideas are easier to find than opportunities to exploit them. So cross-fertilization is fine but it's easy; the hard thing is having a niche where you have fewer design constraints and fewer competitors. I wonder whether there's already an equivalent argument for biological diversity and the relative numbers of specialist vs. generalist species.
There are a lot more small firms than large firms. It may be that the most innovative firms are small firms, but that is very different from saying that small firms are generally innovative. You don't think that large firms should innovate by buying random small firms, do you?
Also, if a small firm is more specialized, a single innovation could be very visible.
Thanks. Could you please provide your go to source on this? (It's credible to me, but a citation would come in handy in the debates I sometimes have with opponents of big cities).
This suggests to me the slowdown in innovation might be due to a lack of connecting specialists.
The trouble is that innovation is a high-dimensional space. It seems like a lot of what specialization buys you is shrinking the number of dimensions under consideration to something tractable. Yet this leaves us with the problem of many specialists exploring different narrow chunks of the relevant space.
I frequently see arguments about a need for generalists, but they are almost never fruitful; generalist as a term seems to mean "has soft skills" or "anti-specialist" which provides some mysterious advantage. A better way to think about this is a need for specialists in the task of relating different specialized tasks. Math has a lot of kind of thing already - there are multiple whole fields dedicated to connecting algebra and geometry, for example.
Since the dimensionality of innovation is very high, there should be a huge number of possible connecting specialist tasks. Currently this sort of seems to get lumped into general-ish positions like manager or executive, as sort of an afterthought to their people and administrative tasks. Perhaps we are accidentally constraining ourselves to the dimensions of our orgs, like corporations or government labs. The famous success of places like Xerox PARC or Bell Labs could then be reinterpreted as the freedom to specialize differently than their parent orgs.
My main point was to question the means by which you would reduce product variety for which there was consumer demand.
Its still true correcting for that, yes.
One thing I have wanted to ask you - big cities being more innovative, does this adjust for them also tending to attract a higher percentage of elite level human capital?
I have a friend who worked in inkjet technology in the 90s. There are absolutely ways to specialize in painting black.
No, they really are more innovative on average. I've seen the data.
"Larger cities and larger firms both have both more specialization and more (i.e., faster) innovation." ??
I am skeptical that larger firms exhibit more/faster innovation. In my experience, the main way that larger firms innovate is acquiring smaller, more innovative firms.
But does more vertical integration correlate with time & larger cities, firms, & markets?
But innovation is also much easier when processes are more vertically integrated, enabling the same party to control product market fit from both sides. Based on what I have seen working in manufacturing, I would expect that effect to dominate the benefit from specialization.
Very interesting. I've thought a little about this problem in the context of software design, where specialization is especially hard partly because code looks the same everywhere so we tend to jump all over the place. One obvious thing is to try to better separate people who create specifications from those who write implementations. We can try to write/adopt better languages or libraries for specs, particularly for sharing and combining specs.
There sorta was a movement in the 90s for massive Enterprise software systems that was all about giant plans about how everything was going to work, but those largely failed and the "scrappy" methodologies won out. So the "formal plan making" part somehow needs to be much more distributed and adaptable and competitive. Maybe there's an opportunity for distributed project governance models, where anyone can create plans/revisions/pivots and you bet on which ones are likely to succeed.
One down side of specialization is in some ways it hinders very high performers. I remember Gabe Newell was talking about how at Valve their best developer would write 4000 lines a day, as opposed to a highly specialized company like Intel where the average developer writes 1000 lines a year. And you can rest assured that the Valve dev's code was much better. Not quite sure what the trade-off is there.
Tasks don't specialize; people do. I interpreted "task specialization" to mean the specialization of a person in a narrow range of tasks.