There are also many more, clearer descriptions available if you google “page rank algorithm”

Of course there are many details that have not been released but the meaty part is public knowledge.

The particular distinctions between “firms”, “products”, “standards” and “innovations” implicit in this post are an artifact of our current social arrangements. In particular the ability of “firms” to internalize “resources” proportionate to their “wins” is very specific to institutions, and varies greatly even within our current economy. Internalization of value created is very low in the internet space, for example.

Also the post implicitly assumes that the various firms are trying to “win” by competing. Typically firms will try to differentiate, so that they “win” by moving away from their competitors in niche space.

The result is that very often innovation occurs through finding and dominating new niches rather than through making an existing product better or cheaper to produce. Improving existing products or production methods tends to be due to learning curve effects in which there are many very small innovations, which are typically disseminated throughout the industry fairly quickly.

I’d add another related point, modularity, which is essential to allowing offerings to evolve incrementally. This can cut both ways depending on the amount of information transfer.

There’s a somewhat counter-intuitive pattern here. If a single design becomes dominant, usually but not always through network effects, that greatly enhances information transfer (as implied by (8)). So often a single “firm” becoming totally dominant can create a playing field in which success variance is low due to information transfer.

Arguably network effects could be subsumed under (2) but the issue isn’t resources, as (2) states, but enhanced value for customers based on use by other customers.

I attribute this mainly to info becoming much leakier

Uh huh. Call us when Google release their page rank algorithm – or when you find out the details of how James Harris Simons handles his investments. That is the type of “leaklessness” we should be concerned with. It seems pretty leakless to me – and I see no sign it is getting “leakier” as time passes.

“If you worry that one competitor will severely dominate all others in the next really big innovation, forcing you to worry about its “friendliness,” you should want to promote factors that reduce success variance.”

Worrying about Friendliness is orthogonal to whether dominating dynamic is a “single” winner, or a whole economy of competing players. It’s the risk of dominating dynamic doing (irreversible) damage that is the problem. In low-tech environment it’s reduced by having little power, whatever you do with it; in mid-tech environment like ours by human control at all stages, barely sufficient to prevent serious catastrophes; in higher-power environment you’d need a Friendly AI to correctly guide the tech, using carefully calculated planning rather than now-fatal adversarial dynamic. So, reducing the progress towards extremely powerful technology is the only way to postpone the problem of Friendliness (not that I believe at this point it could seriously help).

Reduced success variance contributes minimally to this effect, and I’m not even sure it contributes in the right direction.

Do you count domain knowledge and a flexible culture of innovation as “resources?” If not, I would suggest these as factors.

Eliezer, I edited the list to include many of your suggestions. Not sure I understand “recursivity.” I don’t see that AIs have more cumulative advantage than human tool teams, and I suspect this CA concept is better broken into components.