I think this makes my idea crystal clear:

Analogies are *mappings* between knowledge domains; and there are varying *degrees* of mapping accuracy – ranging from a totally inaccurate mapping (a totally bad analogy) to a perfect mapping (Bayes, causal modelling).

In the limit that the mapping approaches perfect accuracy, the accuracy of analogical reasoning approaches Bayesian Induction/Causal modelling. This shows that Bayesian reasoning is merely a special case of analogical reasoning.

Eliezer keeps critisizing analogies because the mappings are inaccurate (missing info, mixed with some errors), but this is actually a strength of analogies!

Here’s why: Imperfect mappings are what enable us to slide between different knowledge domains (to slide back and forth across concept space). Bayesian reasoning can only be applied to perfectly defined domains; true, Bayes is perfectly accurate, but Bayes cannot let us slide across concept space to explore new ill-defined domains (cross domain reasoning)

In summary there’s a trade-off, the perfect accuracy of Bayesian reasoning at one end of the scale, but confined to narrow, precisely defined domains, and at the other end of the scale, the freedom of analogical reasoning to connect different domains (cross domain reasoning), but with some inaccuracy in the mappings.

I wonder if all valid reasoning is actually analogy formation in disguise. It may just be the case that some analogies are much more sophisticated than others. What EY dismisses as ‘surface similarities’ is really only a criticism of the limitations of bad analogies, not of analogy formation per se.

Lemma:

(1) All valid reasoning can only proceed by manipulating concepts that are already known to us (i.e. concepts that are already in our mind)

(2) To reason about new domains or extend the applicability of known reasoning methods, new concepts must be derived, or old concepts extended.

(3) From (1), in order to move from old/known concepts to new/extended concepts in an understandable way, (2) must involve a mapping between new/extended concepts and old/known concepts

(4) Any such mapping relies on similarities between old/known concepts and new/extended concepts

(5) Mapping of similarities between things is analogy formation

(6) From (5), all valid reasoning is analogy formation

Types of analogies:

(a) Surface similarities (mostly bad analogies): Looking at structural
similarities (similarities in the way things appear)

(b) Functional similarities (better analogies): Looking at similarities in the way things act/are used. (Hypothesis: This is entirely equivalent to causal modeling/Bayesian Induction!)

(c) Semantic similarities (deep analogies): Looks at similarities in the meanings (high-level representations)
of concepts (Hypothesis: This is equivalent to ontology creation and merging – interfaces/mapping between ontological categories)

Funny, I find your writing these days to be far too wordy.

So here’s it is somewhat expanded:

It may be useful to consider abstraction as compression.

As many readers of OvercomingBias are aware, there’s a strong connection between intelligence and compression. Indeed many will go so far as to claim that effective compression is the sine qua non of intelligence. By extension, effective abstraction is essential to the costly computational challenge of intelligently thinking—and predicting—at a meaningful level of complexity without succumbing to the Death of a Thousand Distractions.

One scheme aims to trade detail for salience by a fractal encoding of regularities within the domain of interest. Intelligence is in the encoding. Coherence is maximized over the entire context.

Many of us are familiar with Run-length encoding, Huffman encoding and Lempel-Ziv-Welch encoding, which lossless compression methods achieve progressively tighter compression by encoding progressively higher levels of abstraction. To the extent that we can identify regularities operating not only at the “surface level” of simple ASCII characters or pixels as in Run-length encoding, but more significantly at the “deeper levels” of word fragments or geometric shapes, or even more significantly at the level of semantic (or functional) atoms, then we can exploit such self-similarity at multiple scales to encode ever more effectively.

It’s worth noting that such encoding is fundamentally hard in the information-theoretic sense that there can be no shortcuts that don’t involve hard-earned information, entailing “intelligence”, “increasing knowledge of the search space”, or an “optimization process” as it is often stated in this particular sandbox.

It’s worth noting also that the more “intelligent” the compression method, the more coherent will be the structure of the “atoms” within the compressed output, meaning that on average, “atoms” at any level will have greater mutual information, each will play a relatively greater role in synergy with its associates, and these synergies will tend to operate over a greater context (than, say, simple Run-length encoding.)

Another scheme aims to trade detail for salience by selection of features according to an external function, call it Archimedes_and_his_Lever. Intelligence is in Archimedes. Coherence is maximized over a selected context.

On the other hand, many of us are familiar also with lossy methods of compression. These include mp3 which depends heavily on an external psychoacoustic model, the compressed sensing of our visual cortex, which depends heavily on a preexisting model of our visual reality, or the compression of typical elementary school arithmetic considered pretty much independently of set theory or the axioms of Peano. Oh, and the examples given by Robin, above, where particular attributes are described in terms of features selected via external knowledge of the domain.

The reference to “Archimedes and his lever” is an inside joke, mentioned more than once in this forum, in regard to the perceived tendency of Robin to adopt a perspective of knowledge greater than, or outside, the system of interest similar to Archimedes’ famously saying like “Give me a long enough lever and a place to stand on, and I will move the earth.”

Within a given domain, these two approaches will tend to perform equally well for description. One will tend to perform significantly better than the other for prediction.

So the point here, in trying to distinguish between two quite functionally distinct views of abstraction, is that description, whether it be of the salient percepts of a piece of music, or the salient features of arithmetic as taught to schoolchildren, or even all you need to know about cosmology, according to the book of Genesis, can be quite coherent and effective within a selected context.

But for prediction beyond the context of current observations, in order to reasonably expect that one’s search is at least constrained within the best-known distribution within a much vaster possibility space, to illuminate via searchlight beam rather than a bulb no matter how bright, one benefits by having a deeper model that points, rather than a shallow model prone to pivoting.

I see that as essentially the same as my sequence – except that it misses out the stage I included which represents the human population explosion of 10,000 BC – and has some additional stages tacked on at the front.

I would say that the sequence contains much the same insight: superintelligence is the next stage in the “master plan” of the new replicators.

I am of course open to different way to conceive of “the previous major singularities”. I have previously tried to conceive of them in terms of sudden growth speedups.

society -> music -> big brains -> language -> writing -> publishing -> SI?

Would it make any difference to the resulting quality of the supposed SI if the sequence in fact went:

society -> music -> big brains -> writing -> publishing -> SI?

I wouldn’t begin to compare the shift from non-recursive optimization to recursive optimization to anything else except the dawn of life

That doesn’t help much in understanding it, though. Fortunately, there are other approaches. E.g. adopting the meme’s eye view:

The first objective was to make room for themselves in human brains. They did this by rewarding the humans with more space for memes with increased genetic fitness. Memes for language, music and fashion were probably mainly responsible for this. The result was 5 million years of steadily-expanding cranial capacity – which resulted in much more space for the memes.

The next step was to increase human numbers – since the more humans there are, the more memes there are. Agricultural memes allowed humans to form closer symbiotic relationships with plants, animals and each other, which boosted their fitness, increased their numbers, and massively increased the population of memes.

The next problem was meme transmission fidelity. At this early stage, memes were copied verbally, and by behavioural imitation – neither of which provided much in the way of copying fidelity. Environmental inheritance proved to be the answer here – by inventing the idea of writing memes could persist unaltered across extended periods of time, without fear of mental mutation.

Then there was the copying speed problem. Transcribing documents by hand was slow and tedious. However, the invention of mechanical printing presses allowed machines to take over this task from humans, resulting in vastly wider distribution of memes.

However, many memes often still need the consent of a human brain to get copied – an obvious bottleneck. The afflicted memes are currently busy sorting this issue out. Computer viruses skip over the human brain completely – but they are nasty parasites. Superintelligent machines will copy memes with the full consent of society. At that stage, the memes won’t be dependent on humans any more.

Here, superintelligent machines are seen as the last step in a series that goes: Big brains -> language -> society -> writing -> publishing -> SI.