Say Not "Complexity"

Once upon a time...

This is a story from when I first met Marcello, with whom I would later work for a year on AI theory; but at this point I had not yet accepted him as my apprentice.  I knew that he competed at the national level in mathematical and computing olympiads, which sufficed to attract my attention for a closer look; but I didn't know yet if he could learn to think about AI.

I had asked Marcello to say how he thought an AI might discover how to solve a Rubik's Cube.  Not in a preprogrammed way, which is trivial, but rather how the AI itself might figure out the laws of the Rubik universe and reason out how to exploit them.  How would an AI invent for itself the concept of an "operator", or "macro", which is the key to solving the Rubik's Cube?

At some point in this discussion, Marcello said:  "Well, I think the AI needs complexity to do X, and complexity to do Y -"

And I said, "Don't say 'complexity'."

Marcello said, "Why not?"

I said, "Complexity should never be a goal in itself.  You may need to use a particular algorithm that adds some amount of complexity, but complexity for the sake of complexity just makes things harder."  (I was thinking of all the people whom I had heard advocating that the Internet would "wake up" and become an AI when it became "sufficiently complex".)

And Marcello said, "But there's got to be some amount of complexity that does it."

I closed my eyes briefly, and tried to think of how to explain it all in words.  To me, saying 'complexity' simply felt like the wrong move in the AI dance.  No one can think fast enough to deliberate, in words, about each sentence of their stream of consciousness; for that would require an infinite recursion.  We think in words, but our stream of consciousness is steered below the level of words, by the trained-in remnants of past insights and harsh experience...

I said, "Did you read A Technical Explanation of Technical Explanation?"

"Yes," said Marcello.

"Okay," I said, "saying 'complexity' doesn't concentrate your probability mass."

"Oh," Marcello said, "like 'emergence'.  Huh.  So... now I've got to think about how X might actually happen..."

That was when I thought to myself, "Maybe this one is teachable."

Complexity is not a useless concept.  It has mathematical definitions attached to it, such as Kolmogorov complexity, and Vapnik-Chervonenkis complexity.  Even on an intuitive level, complexity is often worth thinking about - you have to judge the complexity of a hypothesis and decide if it's "too complicated" given the supporting evidence, or look at a design and try to make it simpler.

But concepts are not useful or useless of themselves.  Only usages are correct or incorrect.  In the step Marcello was trying to take in the dance, he was trying to explain something for free, get something for nothing.  It is an extremely common misstep, at least in my field.  You can join a discussion on Artificial General Intelligence and watch people doing the same thing, left and right, over and over again - constantly skipping over things they don't understand, without realizing that's what they're doing.

In an eyeblink it happens: putting a non-controlling causal node behind something mysterious, a causal node that feels like an explanation but isn't.  The mistake takes place below the level of words.  It requires no special character flaw; it is how human beings think by default, since the ancient times.

What you must avoid is skipping over the mysterious part; you must linger at the mystery to confront it directly. There are many words that can skip over mysteries, and some of them would be legitimate in other contexts - "complexity", for example.  But the essential mistake is that skip-over, regardless of what causal node goes behind it.  The skip-over is not a thought, but a microthought.  You have to pay close attention to catch yourself at it.  And when you train yourself to avoid skipping, it will become a matter of instinct, not verbal reasoning.  You have to feel which parts of your map are still blank, and more importantly, pay attention to that feeling.

I suspect that in academia there is a huge pressure to sweep problems under the rug so that you can present a paper with the appearance of completeness.  You'll get more kudos for a seemingly complete model that includes some "emergent phenomena", versus an explicitly incomplete map where the label says "I got no clue how this part works" or "then a miracle occurs".  A journal may not even accept the latter paper, since who knows but that the unknown steps are really where everything interesting happens?  And yes, it sometimes happens that all the non-magical parts of your map turn out to also be non-important.  That's the price you sometimes pay, for entering into terra incognita and trying to solve problems incrementally.  But that makes it even more important to know when you aren't finished yet.  Mostly, people don't dare to enter terra incognita at all, for the deadly fear of wasting their time. 

And if you're working on a revolutionary AI startup, there is an even huger pressure to sweep problems under the rug; or you will have to admit to yourself that you don't know how to build an AI yet, and your current life-plans will come crashing down in ruins around your ears.  But perhaps I am over-explaining, since skip-over happens by default in humans; if you're looking for examples, just watch people discussing religion or philosophy or spirituality or any science in which they were not professionally trained.

Marcello and I developed a convention in our AI work: when we ran into something we didn't understand, which was often, we would say "magic" - as in, "X magically does Y" - to remind ourselves that here was an unsolved problem, a gap in our understanding.  It is far better to say "magic", than "complexity" or "emergence"; the latter words create an illusion of understanding.  Wiser to say "magic", and leave yourself a placeholder, a reminder of work you will have to do later.

Comments

[Felix]

Quote: "We think in words, "

No we don't. Apparently you do, though. No reason to believe otherwise. :)

Please keep up these postings! They are very enjoyable.

Going back to "explaining" something by naming it (from a couple of your earlier posts):

e.g. Q: Why does this block fall to the floor when I let go of it? ... A: Gravity!

I always thought that such explanations were common side-effects of thinking in words. Sort of like optical illusions are side-effects of how the visual system works. Perhaps not. One does not need to use words to think symbolically. There are, after all, other ways to do lossy compression than with symbols.

Anyway, I'll still assert that it's easier to fall for such an "explanation" if you think in words. ... An easy assertion, given how hard it is to count the times one does it!

[Rick Smith]

Aren't we understating the role of labels in brevity here?

Where the labelled thing is understood well enough by the labeller and listener or of trivial importance to the problem domain, don't labels contribute to cognitive economy?

I'd have said when you need to get things done, fear of wasting time is desirable rather than deadly.

[Valter]

Actually, the "emergence" and "complexity" pseudo-causal explanation are much worse than Felix's "gravity" example: the answer "Gravity!" does explain the fact that the block falls to the floor by noting that it is a specific instance of a general phenomenon for which we have very precise information on how it works (attraction force is constant x m1 x m2 /d^2). We may not know why gravity exists, but that is a different (higher level?) problem.

In the case of "emergence" and "complexity", we just don't know.

P.S.
I do think that "emergence" is a useful concept to describe situations where modelling is more conveniently done at a (more) aggregate level, but that's yet another story.

[Hopefully Anonymous]

I don't think these parable posts convey information efficiently to the overcomingbias audience, but I like your point at the end. Specifically, I agree it's better to use placeholders that make lack of knowledge/understanding clear, rather than placeholders that seem to cover up such lack of knowledge/understanding.

[Nato Welch]

"Then a miracle occurs..."

;p

I wonder if memetics would serve as a good candidate for the category of things that satisfy without explaining or predicting anything, along with phlogiston, emergence, and complexity. The analogy to biology seems interesting and fun, but is it more useful than as just a way to re-formulate our perspective?

[Gray Area]

In computer science there is a saying 'You don't understand something until you can program it.' This may be because programming is not forgiving to the kind of errors Eliezer is talking about. Interestingly, programmers often use the term 'magic' (or 'automagically') in precisely the same way Eliezer and his colleague did.

[Bernard Guerrero]

Step 1: Steal Underpants
Step 2: ?????
Step 3: Profits!!!!

[Eliezer Yudkowsky]

Programming is not forgiving to the kind of errors Eliezer is talking about.

But it's a lot better to be unforgiving of yourself than to wait for reality to hit you over the head with it. It's better to notice in 10 seconds that you don't understand something, than to realize this only after 20 people spend 5 years and $10 million of venture capital and the "emergent behavior" you pinned your hope on fails to materialize. It's all too easy to program "chaos", "complexity", or "emergence", so long as you tell yourself that you need to program more of it before you reach Step 3 and Profit.

[Tom McCabe]

"That was when I thought to myself, "Maybe this one is teachable.""

*How* many people have asked you about becoming an AGI designer? It sounds like you have a good deal of experience with rejection, even after weeding out the obvious crackpots.

[Barkley Rosser]

Well, this is partly a matter of what discipline one is dealing with. So, sure, for AI or computer science more generally, Kolmogorov or Chaitin or Rissanen measures are more useful and reasonably well defined. For other disciplines, other definitions may be more suitable. Thus for economics, I have (following Richard Day) defined complexity in a dynamic way based on erratic dynamics appearing endogenously out of the system (with "erratic" defined more specifically). I laid this out in a paper in 1999 in the Journal of Economic Perspectives, and have a more recent paper up on my website ("Computational and Dynamic Perspectives on Economic Complexity") comparing the two approaches, at http://cob.jmu.edu/rosserjb.

[Benjamin Zivan]

As a current student, I can confirm your suspicions about a seemingly complete paper being preferred over one that addresses all information about the topic. "I don't know" still is not an acceptable answer in many circles and I regard it as an unfortunate phenomenon.

[Bill Harris]

Gray Area wrote, "You don't understand something until you can program it." As somewhat of an aside, Randy MacDonnell has written about APL and J (http://facilitatedsystems.com/weblog/2007/07/if-you-can-say-it-its-done.html), "If you can say it, it's done."

On a different note, I took a pair of summer high school mathematics courses sponsored by the NSF years ago at the University of Miami. One professor, a Dr. Hermann, I seem to recall, said he often imagined himself wearing "worry beads" and fingering them as he spoke. If he fingered the beads nearer his neck, he was speaking more precisely; if he fingered those nearer his waist, he was speaking less precisely. In reality, he wore no beads, but he did, on occasion, finger imaginary beads as he was explaining certain concepts.

Perhaps the same thing can be adapted here to indicate the level of magic in claims we make. If we finger imaginary beads near our necks, we claim we know what's going on; if we finger those nearer our waists, we admit there's magic here.

[Nathan]

Forgive me for latching onto the example, but how would an AI discover how to solve a Rubik's cube? Does anyone have a good answer?