The surprising power of rote cognition

Even if you're familiar with the ideas that are presented on this blog, it can be surprising just how strong the forces of habit and rote cognition and behavior can be.

One of the schools of cognitive psychology that addresses biases describes "system 1" and "system 2" thinking, where "system 1" is everyday automatic processing, deciding by intuition, relying on heuristics, and totally filled with biases, and "system 2" is thoughtful and careful consideration, logical and methodical. But this seems inadequate, because we can slip into automatic cognitive patterns even when we are consciously trying to be careful.

A few examples from personal experience below the fold…

I experienced this directly when playing the gatekeeper in an AI-box experiment arranged with another OB reader. I agreed not to give the details of what happened, but I think I can say this much without violating that agreement: I found my own mind responding to the person playing the AI as if he were actually an AI. Even though I knew it was a person who was playing the role of an AI, and even though I knew it was perfectly permissible to completely disregard everything my partner/opponent was saying and recite Blake poetry at him, something in the fact of the assigned roles — some kind of automatic suspension of disbelief — caused me to think as if there was actually an AI on the other end of the internet. I still won, but it was surprisingly difficult.  I think that surprising difficulty came about in part because I responded in that way: I took what my partner was saying <i>seriously</i>, even though there was no need to do so. 

Similarly, I was recently on the receiving end of a bizarre bit of rote behavior. Someone did something obnoxious, I caught him and called him out, and he tried to lie his way out of it with the classic phrase "I have no idea what you're talking about." I'm pretty sure nobody has ever uttered that phrase honestly — it's like something that's culturally encoded in us: someone calls us out on something, we activate the Outraged Denial Subroutine, and the phrase "I have no idea what you're talking about" flies out of our mouths (or out of an e-mail, which took some time to write, from an intelligent person!). (Dear readers: ask yourself whether you have ever used that phrase sincerely.) But do we believe it by rote too?

A third example: I play a lot of chess, particularly three-minute speed chess on FICS. At that speed, there's no time really to calculate long lines of moves, there's just time to play tactical swipes that you see (for non chess-players, a tactic is a short series of moves that are meant to achieve some kind of advantage, usually material — like attacking two of your opponent's pieces at once). There's also time to follow intuitions: to move pieces into positions that you know are strategically good, or play intuitive attacks (sacrificing material when the enemy king looks vulnerable, for example). Chess is a favorite game for studying bounded rationality and the ways that skilled versus unskilled players think — Herbert Simon influentially thought that the difference between grandmasters and ordinary patzers mostly was in the number of "chunks" — of memorized positions — accessible to immediate cognition. But it seems like there's more, in this sense. I notice in three-minute chess that I have tactically good periods and bad periods, even thought my overall (low) level of skill is presumably the same throughout, and even though I am trying just as hard. Sometimes, I play by pure intuition, and sometimes I have a calculation capacity available to me and can perform tactics, and there's no visible difference between those times, except that I lose a lot in the first case and win a lot in the second.  Chess players talk a lot about stamina, and this might be what they mean: the mental discipline to play well might just run out after a while.

This is all very strange, and it makes me wonder if we can think properly even if we try. How much of rationality — of being a good Bayesian Ninja or whatever — isn't about intelligence, or knowing how to think, but about having the self-control and discipline to exercise those capacities? And what does it mean for our attempts to become more rational if, as a lot of recent psychological research has been suggesting, our self-control generally is a limited resource?

How can we overcome rote cognition, if it sticks around even when we're trying our best to be mentally alert and careful?

GD Star Rating
a WordPress rating system
Tagged as:
Trackback URL:
  • anon

    I have used that phrase in complete honesty.

  • Constant

    I have no idea what you’re talking about.

  • http://profile.typekey.com/paultopia/ Paul Gowder

    Uh, that should be suspension of disbelief. Jeez.

  • Tom

    How can we overcome rote cognition, if it sticks around even when we’re trying our best to be mentally alert and careful?

    Bayesian Wannabe: “Roshi, I long to master the skills of the beisutsukai, but I fear my self-control is a limited resource.”

    Master: “You rack disicprine!” strikes wannabe with a keisaku

  • http://explorerstreet.blogspot.com James

    How much of rationality […] isn’t about intelligence, or knowing how to think, but about having the self-control and discipline to exercise those capacities?

    […]

    How can we overcome rote cognition, if it sticks around even when we’re trying our best to be mentally alert and careful?

    I think practise is important. I think it’s a skill-based capability – you get better at invoking that capability through accumulated effort. You can’t do it well straight away, and you certainly can’t do it well by simply trying really hard when you go to perform it. Crucially, this means that you can’t only try and do it when you feel like you’ve got an important problem to solve or important situation to deal with. You’ve got to be trying to do it all the time, through the important and the trivial.

  • Zack

    I have also used that phrase sincerely.

  • http://profile.typekey.com/paultopia/ Paul Gowder

    I now realize that the problem with the “have you used the phrase sincerely” question is going to be sampling bias, since obviously those who use it only insincerely won’t comment.

  • Julian Morrison

    It doesn’t surprise when you realize the relative size and computing horsepower of systems 1 and 2.

    Put it this way: system 1 is everything you have in common with a chimp, and most of what you have extra.

  • Less Anonymous

    How much of rationality — of being a good Bayesian Ninja or whatever — isn’t about intelligence, or knowing how to think, but about having the self-control and discipline to exercise those capacities? And what does it mean for our attempts to become more rational if, as a lot of recent psychological research has been suggesting, our self-control generally is a limited resource?

    It means we’ll be forced to acknowledge that there are a few more low-hanging apples in the cognitive enhancement tree aside from caffeine (e.g., ritalin, modafinil), and the only thing currently restricting their availability to those with abnormal cognitive deficiencies is a social taboo against drug use by “healthy” people.

  • http://t-a-w.blogspot.com/ Tomasz Wegrzanowski

    I did say “I have no idea what you’re talking about” (with a wtf somewhere added in it) with full honesty just a few days ago. I still have no idea wtf it was about. I made a few guesses of what it was about, but none of them made any sense, so that’s what I just said.

  • Aurini

    I’ve always thought that the majority of the time, we’re all on autopilot, not really thinking, just following developed subroutines. It is rare for moments of ‘sentience’ to occur more than once a week(forgive the terminology, this was my thinking five years ago).

    I’ve gone to great lengths to reprogram the subroutines, with a decent ammount of success, through rote practice of modifying behaviours patterns. Do you think the same could be done for biases? Maybe not always being rational, but creating bulwarks against common irrationalities? And would that be sufficient?

  • http://topologicalmusings.wordpress.com/ Vishal

    Paul,

    I wonder if you would make public your FICS handle to the readers of this blog! Some of us might enjoy playing a few blitz games with you. I also wonder how many other blog authors and readers play chess on FICS.

  • http://Mascotts.ca Mascott

    If I always think as I always thought I will always get as I always got. Rote thinking is the prejudiced mind that is stuck in reaction.
    Remember this: I Love You and Accept You Even when I need to Understand You Better. Say that to the person in the mirror and then to an other you may need to know seperate from that ROTE/Prejudiced thinking.

  • http://profile.typekey.com/paultopia/ Paul Gowder

    But then you all would get to see my humiliating rating, and all my cred as a rationalist would go out the window!

  • C

    This is only a guess, but it seems the first step towards overcoming rote behavior is to recognize it when it happens. I can think of two possibilities, though both have flaws:

    The first is to develop the habit of constantly questioning your immediately preceding actions or words. If you do this enough, perhaps it will start to come easily, as this habit will itself become a rote behavior. So if you then catch yourself saying, to use the above example, “I have no idea what you’re talking about”—and yes, I admit to having used the phrase insincerely—take the immediate step of correcting yourself. “Wait, that’s not true, you’re right, I do know. Ya got me.” (That assumes, of course, that in hindsight you’ll wish you’d been truthful.)

    Second, keeping in mind that self-discipline is indeed a limited resource, wait until you’re in a more relaxed setting and then review in your mind the instances where you’ve acted out of thoughtless habit and later regretted it. Recall the situation as vividly as possible, and think to yourself how you would have acted if you had kept an open mind and avoided the cliched response. Then visualize yourself harnessing that mindset the next time you need to react quickly.

    I doubt it’s possible to eliminate rote behavior completely, but maybe if you do these things, you’ll reduce its control over some of your actions.

    Of course, part of the problem is that predicting the situations where you’ll lapse into rote behavior can’t be done. And my guess is that new, unpredictable and therefore uncomfortable settings are those where you’re most likely to react without deliberation. Not sure how to overcome this…