31 Comments

Well, interestingly, Stanley Kubrick once said:

“Among a great many other things that chess teaches you is to control the initial excitement you feel when you see something that looks good. It trains you to think before grabbing, and to think just as objectively when you’re in trouble. When you’re making a film you have to make most of your decisions on the run, and there is a tendency to always shoot from the hip. It takes more discipline than you might imagine to think, even for thirty seconds, in the noisy, confusing, high-pressure atmosphere of a film set. But a few seconds’ thought can often prevent a serious mistake being made about something that looks good at first glance. With respect to films, chess is more useful preventing you from making mistakes than giving you ideas. Ideas come spontaneously and the discipline required to evaluate and put them to use tends to be the real work.”

Expand full comment

Poker is a decent alternative, but Go is already a much better chess than chess is, according to the given criteria.

It's only achieved much of that in southeast Asia, though, which is a majority of the world's population, but not cultural influence, with Manga/Anime probably it's highest world-wide appeal.

Still, it already has a tradition as a game to be played both by professionals for multi-million dollar prizes as well as serious businessmen, etc... as a way to learn more about each other, so of any game, I'd say its clearly the closest currently to the desired ideal. It used to also have status as the only major game computers couldn't beat humans at, but that's recently fallen.

Expand full comment

You should also fix your game design so it teaches a slightly smug liberal lesson.

Expand full comment

Phil Tetlock and colleagues have run long forecasting tournaments that show how to detect, reward and teach reducing overconfidence, learning from others, and being willlng to update. Those skills are key elements of this challenge's "brutal honesty." Turning their approach into a more playable game is a challenge. Nevertheless, I suspect any winner of Hanson's challenge can learn a lot from Tetlock's approach.

Expand full comment

I'd second a roundup of game design literature. Sounds fascinating!

Expand full comment

Ah yes, the usual literature on game design. Which of course all of us know. However, just in case some confiding person, perhaps very young, were to glance through this discussion some time, perhaps we might refer to a roundup

Expand full comment

I guess that the biggest difference between chess and the "real World" is that chess is a game of perfect information while in the world we have to act with imperfect information. I consider the idea of bidding chess (https://arxiv.org/abs/0903.... as an good way to improve Relevant,while keeping the others roughly on the same level.

Expand full comment

Chess players of a high caliber will be extremely cautious about not making mistakes; usually the higher your chess level, the more you hate losing, and the more afraid you are to make certain types of mistakes. You will develop a strong desire for a feeling of completeness in your strategies and, over years, this will translate into other domains of your life. That's why you see many chessplayers sticking to limited, but robust, opening repertoires instead of striving for adventures, as their experience has taught them that knowing one's turf well is much more important than trying to get some last-minute excitement of choosing a dubious, but more interesting, line (they have been punished often enough). In a sense, then, each game played and properly analyzed will make you more honest with yourself. The more such games played, the higher the rate of becoming more self-aware, or honest with yourself. Now, not every chessplayer at a level comparable with Kasparov's exhibits so much interest in so many areas outside of chess as he does. Actually, it's pretty safe to say, that relatively few do, with the rest just having fully committed to excelling at the game and only at the game. Myself, as a chess grandmaster, although not nearly at the top level, I value my chess experience greatly and clearly see how it gives me steam to climb other learning curves which have nothing to do with the game. The feeling that I may be making a mistake somewhere is essentially always there. As far as designing another game with the intention of increased rate of becoming more honest, I doubt such effort would yield any substantial improvement over chess, as for it to succeed it would need to be the precision of the game rules to be the contributing factor. And to my taste, and 20 years of experience of playing/ coaching chess, it's enough for the game to be complex enough to secure a life-long learning experience to humble the player and hence induce the feeling of weakness-awareness, or honesty. Even the current World Chess Champion and the strongest player of all times, Magnus Carlsen, likes to express his disappointment with his own level of play, and the truth is that he cannot realistically compete with computer programs. If Carlsen gets repeatedly humbled about his play, I simply cannot see along which trajectory a more honesty-inducing game could be constructed.

Expand full comment

Stalin intentionally falsified history. (See the work of the Dewey Commission - https://en.wikipedia.org/wi... ) Was he ruthlessly self-critical in the manufacture of lies? Quite likely. But this doesn't keep him from being an intellectual fraud.

The thesis being considered is that instrumental self-criticism promotes intellectual honesty. We have two issues. The first is how far cognitive habits generalize. But it now seems to me the more important point concerns the very nature of intellectual honesty, which I take to be part of an identity rather than a collection of instrumental habits. The nature of scientific integrity was described by Feynman in "Cargo Cult Science." (http://calteches.library.ca... )

[Added.]Stalin also promoted fake science - in particular, Lysenkoism - which he uncritically accepted.

Expand full comment

In short, yes. I think it's the rule that all principles that lead to success generalize -- even to those simply observing. For the actual practitioner it's a no-brainer.

At least that's been my experience.

And finally, we know Stalin was ruthlessly evil, but do we know he was a fraud in the sense we are talking about here? Stalin's overall performance as Dictator says no.

Expand full comment

Off the top of my head something like MUN or Mafia would accomplish a usefulness in social interactions when approached within a community and with the principles of competition laid out above.

Expand full comment

I question whether intellectual honesty acquired in one endeavor generalizes. Does becoming a ruthlessly self-critical chess player make it less likely that one will be an intellectual fraud? Stalin was reportedly a good chess player. (Perhaps not that good, but why not in principle?)

Put another way, I don't think a great chess players acquire the identity of being an honest intellectual. Honesty is instrumental to the chess master's competitive success, not an end in itself (as it is in science).

Expand full comment

I think honesty *to yourself* (or resistance to self-delusion) is pretty useful. That's what chess requires (if you want to be really good).

Expand full comment

The stock market could be seen as a big prediction tournament.

Expand full comment

Which raises the question: can we design a game that promotes even more useful honestly? [italics mine]

Outside of chess, how exactly is honesty of the sort required of chess players useful?

Expand full comment

Interesting idea. Certainly it is a useful skill to be able to get others to do something useful and give them good instructions. I am not as concerned as Frederic about the 'unfairness' of who you get, because both that can be one of the useful skills we train and because we could develop ratings for the 'executors'. We could even have two types of competition, with both 'instructor' and 'executor' being roles people can play and excel at - you form a team and have to work together.

We could develop a set of rules by which a victory condition is generated, and then it would be up to the instructor to get the executor to achieve that result, maximizing on things like clock time, amount of communication, money/hours spent by executor, and so forth.

You can even think of putting this on TV/stream, with cameras following the executor and instructor around. The Amazing Mechanical Turk?

It's more of a contest than a game, by default, especially if the goal itself doesn't involve anything especially game-like. But let's see where it leads.

Isolated could be an issue if things take a while to play out, but is fine if everything happens in mostly real time. If they happen over more time, we'll have to monitor or trust the players, or compromise this requirement.

Fair will depend on what the goal is, but also on how much you need the right cultural tools as part of the skill set we are testing. If the goals are set well, that end seems workable. If getting a good result from your executor (or being able to understand the instructor) is a culturally-detailed skill, that tells us about the real world skills we want to develop, and maybe we can reconsider our requirements? As long as a game requires allies some amount of cultural knowledge/skill will always be necessary/useful, and we are explicitly testing a somewhat cultural skill. I think we can live with it.

Fragmented pushes us towards smaller and real time goals. These are common on MT, so that should be fine, and Isolated also pushed in this direction, so we want our goals to be short, and the shorter the better as long as we achieve our other goals - ideally we should be able to run a contest in a 22-minute TV show at least once, and maybe multiple times, or we could follow a sports model and try for longer but more involved contests, perhaps involving more people, but I suspect it's very hard to get started with a new hours-long thing at this point, and the existing sports are somewhat grandfathered in here. Certainly more than 1-2 hours is death.

Status of some degree seems achievable, especially if you're making cool stuff happen in the real world, although that seems to have a cap (see Ken Jennings, say). You might be able to build on this by having tournaments of champions and other higher level play with longer and more ambitious goals, up to actually useful real world things being done for charity. That's more stuff we'd like to encourage, I'd assume.

Meanwhile, it would be a format people could practice at home, and compete against friends, on the cheap. You'd have to pay to play, since real people are being hired, or would you? If you developed a ranking system and awarded people status for high rankings as executors, perhaps with the chance to get on TV and otherwise compete if they do well partly via lottery, we might get volunteers for free. And again, if we're smart about this, perhaps there can be real world goals being worked towards the whole time, and/or sponsors, to pay for all that...

What do people think?

Expand full comment