Comments on: Reward Robustness

By: Overcoming Bias : Space vs. Time Allies

Overcoming Bias : Space vs. Time Allies — Wed, 08 Jun 2011 03:33:12 +0000

[...] Fragility – The more that the parts of a civilization depend on one another, the more that damage to one part can put the whole at risk. In a time stretched civilization a very bad outcome [...]

By: Making the Human Brain Stronger; A Theory

Making the Human Brain Stronger; A Theory — Thu, 04 Feb 2010 09:06:44 +0000

[...] [...]

By: Anders Sandberg

Anders Sandberg — Wed, 03 Feb 2010 20:01:04 +0000

Just noticed a correspondance to Nature (http://www.nature.com/nature/journal/v463/n7281/full/463608b.html, Could a boom in technologies trap Feynman’s simulator? by Dietrich Leibfried) that suggests another reason to get substitutability: the need for certain robust technologies even without the disaster. In the letter Leibfried notes that we may need several different technological implementations of quantum physics simulators to make sure they are giving accurate results. In the case of internet standards, there has been the tradition of demanding at least two different implementations of the same proposed standard, so that there is good reason to think it works.

I’m not certain this technological demand is general enough to affect all of society, but maybe one could tweak standardization body rules and public call for bids to favour technologies with multiple implementations.

By: Michael Metcalf Bishop

Michael Metcalf Bishop — Tue, 02 Feb 2010 16:07:33 +0000

I would love it if Robin elaborates his explanation for replying to multiple people in one comment. Even if this is a pure-status-grab, I have a hard time complaining about someone making such moves on the fantastic free blog they write.

By: mjgeddes

mjgeddes — Tue, 02 Feb 2010 12:11:28 +0000

But most trades are settled before the event occurs. I routinely trade on Betfair and the profit/loss is credited to my account right away – I don’t have to wait for the event to occur.

By: anon

anon — Tue, 02 Feb 2010 12:00:39 +0000

why can’t simple trades work even for long-term bets on an existential disaster?

Because prediction markets need to be adjudicated after the event occurs, otherwise they don’t work. Unless the contracts happen to be denominated in paperclips, it’s quite difficult to settle them after an unfriendly AI has taken over the world.

By: mjgeddes

mjgeddes — Tue, 02 Feb 2010 11:20:29 +0000

You can start making money off prediction markets immediately, no matter how far into the future an event is, or what its consequences are. The probabilities shift up and down as more people put money on, and you simply ‘Back’ (bet for the event) at a high price, and ‘Lay’ (bet against the event) at a lower price. Simple trading, why can’t simple trades work even for long-term bets on an existential disaster?

Example:

Initial Market on Unfriendly AI eating the world by 2020:

For $1 investment
Day 1 Market – Best Prices

Back: $100 Lay: $112

Ignorant punters have placed bets after Day 1 , I see that they’ve underestimated the odds and I ‘Back’ for big money.

Day 2 Market – Best Prices

Back: $20 Lay: $23

A lot more money has gone on, some experts on AI have placed their bets, and the odds are now more reasonable. I lay.

Result: Backed at $100, Layed at $23. Profit after only a day.

By: Wei Dai

Wei Dai — Tue, 02 Feb 2010 08:37:25 +0000

komponisto, Robin had previously expressed an interest in finding low cost ways to avoid high-status moves. See this post.

By: Tweets that mention Overcoming Bias : Reward Robustness -- Topsy.com

Tweets that mention Overcoming Bias : Reward Robustness -- Topsy.com — Mon, 01 Feb 2010 21:47:31 +0000

[...] This post was mentioned on Twitter by stefano bertolo, Christopher Patton. Christopher Patton said: http://j.mp/cpatton Reward Robustness http://bit.ly/9BWT1S [...]

By: Tyrrell McAllister

Tyrrell McAllister — Sun, 31 Jan 2010 19:50:58 +0000

Human conversation adaptations do well at not only helping us infer what others know, but also what they know that others know, what they know others know that still others know, and so on up the meta-knowledge hierarchy.

Yes, the human brain is good at keeping track of the threads of a conversation. But it’s not infinitely good. Explicit threading is a technological assist that helps the brain keep track of threads even better.

Analogously, human brains are good at keeping track of their associates, their family, friends, and acquaintances. But we still benefit from having address books.