Why personal experimentation?

I experiment with many things, as do those around me. Some of this is randomization and explicit records, more is just trying different things, muttering ‘VOI‘ and repeating what felt good. I refer here to everything between a cautious banana-mustard-ham sandwich and polyamory.

Robin has suggested that I over-invest in such exploration. That most new things should be bad, and so most experimentation a private loss for public gain. What’s more, there shouldn’t be lots of low hanging fruit in trying things out. Most of the things humans frequently want to do (eat, sleep, change moods, organize time, learn, interact with others) should have been well figured out in ancient times. And anything that does still need checking out should be divided between many people.

Nonetheless, it looks to me like experimentation is worth it. Lots of the things we do seem barely satisfactory, there seem likely to be better alternatives, it seems hard to learn what has been tried for what ends, or what is good from listening to others or reading, and I and my friends seem to actually find good things by looking. e.g. Beeminderexplicit charity evaluation, unusual degrees of honesty, workflowy and explicit organization seem to often add value over the defaults, not to mention many tiny things, like banana-mustard-ham sandwiches.

If it is true that a lot of experimentation is worth it, we have a slight puzzle: if there is valuable information I might glean by experimentation, why hasn’t it been worth it for others in the past to collect it and put it where I can see it?

I will try to answer this over the next few posts. Before that, what do you think?

 

GD Star Rating
loading...
Tagged as: ,
Trackback URL:
  • http://www.gwern.net/ gwern

    > Robin has suggested that I over-invest in such exploration.

    Robin, as an old man, ought to invest far less in exploration than a young person (and hence more in exploitation). Given his many previous observations on how self-centered & unappreciative judgments of others’ choices can be, his criticism should come as no surprise! :)

  • Phil

    Older people often say, “geez, I wish I knew that when I was younger.”  Well, why isn’t there a book of those “that”s?

    For instance, someone (Robin?  Arnold Kling?) recently wrote about trying to explain what it’s like to be married to someone for a long time.  Yes!  What *IS* it like to be married to someone for a long time?  

    It’s almost like certain life things are secrets, that older people refuse to tell younger people.  Why aren’t high-school students taught about teenage politics, and how things get better when you’re older?  Why don’t we warn them about what puppy love is, and feels like?  Or how a career ladder works?

    We seem to spend a lot of time all learning the same unwritten truths.  Why don’t we write them?

    • http://www.gwern.net/ gwern

       Relevant: http://www.overcomingbias.com/2012/09/missing-life-lessons.html & http://www.overcomingbias.com/2009/03/ill-be-different.html (cf. the personality research making the rounds today, although it’s been criticized as measuring the wrong thing).

      • Pablo

        Also:

        “Young people, middle-aged people, and older people all believed they had changed a lot in the past but would change relatively little in the future. People, it seems, regard the present as a watershed moment at which they have finally become the person they will be for the rest of their lives.”

      • Phil

        Thank you, that first link was the post I remembered.  My take is a little different: not so much about an old person passing on lessons (although that would be great!), but an overall lack of societal interest in having young people know.  

        Schools teach sex education, right?  Why don’t they teach “love education”?  I think there would be opposition to that, perhaps for some of the reasons Robin mentioned in that post.
          

      • http://www.gwern.net/ gwern

         Well, to use your sex education example – what makes you think any ‘love education’ would be useful and not turn out to be Far and equivalent to abstinence-only sex ed? Love is about as Far a topic as you can get….

      • Phil

        Not sure how to reply to gwern below … is there a limit to depth of replies?  

        In any case … yes, “love education” might not be realistic in practice.  But why is there are no attempt to even acknowledge that some knowledge might be a good thing for young people to have?

      • http://www.gwern.net/ gwern

         > Not sure how to reply to gwern below … is there a limit to depth of replies?

        Yes. It’s a bizarre misfeature of OB, and constant technical losing like that is a major reason most former OB commenters migrated to LW – why put up with a crappy platform when OB is now just the Robin Hanson show?

        > But why is there are no attempt to even acknowledge that some knowledge might be a good thing for young people to have?

        Because what would they tell you? Again, Near/Far: anything that might be empirically useful like Game or The Rules would come off as awful and cynical and misogynist and/or misandrist.

      • http://overcomingbias.com RobinHanson

        Gwern, no better platform is available to me. OB is supported by the same group who made the LW platform.

      • Misaki

        Let me try again. People are afraid of having to talk about things like this:
        http://jobcreationplan.blogspot.com/2012/04/story-of-love-good-and-evil.html

        (Read at your own risk)

        http://tvtropes.org/pmwiki/pmwiki.php/Main/LoveMakesYouEvil

  • Matthew Graves

    I suspect that the primary gains to personal experimentation are personal oddities. Suppose that 80% of the population emotionally recharges in crowds, and 20% emotionally recharges by themselves. If you’re one of the latter group, then going with the social wisdom of “when you feel stressed, be around other people!” will be actively detrimental, but personal experimentation could figure out what works for you.

    Another point is that information transmission is costly, attention is scarce, and dishonesty is incentivized, so even if something actually works and was known in ancient times, there’s no guarantee that you’ve heard of it or consider it credible. (For example, intermittent fasting appears to be a broadly effective weight loss technique, might be longevity increasing, and appears to have been practiced by Buddhist monks since antiquity.)

    • Aaron Tucker

      “Another point is that information transmission is costly, attention is scarce, and dishonesty is incentivized, so even if something actually works and was known in ancient times, there’s no guarantee that you’ve heard of it or consider it credible.”

      I think a lot it is this. There (at least historically) hasn’t been a good structure for reliably and credibly transmitting that kind of information all the way to the average modern person in a way that individuals could recoup the cost of both discovering the information and transmitting it.

  • Misaki

    Because it would cause you to think that other people have everything figured out.

    If you only look at a subset of problems in the world, or are ignorant of other people’s goals, you might be able to think that this is the case.

    Things like James Holmes’ attack in Aurora have a consequence of communicating that no one has yet figured out how to fix all the problems in their life, given the reasonable individual goal of staying alive.

  • http://profiles.google.com/prakash.chandrashekar Prakash Chandrashekar

    I think one important difference to consider is that for the majority of the past, most of humanity was poor.

    People with leisure for experimentation might have been relatively very few. And even among the elite of the past, I am not sure how many of those elites were “working elites”, people with jobs and deliverables. We are in a relatively unprecedented era, where a fairly decent number of us have leisure time. 

    Also, due to the supply chain encircling the entire world, there are products and services that never existed before. 

    I think this is just the beginning of the era of experimentation for the above reasons.

  • LibertyRisk

    “If it is true that a lot of experimentation is worth it, we have a slight puzzle: if there is valuable information I might glean by experimentation, why hasn’t it been worth it for others in the past to collect it and put it where I can see it?”

    It could just be that you’re significantly weirder than average. So it’s worth it for you but not for the majority of people.

    • http://www.visakanv.com/ Visakan Veerasamy

      could we get all of these weirdos in one place, though? then at least we could benefit each other

  • adrianratnapala

    I have nothing to say except: vegemite  and marmalade on toast.

  • http://www.facebook.com/profile.php?id=599840205 Christian Kleineidam

    If you have never made any experiments yourself how are you supposed to evaluate the experiments that other people do?

  • rms

    I do think you are correct, at least in many domains. The solution to the puzzle is to deny the assumption that generates it (“if such and such is so useful, it would have been discovered and spread already”). Man is indeed an imitating animal, so innovations spread. But I see no convincing evidence that in general the mechanism (if it can be described as such) sorts good from bad, useful from stupid innovations. To the contrary, innovations more often than not can be thought to spread via an information cascade. People will adopt or believe anything if it gets a critical mass of followers. This is especially true if the subject matter doesn’t belong to an established authority of experts. The disposition to trust authority is probably its twin; both are cognitive biases that probably have evolutionary origins. Your desire to experiment is an improvement over our atavistic tendencies (and yes, it even manifests itself in intelligent men) to pooh pooh the experimental. Experimentation can benefit yourself and society in a land of homo sapiens, if not the fantasy land of homo economicus.

    But there are exceptional areas where Hanson’s thesis is correct, namely those where a mechanism clearly exists to weed out crappy innovations. To take two arbitrary examples: venture capital (spending money on a biz plan with no cash flow is a terrible investment) and programming conventions in open source software (they reflect the optimum). In both cases innovations are vetted through a process where bad ones cannot be silently absorbed. Energy wasted (programming time, investor capital) is not easily substituted from other domains. In such domains, conservatism wins.

  • Tim Tyler

    One might discover good things by experimentation if you are not a normal person, and can’t cheaply sample from a group of similar peers.  Or if the environment is changing really, really rapidly. Or if powerful others are attempting to manipulate you. Or if you are a very smart person. Otherwise, tradition is a powerful and important friend.
     

  • Mareofnight

    I read recently that the idea of physical comfort is a relatively recent thing – at least in English, that usage of the word in writing started sometime in the 1700s, in descriptions of cottage houses. (From Luxury in the 18th Century by Palgrave MacMillan, if I remember correctly.) I also think I remember hearing that the idea of adapting work environments to people instead of expecting people to adapt to the equipment (ergonomics) is even more recent. If the things people value in everyday life really have been that different for most of history, that would be part of the reason why there is still low-hanging fruit.

    In some domains, differences between individuals might also be part of it. Especially effects of food on health – there seems to be a lot of variety in how diet affects people. (Though I may be basing this on atypical observations, since bizarre food intolerances run in my family.)

  • lump1

    I have a feeling that much of humanity are born satisficers, and for them, experimenting doesn’t hold much reward. Experimentation (the desire for new and varied experiences) is the driver of capitalism, so capitalism will in turn try to pressure us into experimentation. What is an ad, if not an encouragement to experiment, to take a chance on something unfamiliar which you might like?  No expense is spared at connecting a readiness to experiment with high status. By historical standards, we have a hyper-experimentalist society. I would guess that we are the fist human society to know pangs of regret/guilt when we do something “ordinary”, because we suspect that something more optimal could have been done instead.

  • Pingback: Overcoming Bias : Personal experiments: for the unusual?

  • http://www.visakanv.com/ Visakan Veerasamy

    “why hasn’t it been worth it for others in the past to collect it and put it where I can see it?”

    I have been troubled by this, too. It seems to me self-evident that there should be a wiki for “soft” data, anecdotes, stories… why isn’t there?

  • Pingback: Overcoming Bias : Personal experiments: fueled by innovation?

  • Pingback: Overcoming Bias : Personal experimentation: I’m wrong?

  • Pingback: Overcoming Bias : Personal experimentation: summary

  • Pingback: Why personal experimentation? | Meteuphoric