Personal experimentation: summary

I asked how it could be that experimenting in my own life could be worthwhile, given that if such things were worthwhile other people should have already figured them all out. My suggested explanations:

  1. I am strange: nerdy, recent, young
  2. Innovation: there are constantly new things to experiment with
  3. Findings are not spread: or so much noise is also spread that the information is lost
  4. Context-specificity: your findings don’t apply to me, because people are unique or situations are complicated
  5. I am wrong: it’s easy to underestimate nebulous costs, to overstate fleeting or illusory benefits, to want to be the kind of person who tries new things, or to be too hopeful that life can improve fast

It seems to me that 3 is the biggest: results are collected so badly as to be often worthless and are aggregated poorly. It’s not clear to what extent this is because of 4: other people’s findings are just not so useful. Personal experimentation seems worth it even without good aggregation, but probably only if you avoid the same errors of measurement yourself. It could be worth it even with purely placebo gains, if you enjoy the placebo gains enough. But in this scenario, the gains are much smaller than you imagine, so you are probably over-investing a lot. There also seems to me a real risk that everything is so context specific that what you learn will be worthless as soon as you change many other things (4).

Explanations that involve others finding experimentation a lot less worthwhile (e.g. 1) seem unlikely to help much because it looks like others often find experimentation worthwhile. The problem seems to be somewhere between others making such efforts, and me having useful information as a result. Innovation (2) seems a bad explanation because it doesn’t explain the lack of information about age-old lifestyle questions. It seems likely that I have overestimated gains relative to losses in the past (5), but gains still seem larger than losses (it’s hard to disentangle causes, but my lifestyle has obviously improved substantially over the last  year or more, some of which seems directly attributable to purposeful experimentation and the rest of which seems at least not terribly damaged by it).

GD Star Rating
Tagged as: ,
Trackback URL:
  • Paul Crowley

    It’s telling that you even have to ask. Either someone has already had this discussion and you just haven’t found it yet, in which case it must be pretty difficult to find, or they haven’t, in which case there’s a kind of thinking you’re bringing to personal experimentation that hasn’t been brought to it before.  Either would be a big pointer to why you find it rewarding.  You don’t even list “I am actually unusually well placed to think about this among everyone who has ever lived” as an option, but I think there might be enough evidence to meet the strong prior against it.

    My Exhibit A for “all of humanity can fail to act on what in hindsight seem like really clangingly obvious good ideas for a very long time” is the fact that the beginnings of utilitarianism were proposed in 1725, but the efficient charity movement didn’t start until the 21st century.  That really makes me wonder what else of that magnitude we’re not doing today.

  • Ely Spears

    I still think the primary benefit of personal experimentation is signaling. A personal experimenter gets to look cool, innovative, willing to tinker, willing to shirk conventional wisdom, etc. If your experiment has even marginally successful results, even just for you in a non-generalizable way, you usually gain and it’s easy to publicize (think Jared from Subway) at least in some small circle of your associates if not among a wider group. Since people have well-known lottery biases, it suggests they would ignore the large probability that the experimentation won’t work and focus intensely on the small probability of looking like a genius. We’re also talking about relatively small costs and small gains, which people are probably also poor at assessing. When it comes to big gains, we mostly just copy what successful others have done.

    In my view, this seems far more compelling than the options you list. I also think that ingenious ways of looking like you’re not a copier while still just copying would be among the most frequently copied innovations.

  • Despite the length of this series, I don’t think you’ve defined the problem. Surely it isn’t “Is personal experimentation sometimes worthwhile?” It seems more like “Do I experiment excessively?”

    I submit that you haven’t formulated the question adequately because to do so would make still more obvious that the question is unanswerable except concretely: what experiments do you perform and with what results? The vagueness of your topic seems the result of your conflict between wanting to discuss the issue publicly and wanting to avoid disclosing any details. (Are we to accept that your “life has improved as a result” without even being informed of the putative improvements and the experiments that you think promoted them?)

    I see one way of dealing abstractly with something like you have in mind. You might have asked whether there’s a universal bias toward insufficient or toward excessive experimentation. But this would require a more scientific or philosophical approach, if you will, and it wouldn’t satisfy your desire to discuss the details of your life—while withholding them.

    But I can offer a hypothesis about why “nerds” experiment. Nerds fall toward the autistic side of the personality spectrum and are consequently averse to breaking routine and resistant to change. In this light, excessive experimentation is a reaction formation. (

  • I confess that this series of posts mostly leaves me baffled. When you talk about “personal experimentation” I can’t tell if you mean “buying organic vegetables” or “hooking up with strangers for bondage sex.” Without a sense of what you’re risking and what kind of reward you’re seeking, there’s no way to tell what the tradeoffs are.

  • zz

    This is sufficiently vague as to be meaningless. Of course some personal experimentation is good, and some is bad; despite 4,500 posts on the topic I have yet to see the topic of what is the right level of personal experimentation addressed in a systematic way, or even what categories of personal experimentation are likely to work.

  • Matt

    What about bounded rationality?  Specifically, inconsistent beliefs about the rationality of others, or Level-k thinking.  A sees B do X, and rather than recognizing that B does X due to personal experimentation or learning from others who have experimented, A believes that B acts randomly or naively.  Hence, the conventional wisdom that people learn better from their own mistakes than from the advice of others.  Young people have inconsistent beliefs about the rationality of their more experienced elders.

  • Anon

    Perhaps Katja Grace should personally experiment with writing something interesting.

  • VV

    So this series of posts is finally over?

  • Lord Liege

    I certainly find 2 and 4 prevalent.  A reason for less experimentation is 3 is mostly false. Mostly though, it is a dynamic process.  Most experiments are unsuccessful.  There is a trade off between known goods and unknown mostly bads but an occasional better.  An increase in success at finding betters will encourage more experimentation, a decrease less.  The more satisfaction with the current state, the less likely to pursue experimentation and the less successful it will be.  The more dissatisfied, the more reward from it and the more likely for it to be successful.  

    • Lord Liege

      Experimentation can keep you from being bored and renew your appreciation of what you already have which is why it doesn’t disappear.  A lot of it that is unsuccessful can lead you to re evaluate existing dissatisfaction.   

  • Pingback: Personal experimentation: summary | Meteuphoric()