Tag Archives: Experimentation

New Tax Career Agent Test

If that taxpayer approved, the taxes that he or she pays to the government could be diverted, and instead delivered to a “tax career agent“, who would have, in an auction, won and paid for the right to get such diverted payments from that particular taxpayer. For the government, this would like borrowing, i.e., a way to convert future tax payments into current revenue. This agent would now have incentives to advise and promote this taxpayer, but would have no unusual powers to influence this taxpayer’s behavior.

Previously I used a poll to estimate that career agents today who get 10% of client wages as a result raise those wages by 1.5% on average, suggesting that tax career agents (TCAs) who got ~20% of income might raise those same wages by 3%. But as this effect might be smaller for random workers, and as worker welfare gains would be less than wage gains, I estimated that TCAs raise worker welfare by ~1% on average, which at a real interest rate of 2% suggests a ~$20T present value to the world from adopting TCAs.

In my last post, I sketched a simple experiment design to test the TCA concept: give N random people TCAs, and track their income changes compared to N others who don’t. If TCAs raised wages by 0.3% per year, then given the usual random noise in wage changes, a ten year experiment with N=7000 seems sufficient, but an upper bound cost on this is ~$32M. Which is crazy cheap (~ a part in a million!) relative to TCA social value, but in our broken world we probably need something cheaper.

Here is my new concept: create a TCA for each worker, but get two auction prices per worker, one price if the TCA is active, i.e., free to promote and advise that worker, and a different price if the TCA is instead passive, i.e., prevented from helping this worker. Then randomly pick if the worker gets an active or passive TCA, and use the appropriate bids and prices to pick and charge the new TCA.

If there is sufficient competition in the bidding, then the difference between those two prices is a direct market estimate of how much bidders expect an active TCA to raise worker wages, minus the effort they expect an active TCA expect to put in to make this happen. This estimate is available per worker, and immediately at the experiment start. So even an N=100 experiment at a TCA expense cost of ~$1M for could give valuable data!

In addition to getting TCAs to estimate worker wage increases minus TCA costs, we might also want to get workers to estimate their welfare gains. And we could do this by putting workers into pairs, only one of which gets an active TCA, and making them bid against each other to see who gets that active TCA. Bids should give direct estimates of worker value (i.e., increased wages minus extra effort or inconvenience) if the winning bidder pays the lower bid price. These worker value estimates are also available per worker, and immediately at experiment start. And the extra revenue from worker bids cuts the cost of the experiment.

TCAs and workers would have strong incentives to make good estimates, but their estimates would still be based on pretty limited information. To get better informed estimates, it would help to spread this experiment out across time, and give later participants as much info as possible about earlier participant outcomes. The more time that elapses between the first and last TCA auctions, the more later participants will know, but the longer it will take to learn results from this experiment. Note that such a sequential approach also allows the experiment to better manage its expenses in the face of an initially uncertain costs per worker participant.

Here is a more detailed design based on the above concepts. Offer random workers a sufficient compensation (1% tax rebate?) so that most who are invited agree to participate for Y years. (If Y is short, pick post-college-age workers, so their choice of more schooling is less of an issue.) Participants allow substantial info on them, including their taxes, to be revealed to experimenters and other participants. Match participants into pairs who seem as similar as possible, then auction off these pairs one at a time in sequence over many years, showing all qualified bidders info on outcomes for all prior participants.

Each worker in each pair is asked for the bid B they would pay for a higher chance to be assigned the active, as opposed to passive, TCA for Y years. In addition, each pair auction has eight TCA auction prices, each qualified TCA bidder can bid on any or all of these eight prices, and the highest price wins each price auction, paying the second highest among its submitted prices. To prevent collusion within worker pairs, workers are given little info on their pair partners until they have set their bids.

The eight prices come from all combinations of three binary factors. First, there are the two workers, who will differ somewhat in their info. Second, there are different prices to become an active or passive TCA for Y years. Third, there are different prices depending on if the worker submitted the higher or lower bid to get the active TCA. Worker bids are kept secret until all eight TCA auction prices are set. Then the worker who bid more gets a 2/3 chance of being assigned the active TCA, and a 1/3 chance of being assigned the passive TCA. Given a bid B, we can estimate their added value V of having an active agent via V = 3B.

Note that at a 2% discount rate, the present value of 20% of the median US wage of $31K is ~$450K, 1% of which is ~$4.5K, implying a bid of ~$1.5K, an amount most workers can afford to pay.

This experimental design seems sufficient to extract key info re TCAs at a low cost. But it still needs more work. For example, we need tax experts to think about which parts of typical tax returns to include or not in TCA payments. We need finance experts to think about how to get sufficient numbers of competing TCA bidders, and how the experiment can hold and invest auction assets deposited, to minimize the costs and risks associated with paying off all TCAs as promised. We need labor experts to think about what worker info is sufficient to inform TCA bids. And we need legal experts to figure out how we can do all this within existing law. Any such experts want to help?

Added 20Nov: A similar test could be applied to my Buy Health proposal. For each possible patient get auction participants to bid on their price to provide health and life insurance separately, where different orgs provide the different types and aren’t allowed to coordinate, or as a bundle from a single org that can coordinate. See the per-patient estimated difference in death risk and medical spending.

GD Star Rating
Tagged as: ,

Personal experimentation: summary

I asked how it could be that experimenting in my own life could be worthwhile, given that if such things were worthwhile other people should have already figured them all out. My suggested explanations:

  1. I am strange: nerdy, recent, young
  2. Innovation: there are constantly new things to experiment with
  3. Findings are not spread: or so much noise is also spread that the information is lost
  4. Context-specificity: your findings don’t apply to me, because people are unique or situations are complicated
  5. I am wrong: it’s easy to underestimate nebulous costs, to overstate fleeting or illusory benefits, to want to be the kind of person who tries new things, or to be too hopeful that life can improve fast

It seems to me that 3 is the biggest: results are collected so badly as to be often worthless and are aggregated poorly. It’s not clear to what extent this is because of 4: other people’s findings are just not so useful. Personal experimentation seems worth it even without good aggregation, but probably only if you avoid the same errors of measurement yourself. It could be worth it even with purely placebo gains, if you enjoy the placebo gains enough. But in this scenario, the gains are much smaller than you imagine, so you are probably over-investing a lot. There also seems to me a real risk that everything is so context specific that what you learn will be worthless as soon as you change many other things (4).

Explanations that involve others finding experimentation a lot less worthwhile (e.g. 1) seem unlikely to help much because it looks like others often find experimentation worthwhile. The problem seems to be somewhere between others making such efforts, and me having useful information as a result. Innovation (2) seems a bad explanation because it doesn’t explain the lack of information about age-old lifestyle questions. It seems likely that I have overestimated gains relative to losses in the past (5), but gains still seem larger than losses (it’s hard to disentangle causes, but my lifestyle has obviously improved substantially over the last  year or more, some of which seems directly attributable to purposeful experimentation and the rest of which seems at least not terribly damaged by it).

GD Star Rating
Tagged as: ,

Personal experimentation: context specific?

A last way that personal experimentation could be worth it for me, yet not already completely covered by others, is that most of the facts one is likely to learn are quite context-specific. That way, everyone in history might have figured out for themselves what the best time and sugar-content for lunch is, and it would be worthless to me.

This also seems quite plausible. It could either be that people are so varied that there is just no good answer to whether it is better for productivity to eat snacks throughout the day or a few big meals for instance. Or it could be that which value of one parameter is best depends on all the other ones, so if you tend to eat more sugar than me and sleep less and laugh more, exercise might make you less sleepy than I.

The latter possibility bodes poorly for those who would experiment a lot. After you have determined the best quantity and timing of exercise, you might go on to try to optimize your sleep or sugar intake and make the original finding worthless.

This explanation would also seem to explain the observations in the last post: that many people do seem quite keen advise on the details of one’s life, but that the content of such recommendations seem a bit all over the place. Perhaps each person’s discoveries really do work well for them, but just look like a sea of noise to all the other people.

GD Star Rating
Tagged as: ,

Personal experimentation: not shared?

I’ve been talking about how personal experimentation could be worth it for people like me, without relevant info being depleted long ago.

My next potential explanation is that people do experiment, but results aren’t aggregated and spread, so everyone has to reinvent everything.

This is exactly what you would expect in a simple model where people benefit from information, but bear a net cost from spreading it. Without incentives to contribute one’s own findings to others, there is no reason information should spread. But on closer inspection this is roughly the opposite of what the world looks like. There is a lot of advice about how to run the details of a life. Sometimes it is offered for money, but often so enthusiastically and freely as to make the most curious life-optimizer want to run away. The problem seems to be more that there is so much advice, advising pretty much the full range of behavior. There are apparently incentives for spreading such ‘information’, but not incentives to actually find any information to begin with.

This is doubly puzzling. It’s not surprising if all the possible self-help books exist. But for folks volunteering their own time to tell me about whatever relaxation technique or diet, spreading random misinformation seems low value. And again we have the question of why it wasn’t worth it, for their own benefit, to get some actual information to begin with.

A plausible explanation to me for both of these things is that just about any random innocuous change makes life seem better, and people are genuinely trying to be helpful by telling others about such ‘discoveries’. So the problem then would be widespread use of informal data collection, which is much more unreliable than people think. In which case, my own experimentation is just as likely to fail if I rely on such data collection. Experimentation in general would not be as useful as suspected – continually experimenting would make you feel like things were good, but none of your efforts would have long term payoffs.

This leaves the questions of whether and why people would be misinformed about their abilities to casually collect information about the effects of interventions on their lives. What say you?

GD Star Rating
Tagged as: ,

Personal experiments: fueled by innovation?

Another way personal experimentation might be worth it for me, yet not used up by those before me: there is so much innovation that there are constantly new things to test, even if people experiment a lot. Beeminder and Workflowy are new. The abilities to prompt yourself to do things with a mobile phone or eat Japanese food or use your computer in a vast number of ways are relatively new.

I doubt this explains much. The question applies to many things that have been around and not that different for a long time, e.g. wheat, motivation, reading, romantic arrangements. And even if Beeminder is new, many of the basic ideas must be old (e.g. ‘don’t break the chain‘). As a society we don’t seem to have a much better idea of the effects of wheat on a person than we do of Beeminder.

Another way innovation could explain the puzzle is if all kinds of innovations change the value of all kinds of ancient things e.g. prevalence of internet use changes the effects of going to bed early or sitting in a certain way or doing something with your hair or knowing a lot of stories. If this is the case, experimentation is worth less than it seems, as the results will soon be out of date. So this goes under the heading ‘I’m wrong: experimentation isn’t worth it’, which would explain the puzzle, except the bit where everyone else perceives this and knows not to bother, and I don’t. I will get back to explanations of this form later.

GD Star Rating
Tagged as: ,

Personal experiments: for the unusual?

I asked in what kind of world personal experimentation could seem worth it for me yet not already exhausted. Today I’ll look at one potential explanation, popular with commenters last time: I and my friends are weird in some way that causes to benefit more than usual from experimentation. Several of the suggestions below were seconded by commenters.

Robin makes a plausible suggestion in this realm: in general people do quite well by copying the right other people’s behavior in some kind of clever, intuitive, context specific way. Nerds are terrible at this though (either because they fail to copy at the outset, or because they can’t do the social interpretation necessary to correctly generalize). So they have the choice to copy other people badly, or try to reinvent a lot of things from scratch. So experimentation is much more useful for nerds. Coupled with the premise that I’m a nerd, this explains the observations and has some intuitive appeal.

If something like this is true, there seem to me to be traits beyond lack of copying skill that incline nerds toward working such things out from scratch. In general if you are already unusual on many axes, copying others on a particular one is less good, so you will have to figure things out for yourself more. Once you have determined to sleep in the daytime and practice radical honesty, the usual answers about how to improve your mood or attract a partner may not apply as well. Nerds are also more likely to have the quantitative skills to do experiments well. And nerds seem more unsettled by adherence to traditions handed to them without explanation or instructions.

These things might explain enthusiasm for explicit experimentation and innovation, but the reasons experimentation seems worth it didn’t make reference to enthusiasm. Non-nerds may copy one another fine, but there seem to be better things to do than copying. It could also be that experimentation is not worthwhile, and nerds just tend to over-rate it. But fits nicely into a category to be explored later: ‘I’m wrong’.

Another relevant way I and my friends might be weird is that we live so late in history, and in such a rich world. Perhaps it has only recently become cheap enough to track such experimentation usefully. This seems important for the more elaborate data-tracking kinds of experiments. But it seems like you can do a lot with a pen and paper, and maybe a calculator and a coin. Also, as Robin points out, there are more people to copy now, so the ‘experiment little’ path is also easier. Arguably, I say.

Another way I am strange is in being relatively young. Youth clearly makes experimentation more valuable. However I feel like it is valuable enough, and that the gains are soon enough, that I would want to do it if I were similar to myself apart from having thirty years less to live. It could be that older people are unlike me however, in that they have learned a lot more by experimentation when they were young. Is this so? It’s not clear to me.

None of these explanations seem that great. Are there other ways I’m weirdly good at benefitting from experimentation?

GD Star Rating
Tagged as: ,

Why personal experimentation?

I experiment with many things, as do those around me. Some of this is randomization and explicit records, more is just trying different things, muttering ‘VOI‘ and repeating what felt good. I refer here to everything between a cautious banana-mustard-ham sandwich and polyamory.

Robin has suggested that I over-invest in such exploration. That most new things should be bad, and so most experimentation a private loss for public gain. What’s more, there shouldn’t be lots of low hanging fruit in trying things out. Most of the things humans frequently want to do (eat, sleep, change moods, organize time, learn, interact with others) should have been well figured out in ancient times. And anything that does still need checking out should be divided between many people.

Nonetheless, it looks to me like experimentation is worth it. Lots of the things we do seem barely satisfactory, there seem likely to be better alternatives, it seems hard to learn what has been tried for what ends, or what is good from listening to others or reading, and I and my friends seem to actually find good things by looking. e.g. Beeminderexplicit charity evaluation, unusual degrees of honesty, workflowy and explicit organization seem to often add value over the defaults, not to mention many tiny things, like banana-mustard-ham sandwiches.

If it is true that a lot of experimentation is worth it, we have a slight puzzle: if there is valuable information I might glean by experimentation, why hasn’t it been worth it for others in the past to collect it and put it where I can see it?

I will try to answer this over the next few posts. Before that, what do you think?


GD Star Rating
Tagged as: ,