Your model implicitly assumes that it's costless to evaluate options.
Suppose I want to find a proof of some mathematical claim. Do I just look at options offered by professional mathematicians who have claimed to prove it or should I also consider a dumb automated reasoning system's submissions that simply iterates through all possible proofs?
In some cases evaluating options is cheap compared to the expected benefits. In other cases the cost of evaluating potential solutions is the dominate cost (in mathematics which is essentially just a search for proofs its really the only cost). You can't really derive what's more important in some context without more information about costs, benefits etc etc.
Your stretching the claim to make it true. If you alter the statement to really mean "having more options that are worth the cost to harvest and evaluate" then it's trivially true but says nothing. It's also true that it's better to have more cocaine -- when that cocaine is worth the costs and side effects in that situation.
Just because it's expensive to evaluate options doesn't mean you don't have many options or even that you can't evaluate them -- sell your house and pay the cost of evaluating an unreasonably large number of options. And it's easy to give examples of things that intuitively count as having lots of options but you don't have time to evaluate them appropriately. Indeed, the example I gave above with a bunch of math papers is one you could just pay mathematicians to help you evaluate or spend your own time reading but you'd be better off just never harvesting so many options in the first place.
--
But maybe this is just a linguistic disagreement. If you want to give necessary and sufficient conditions for what constitutes an option that don't allow counterexamples that would solve the issue. But my fear is that it's not possible to do while preserving the desired implication because it's really our disposition to only call things options if they are worth considering that's making more options seem obviously better but the implications Hanson wants to draw seem to depend on a more expansive notion of option.
Pardon, but, so what? Having many options but only being willing to evaluate 20% due to cost is functionally equivalent to having 20% to evaluate for free. As evaluation costs go down you can evaluate more. The logic still holds: the more options you have to evaluate the more likely you are to find higher quality. Whether the constraint on options comes from the number of options extant to evaluate or the number of options you are willing to evaluate is irrelevant to the point that more options leads leads to better outcomes.
Point Is that it can be better to have some options of decent average quality than to have those same options plus a bunch of extra options that have lower average quality since -- if it costs alot to evaluate the quality -- you now have to pick blindly from a lower average quality set. So it can definitely be the case that getting a bunch more options can make things worse.
And that means that the argument Hanson wants us to follow doesn't necessarily go through.
Ok, I see the argument you are making. I would counter that while it CAN be better to have a limited range of options, it need not be at all, and in fact depending on how you get those options it can make you much worse off. One has to ask how one is limiting the options, the preselection that goes into making that decent average quality set. If it just happens by celestial fiat, great, you are ahead and have to spend less on evaluation. If someone else has to evaluate first to curate your options as it were, then you have a new principle/agent problem to deal with as you now have to evaluate the curators to decide which giving you the better set, which has objectives more closely aligned with yours, and which one has the same definition of "decent average quality" that you do for a particular purpose.
I would go so far as to say that humans cannot escape the need for judgements except by adopting indifference; minimizing evaluation costs in one realm only creates new costs in another, so it comes down to at best determining what you prefer to evaluate and what you prefer to leave to others to choose for you.
Totally agree but with these caveats it no longer justifies the sweeping conclusions Hanson wants to draw. I mean I'm sympathetic those are valuable things for other reasons but the argument isn't valid.
"Why does it often seem otherwise, that we face hard choices between good things? Because you are often looking a sets of options that have already undergone many rounds and processes of selection. The more you select, the fewer remaining choices you’ll have, and the more you will face stronger tradeoffs between them."
That sounds to me like the tails coming together, as the choices become so similar that it is hard to distinguish between them. I might be misunderstanding.
Ok I think I see what you mean. The trade offs are smaller as the obvious worse options go away, so the remaining differences are more difficult to decide between.
It's more that as you get to the tail in one respect, you are unlikely to be near the tail in another respect. When you are in the middle of the range, it is relatively easy to find something that is better in both respects.
Heck, what even is an option? I worry we are kinda reading in an assumption of useful or worth considering in. If I try to imagine an objective definition it's not at all so clear.
I mean at some level of description the number of options we have never changes because it's always just the physical movement of our body in space. Do modern chemists have more options than medevil alchemists did? Even though those alchemists could have synthesized all modern reactants if they'd choosen the right options to start with?
I fear what's really doing the work here is that when we think of options we intuitively think of plausible, useful actions worth considering so you've kinda just defined them to be the things that are good to have (ow it wouldn't be useful to consider) and it's not clear why one should infer it's wealth that generates options or whatever else.
"Why does it often seem otherwise, that we face hard choices between good things? Because you are often looking a sets of options that have already undergone many rounds and processes of selection."
Berkson's paradox/collider bias is probably the simplest thing that's counterintuitive in causal inference, and it's always a fun one to discover in the wild. When you make a selection based on multiple traits, you can introduce an (anti-)correlation between them.
True multi-criteria optimisation doesn't exist. You also can't sort in multiple dimensions either for the same reason. You need to optimise or sort by a benefit, cost, or error function - something that gives an overall value and relationship between the characteristics you are interested in. So in effect sorting or optimising according to a single dimension created for the purpose. Any way you think of for optimising in multiple dimensions has an implied relationship between the two dimensions, if not an explicit one.
But also, in the real world, distributions are fundamentally tricky when it comes to the tails and their relationship to other dimensions. Most real single dimensional distributions have a bulge in the middle. It's because real distributions are made of many constituent factors. If the constituent factors have an additive effect, you get a normal distribution. If multiplicative, you get a log normal. If combined in most ways, you still get a bulge of some sort.
So , searching towards the tail of one property will likely take you away from the tail in another and towards the middle where the bulge in the other is, unless they are perfectly correlated., in which case you are really only optimising one thing anyway.
In practice, the method of choosing 0.1% by one criteria, and then 0.1% of these items by another criteria doesn't get you much better than the initial 0.1% search.
But yeah, if you have more choices, there is more chance there will be one a bit further up the tail of your optimisation than otherwise. But many tails drop off quite quickly and it often takes really a lot of choices to go far up the tail.
So if you try to optimise by multiple characteristics, it is fundamentally confusing. And optimising even by one is fundamentally difficult. Adding more choices past enough to work out the general shape of the distributions and relationships involved and getting an understanding if what is going on often doesn't help very much.
But you don't always know when you are going to find a fat tail, or a second part of the distribution, so looking way up the tails can be worth while. Go extreme on purpose, provided you are ready to fail and learn from it, and if you can do it, is probably more beneficial than just having more quantity once you have some quantity to inform you of what normal and extreme is.
Correct. The expansion of human knowledge, the underpinning of all economic growth, breeds the expansion of human capabilities. This then feeds back into the expansion of knowledge, and so on….
You're assuming the million projects are randomly generated instead of intentionally crafted. If there is no tradeoff to be made, why would there be ANY of them scoring poorly on a desirable metric with no downside?
Your model implicitly assumes that it's costless to evaluate options.
Suppose I want to find a proof of some mathematical claim. Do I just look at options offered by professional mathematicians who have claimed to prove it or should I also consider a dumb automated reasoning system's submissions that simply iterates through all possible proofs?
In some cases evaluating options is cheap compared to the expected benefits. In other cases the cost of evaluating potential solutions is the dominate cost (in mathematics which is essentially just a search for proofs its really the only cost). You can't really derive what's more important in some context without more information about costs, benefits etc etc.
Having higher coat of evaluation = having less quantity bc you can’t afford to see more. So driving down cost of evaluation = driving up quantity
Your stretching the claim to make it true. If you alter the statement to really mean "having more options that are worth the cost to harvest and evaluate" then it's trivially true but says nothing. It's also true that it's better to have more cocaine -- when that cocaine is worth the costs and side effects in that situation.
Just because it's expensive to evaluate options doesn't mean you don't have many options or even that you can't evaluate them -- sell your house and pay the cost of evaluating an unreasonably large number of options. And it's easy to give examples of things that intuitively count as having lots of options but you don't have time to evaluate them appropriately. Indeed, the example I gave above with a bunch of math papers is one you could just pay mathematicians to help you evaluate or spend your own time reading but you'd be better off just never harvesting so many options in the first place.
--
But maybe this is just a linguistic disagreement. If you want to give necessary and sufficient conditions for what constitutes an option that don't allow counterexamples that would solve the issue. But my fear is that it's not possible to do while preserving the desired implication because it's really our disposition to only call things options if they are worth considering that's making more options seem obviously better but the implications Hanson wants to draw seem to depend on a more expansive notion of option.
Pardon, but, so what? Having many options but only being willing to evaluate 20% due to cost is functionally equivalent to having 20% to evaluate for free. As evaluation costs go down you can evaluate more. The logic still holds: the more options you have to evaluate the more likely you are to find higher quality. Whether the constraint on options comes from the number of options extant to evaluate or the number of options you are willing to evaluate is irrelevant to the point that more options leads leads to better outcomes.
Point Is that it can be better to have some options of decent average quality than to have those same options plus a bunch of extra options that have lower average quality since -- if it costs alot to evaluate the quality -- you now have to pick blindly from a lower average quality set. So it can definitely be the case that getting a bunch more options can make things worse.
And that means that the argument Hanson wants us to follow doesn't necessarily go through.
Ok, I see the argument you are making. I would counter that while it CAN be better to have a limited range of options, it need not be at all, and in fact depending on how you get those options it can make you much worse off. One has to ask how one is limiting the options, the preselection that goes into making that decent average quality set. If it just happens by celestial fiat, great, you are ahead and have to spend less on evaluation. If someone else has to evaluate first to curate your options as it were, then you have a new principle/agent problem to deal with as you now have to evaluate the curators to decide which giving you the better set, which has objectives more closely aligned with yours, and which one has the same definition of "decent average quality" that you do for a particular purpose.
I would go so far as to say that humans cannot escape the need for judgements except by adopting indifference; minimizing evaluation costs in one realm only creates new costs in another, so it comes down to at best determining what you prefer to evaluate and what you prefer to leave to others to choose for you.
Totally agree but with these caveats it no longer justifies the sweeping conclusions Hanson wants to draw. I mean I'm sympathetic those are valuable things for other reasons but the argument isn't valid.
FELICES QUANTITY ENABLES QUATItY
This is relevant for your conclusion:
https://slatestarcodex.com/2018/09/25/the-tails-coming-apart-as-metaphor-for-life/
I've long been well aware of that, but it didn't seem to me relevant to this post. How is it relevant?
"Why does it often seem otherwise, that we face hard choices between good things? Because you are often looking a sets of options that have already undergone many rounds and processes of selection. The more you select, the fewer remaining choices you’ll have, and the more you will face stronger tradeoffs between them."
That's the tails coming apart.
That sounds to me like the tails coming together, as the choices become so similar that it is hard to distinguish between them. I might be misunderstanding.
Selection means you no longer have the options that are worse by both criteria. The way you distinguish the remaining options is by their tradeoffs.
Ok I think I see what you mean. The trade offs are smaller as the obvious worse options go away, so the remaining differences are more difficult to decide between.
It's more that as you get to the tail in one respect, you are unlikely to be near the tail in another respect. When you are in the middle of the range, it is relatively easy to find something that is better in both respects.
Heck, what even is an option? I worry we are kinda reading in an assumption of useful or worth considering in. If I try to imagine an objective definition it's not at all so clear.
I mean at some level of description the number of options we have never changes because it's always just the physical movement of our body in space. Do modern chemists have more options than medevil alchemists did? Even though those alchemists could have synthesized all modern reactants if they'd choosen the right options to start with?
I fear what's really doing the work here is that when we think of options we intuitively think of plausible, useful actions worth considering so you've kinda just defined them to be the things that are good to have (ow it wouldn't be useful to consider) and it's not clear why one should infer it's wealth that generates options or whatever else.
"Why does it often seem otherwise, that we face hard choices between good things? Because you are often looking a sets of options that have already undergone many rounds and processes of selection."
Berkson's paradox/collider bias is probably the simplest thing that's counterintuitive in causal inference, and it's always a fun one to discover in the wild. When you make a selection based on multiple traits, you can introduce an (anti-)correlation between them.
https://en.wikipedia.org/wiki/Berkson%27s_paradox
Surprisingly positive from you.
True multi-criteria optimisation doesn't exist. You also can't sort in multiple dimensions either for the same reason. You need to optimise or sort by a benefit, cost, or error function - something that gives an overall value and relationship between the characteristics you are interested in. So in effect sorting or optimising according to a single dimension created for the purpose. Any way you think of for optimising in multiple dimensions has an implied relationship between the two dimensions, if not an explicit one.
But also, in the real world, distributions are fundamentally tricky when it comes to the tails and their relationship to other dimensions. Most real single dimensional distributions have a bulge in the middle. It's because real distributions are made of many constituent factors. If the constituent factors have an additive effect, you get a normal distribution. If multiplicative, you get a log normal. If combined in most ways, you still get a bulge of some sort.
So , searching towards the tail of one property will likely take you away from the tail in another and towards the middle where the bulge in the other is, unless they are perfectly correlated., in which case you are really only optimising one thing anyway.
In practice, the method of choosing 0.1% by one criteria, and then 0.1% of these items by another criteria doesn't get you much better than the initial 0.1% search.
But yeah, if you have more choices, there is more chance there will be one a bit further up the tail of your optimisation than otherwise. But many tails drop off quite quickly and it often takes really a lot of choices to go far up the tail.
So if you try to optimise by multiple characteristics, it is fundamentally confusing. And optimising even by one is fundamentally difficult. Adding more choices past enough to work out the general shape of the distributions and relationships involved and getting an understanding if what is going on often doesn't help very much.
But you don't always know when you are going to find a fat tail, or a second part of the distribution, so looking way up the tails can be worth while. Go extreme on purpose, provided you are ready to fail and learn from it, and if you can do it, is probably more beneficial than just having more quantity once you have some quantity to inform you of what normal and extreme is.
I feel this way about diets. It’s fun to try new health trends even though almost all are fake
Correct. The expansion of human knowledge, the underpinning of all economic growth, breeds the expansion of human capabilities. This then feeds back into the expansion of knowledge, and so on….
This is false and corrupting the children. Quality determines quantity even in your quantity formation. Quantity isn't actually a thing.
I care about things ranked on non-ordinal scales
Interesting. This applies to dating, too, and is one reason young people flock to major cities.
You're assuming the million projects are randomly generated instead of intentionally crafted. If there is no tradeoff to be made, why would there be ANY of them scoring poorly on a desirable metric with no downside?