Super-Factor Scenario

A man always has two reasons for doing anything: a good reason and the real reason. J. P. Morgan

In economics today, as in many related fields, data analysis is king, and theory takes a back seat, at least as far as status goes. When people celebrate particular exemplary data analyses, they usually point to a use of difficult statistical techniques, or more commonly to some clever idea for how certain data could speak to an important question. They point far less often to what is more often the real limiting factor: access to relevant data, and to resources (such as time and student assistance) to process that data. Organizations with data are far more willing to show them to academics from prestigious institutions.

This is part of a more general pattern: when we give people status, the criteria we claim to use to choose who gets status often differs substantially from our real criteria. Let’s see how that might play out regarding the strong claim I posted on Saturday:

If we put together a huge super-dataset describing many individual people in as many ways as possible, a factor analysis of this dataset may find important new super-factors that span many of these features domains. Such super-factors would be promising candidates to use in a wide range of social research, and social policy. (more)

When someone finally does this data analysis that I’ve proposed, and finds some super-factors, they will be rightly celebrated. But what will they be celebrated for? Their main actual contribution will have been to get some organization to pony up enough resources to look for super-factors. But that’s not the sort of thing for which we like to celebrate intellectuals. So I predict that such people will instead be celebrated for the very idea of looking for super factors, for looking for a certain kind of super-factor, or for a clever computational or statistical technique used in the search.

There isn’t much risk of people finding my post and using that to undercut this celebration. I know of many cases where prestigious academics were celebrated for “insights” that others had expressed beforehand. As long as those others and/or their venues were of sufficiently lower status, academics see no conflict. Should anyone make an issue of it, there are always differing details that can be seized on to explain why the two ideas were really quite different.

If we had prediction markets on such things, and used them as the main way we allocated credit on such claims, well then in that case I might be able to lock in great rewards now, rewards that others couldn’t steal later. But that is one of the reasons we don’t want prediction-market-based rewards. In the end we like most of our hypocrisies, including those involving giving people status for different reasons than we claim.

GD Star Rating
loading...
Tagged as: ,
Trackback URL:
  • IMASBA

    “If we had prediction markets on such things, and used them as the main way we allocated credit on such claims, well then in that case I might be able to lock in great rewards now, rewards that others couldn’t steal later.”

    So with prediction the markets people will suddenly stop arguing that they developed the idea independently from you or that it differs in some way so it’s not “really” the same idea? Who watches the watchers: who gets to judge the outcome of a prediction market?

  • http://juridicalcoherence.blogspot.com/ Stephen Diamond

    Am I misunderstanding you? You actually believe that if someone demonstrates an important superfactor, you deserve substantial credit based on having come up with the bare idea of superfactors? [I think the R.B. Cattell beat you to the bare idea, but his research results discouraged it.]

    • Daniel Carrier

      I think he was saying that they deserve credit for convincing an organization to give them the resources to perform the search, but that sounds a lot less impressive so people aren’t willing to explicitly give them credit for it.

      • http://juridicalcoherence.blogspot.com/ Stephen Diamond

        Of course he’s saying that. But he implies more:

        If we had prediction markets on such things, and used them as the main way we allocated credit on such claims, well then in that case I might be able to lock in great rewards now, rewards that others couldn’t steal later.

        Robin’s apparently saying he’s entitled to the reward. Prediction markets would right the injustice of his not getting credit (“stealing”)? I realize it’s not his main point. Were it most anyone else, I would have guessed he’s being ironic.

      • http://overcomingbias.com RobinHanson

        My prediction market reward would be to win bets on whether super-factors exist.

  • free_agent

    “Their main actual contribution will have been to get some organization to pony up enough resources”

    That’s true, of course, but actually getting someone to pony up enough resources is the limiting step in a lot of activities, and so it should be rewarded. Certainly, that’s not celebrated in the popular press, but whenever I’ve heard from behind the scenes in research academia, obsession with maintaining one’s funding is ubiquitous. OTOH, to build and maintain the sort of reputation that can obtain funding requires a good sense for what questions can profitably be investigated, obtaining and managing the team of top-notch researchers under you that can answer the questions, etc. Sort of like being CEO of a startup company, but you never get to the “steadily making a profit” stage.

    I’m reminded of “the inventor’s delusion”, the idea that if one comes up with the idea for a new product, one should get all the profits of selling that product. Whereas in reality, conceiving of the product is often the easiest step, and building a business that can make and sell the product (which is necessary for the public to actually benefit from it) is a much more difficult step, which many fewer people can do well.

    • IMASBA

      “Whereas in reality, conceiving of the product is often the easiest step, and building a business that can make and sell the product is a much more difficult step, which many fewer people can do well.”

      Is it though? My own thoughts on this are that setting up such a successful business requires a degree of luck that you can’t really force. The process selects for “ok-ish” business skills but not much more and it’s nothing millions of other people couldn’t have done either at the same time or at the most a couple of years later. With some inventions there are really only a few thousand or even just a few hundred other people in the world who could have made the invention around the same time. Still, being the first to invent something is down to quite some luck as well (you’re probably not really smarter than the other few hundred/thousand). And in everything we do we stand on the shoulders of giants, or an army of employees/assistants, or (inherited) networks of contacts, or investors, or inherited money, or government subsidies, etc… so in my view both inventors and businesspeople do not deserve “all” the rewards: I contend that there is no person alive who really has created billions of dollars of added value all on his/her own and who isn’t immediately replaceable.

      • free_agent

        I’d like to hear about this subject from someone who has built a business. I tried once, and failed miserably.

  • LemmusLemmus

    The “data analysis that I’ve proposed”, eh? Give me a break. It’s long been known in the social sciences that “bad things” (low IQ, unemployment, unstable relationships, imprisonment and so forth) go together, and so (more or less by implication) do “good things”. When Shaw and McKay found this at the neighbourhood level in 1942, they remarked that they were replicating earlier findings. This has also been confirmed by factor analysis since at least 1990 (the famous Land et al. paper in the AJS). Basically the same is true at the individual level (e.g., see Gottfredson & Hirschi, *A General Theory of Crime*). This is also widely known; and the idea of r/K selection isn’t exactly recent.

    • http://overcomingbias.com RobinHanson

      By saying that I propose a data analysis I do NOT say that I’m the first to propose anything similar! But noting that important things tend to be correlated it not the same as actually working out concrete ways to estimate and measure those common factors. For example, people have long noted that those who are smart at some things tend to be smart at others, but even so the development of measures of the factor “g” was a substantial advance.

      • http://juridicalcoherence.blogspot.com/ Stephen Diamond

        At root, I think you probably overstate the independent value of factor analytic research.

        If measures of g were so important, then I suppose the Nobel Prize should have gone to John Raven. But Raven himself saw Spearman as the more greater psychologist. He was Spearman’s disciple. Raven’s Matrices was motivated not primarily by Spearman’s results but his theory, which largely determined the test’s form.

        Spearman held that the general factor of intelligence (later in his life, one of two general factors) was that the capacity involved in the eduction of relationships. Practically this mean that an analogies test was the favored vehicle, and the Ravens is a nonverbal analogies test. (Nonverbal for the sake of being culture-free.)

        Today, Ravens is best known for showing the near-greatest Flynn effect (next to WAIS Similarities). This makes it interesting, but less so as a measure of general intelligence.

        The Big-Five Personality Factors were also preceded by theory-guided work. Without some reason to think some particular general factors would emerge, I doubt your study is worth doing.

      • stevesailer

        Right, Spearman came up with the concept of a general factor of intelligence in 1904. But how lauded is Spearman?

        Arthur Jensen argued in his 1998 book “The g Factor” that Spearman’s general factor theory was the greatest social science idea of the 20th Century. But it took Jensen years to find a publisher for that book.

  • serhat

    selam