Intelligence Futures

For many purposes, such as when choosing if to admit someone to a college, we care about both temporary features, who they are now, and permanent features, who they have the ultimate potential to become. One of those features is intelligence; we care about how smart they are now, and about how smart they have the potential to become.

A standard result in intelligence research is that intelligence as measured late in life, such as at age fifty, is a much better indicator of ultimate potential than is intelligence measured at early ages. That is, environments have a stronger influence over measured intelligence of the young, relative to the old.

So if you want a measure of an ultimate potential, such as to use in college admissions, then instead of using current tests like SAT scores, you’d do better to use a good prediction of future test scores, such as predictions of related tests at age fifty.

Now of course colleges could try to do this prediction themselves. They could collect a dataset of people where they have late life test scores and also many possible early predictors of those future test scores, and then fit a statistical model to all that. But such data is hard to collect, this approach limits you to predictors available in your dataset, and the world changes, so that models that work on old data may not predict new data.

Let me propose a prediction market solution: create prediction markets on late life test scores. To make sure people try hard enough later, collect a fund to pay out to the person later in proportion to their late life test score. Then open (and subsidize) a market today in that future test score, and post any associated info that this person will allow. Speculators could then use that info, and anything else they could figure out, to guess the future test score. Finally, use market prices as estimate of future test scores, and thus of ultimate potential, in college admissions.

This approach could of course also be used by employers and other individuals or organizations that care about potential. A single market on a future test score could inform many audiences at once. And this approach could also be used for any other measures of potential where late life measures are more reliable than early life measures.

GD Star Rating
loading...
Tagged as: ,
Trackback URL:
  • Albert Ling

    Hmm I feel that future markets only work on big enough events… Who exactly has the incentive to subsidize this market too the degree that it becomes liquid and informative? Even in most sports markets where the event is not big enough (division 2 soccer for instance) the liquidity is just not there and all you have is a huge bid-ask spread of a bookmaker that just quotes outrageous prices and most transactions I assume are uninformed fans who would bet on his team without considering odds.. betting not as an investment but for signaling or other reasons… The idea is great for evaluating the value of CEOs of fortune 500 companies but I don’t know about average Joes.

    • http://overcomingbias.com RobinHanson

      The large literature on prediction markets shows that small markets can be informative; they don’t need thousands of traders or billions of dollars traded.

      • Bo Anderson

        But what Albert is saying is imagine how few people might be attracted by some of this. Imagine how many students apply to a college and they may allow varying amounts of info allowed. Who is gonna go and competitively bid those and not know if their algorithm will pay out for 30 years

      • http://overcomingbias.com RobinHanson

        A sufficient subsidy will get sufficient trading. With sufficient scale of activity, the costs seem modest to me.

  • Robert Koslover

    Oh c’mon! Isn’t it nearly a certainty that any school or any employer in the USA trying to implement a program even remotely similar to this would be accused of racism, regardless of the details of how the program actually worked? I think this is one of those ideas that, regardless of whether it is good or bad, will have to be set aside for at least a couple of generations or so, and maybe even longer than that. In fact, I’m not sure you can even publish or advocate such an idea (let alone try to implement such a program!) without being accused of racism. Maybe you can talk about it in non-threatening academic-style language on this blog, but that may be about as far as you can take it. Good luck. Just don’t say you weren’t warned.

    🙂

    • http://overcomingbias.com RobinHanson

      Estimating intelligence is NOT intrinsically racist.

      • Just Curious

        No it’s not, but the results generate by such a scheme would inevitably show racial disparities between groups of test-takers, and that would be tantamount to “disparate impact”, so employers using this scheme would be opening themselves up to any number of expensive lawsuits. Alternately, to avoid the disparate impact aspect, the cognitive tests would have to be gutted to the point that they lost all of their predictive power (which has become something of an evergreen news story when it comes to fire and police departments around the US).

        Also, you could be forgiven for believing that IQ is as malleable over time as you seem to think it is; that’s what popular news articles shriek out all the time — “Contrary to what wrong-thinking IQ researchers claim, your IQ can and does actually change!”. Of course it changes, from one day to the next, and from age 10 to age 65, sometimes a little bit, sometimes quite a lot. But, and that’s the key point, the changes are usually quite moderate, about +/- 5 points either way is typical. A change from 95 up to say 112 is also possible, and will likely be highly publicized by the researcher in question as more evidence of bunky IQ’s malleability, but cases like this are clearly outliers.

        Re IQ changes across the lifespan, here is a chart from Nicholas Mackintosh’s textbook “IQ and Human Intelligence,” showing that IQ tested at age 18 correlates about 0.7-0.8 with IQ tested at age 40; in fact the correlation already reaches this strength by age 8. Not a perfect correlation obviously, but a pretty strong one: http://imgur.com/f6IsV4V

        Interestingly, IQ tests’ margin of error is less than that of many common medical/physical assessments, such as blood pressure and even height (really!)

      • IMASBA

        “No it’s not, but the results generated by such a scheme would inevitably show racial disparities between groups of test-takers”

        But probably nothing that is statistically significant once you factor in all the other known variables. I guess a university could choose to focus on “race” (by whatever definition) if it really wanted to but why would it do so when there are 10+ other variables that are more predictive and not considered so offensive?

        “But, and that’s the key point, the changes are usually quite moderate, about +/- 5 points either way is typical. A change from 95 up to say 112 is also possible […] but cases like this are clearly outliers”

        The whole point of the testing would be for universities to select for those people who keep developing and do show above average IQ-malleability (usually because they do more/better mental exercises, so that exercise is a predictive factor). Also, IQ is not a linear scale: someone with an IQ of 112 will solve a problem more than 18% faster than someone with an IQ of 95.

      • Robert Koslover

        Re: “Estimating intelligence is NOT intrinsically racist.” I agree, and I never asserted otherwise. I stand by my remarks above.

      • http://juridicalcoherence.blogspot.com/ Stephen Diamond

        Speculators can (and will) use race as a basis for prediction. (Whereas race can be excluded or even partialed out of formalized prediction.) ‘Antiracist’ norms require its exclusion.

      • Fab Ocho

        Of course estimating intelligence is not intrinsically racist. Jason Richwine in his PhD at Harvard proposed using such estimated intelligence to inform US immigration policy.

        How did that work out for him?

        The correct facts given to us is that IQ is just a number, racism is the reason for social ills and violent crime is caused by poverty. And if you don’t believe that you will be excommunicated.

  • arch1

    Robin, I don’t see how the assertion in your 3rd paragraph follows from the first two. It seems it could go either way. How do you know that the uncertainty in predicting a young person’s midlife intelligence score won’t swamp the uncertainty in predicting that young person’s ultimate potential (however defined) based on their (known) current intelligence score?

  • http://juridicalcoherence.blogspot.com/ Stephen Diamond

    What information does this market – where all the speculators have the same information – serve to aggregate?

  • IMASBA

    Ok, I think I get how a prediction market is supposed to work in a firm (or a civil service institution). You harvest a lot of latent (expert) knowledge in a way that takes out all the signalling and ass-kissing and individual biases tend to cancel out. These markets can be quite small and the result will be verified rather quickly so it won’t be lagging behind changing circumstances by much and people don’t have to wait ages for the return on their bet. You can get a result that’s as good as an independent (not paid off/selected) study with and better than a realistic study, probably cheaper as well.

    Now for these intelligence futures. You mention one problem is getting students to provide even more information than universities already have on them. Another problem you mention is that circumstances change a lot in 30 years so a statistical study would always be playing catch-up. How are cheap prediction markets going to work here? Buying highly private info of students won’t be cheap (remember, we need info that universities don’t already have on students), so that means a high subsidy. More importantly, bettors have to wait a whopping 30 years for their return and their results will still be playing catch-up just as much as a study by the universities would. I just don’t see how the necessary information is out there in the first place, it would be like predicting which team wins the supwerbowl of 2050.

    • http://overcomingbias.com RobinHanson

      We don’t need these markets to have access to more info that colleges already have on students. We don’t need them to know things that won’t be revealed for decades. We just want the best currently available estimate given currently available info.

      • IMASBA

        “We don’t need these markets to have access to more info that colleges already have on students.”

        Then why did you say it was hard to collect data if the data you envision being used is already in possession of the universities? Of course a predict market could still be run using the data from the universities, but what you said sounded like the prediction markets would collect extra info that was otherwise difficult to get.

        “We don’t need them to know things that won’t be revealed for decades. We just want the best currently available estimate given currently available info.”

        Well you said using past correlations isn’t good enough because too many things change in 30 years time. How would a prediction market not run into exactly the same problem? What latent knowledge, not available from past correlations, would bettors have here? In this case a prediction market couldn’t even select habitually “good” bettors because no one lives for more than a couple of 30-year cycles anyway…

        Do you not trust predictive studies in this field to be independent enough (like consultancy in business being hired to give status to supporting a decision that was already made)? If so that could be an argument for wanting a prediction market instead of studies, but I’m not getting the arguments you’ve presented so far (which could be my fault, but either way I need more explanation to be convinced).

      • http://overcomingbias.com RobinHanson

        There is a world of difference between a person privately having info they are willing to act upon and creating a shared statistical model that a wide community can accept has having been proven sufficiently reliable.

      • IMASBA

        Yeah, but I’m assuming we care about “objectively” good results. By “privately having info” do you mean speculators would predict for individual outcomes, like what lump1 asked about?

      • lump1

        I take it that a part of the idea of prediction markets is that the price will contain all kinds of information that may not be published. For example, a college roommate who sees you spiraling out of control will find your futures price overvalued, and can probably profit from putting money on a lower prediction. His doing so will immediately put a downward pressure on your futures price. Do I understand that right?

  • http://juridicalcoherence.blogspot.com/ Stephen Diamond

    A standard result in intelligence research is that intelligence as measured late in life, such as at age fifty, is a much better indicator of ultimate potential than is intelligence measured at early ages.

    I’ve seen this result (although I can’t locate it), but I’m not aware of its being well-replicated. [One standard (countervailing) result is that with age abilities become more differentiated.]

    Do you have a link or cite?

    • stevesailer

      “A standard result in intelligence research is that intelligence as measured late in life, such as at age fifty, is a much better indicator of ultimate potential than is intelligence measured at early ages.”

      Do you have multiple sources for that? Are you sure you aren’t getting age 4 and age 14 confused with 18 and 50?

      Haven’t we seen enough evidence over the decades to say that Herrnstein’s 1971 prediction that having pretty good college admissions tests would have stratifying effects on society has turned out to be accurate?

      • stevesailer

        And doesn’t it look like Harvard, Yale, et al are already doing a pretty good job of predicting at age 18 who will be writing the biggest donation checks at age 50? There’s very little turnover in which colleges have the biggest endowments, so I’d say that Harvard is already pretty good at predicting lifetime earnings.

      • IMASBA

        Harvard is a brand, having it on your resumé opens doors on its own. It’s a classic example of a self-fulfilling prophecy, this can go on because, surprise, surprise, you don’t need to be a super creative genius for most jobs, even high level white collar jobs.

      • stevesailer

        But where is the evidence that any underdog has outsmarted Harvard at the admissions game? My impression is that Harvard has been since the 1940s or so pretty ruthless at researching the admissions game internally, but tries to give naive outsiders a mis-impression of how much quantitative research it has done for itself regarding testing and the like.

  • Joseph Wilks

    robin contact me ASAP

  • Seth Stephens-Davidowitz

    If you are trying to predict future intelligence because it’s a better predictor of future outcomes, shouldn’t you just predict future outcomes and skip the intelligence measure?

    • IMASBA

      The implicit assumption is that predicitng future outcomes in their entirety is too hard.