Tag Archives: Academia

High Road Doubts

By Robin Hanson · May 16, 2013 4:00 pm · Comments (31)

According to the intellectual norms that I learned when young, there is a high road and a low road for proposing reforms. The low road is populist and pandering – you ignore critics and try anything to get folks who could do something excited about your idea – sex appeal, group loyalties, demonizing opponents, overselling gains, whatever it takes. The high road is elitist and analytical – you carefully write up arguments, ideally with math models, randomized trials, and stat analysis, and present them to elites for evaluation.

Academics usually see the low road as deceptive – by ignoring critics and refusing to present careful arguments for evaluation, you admit your arguments are weak. Low road advocates counter that academic models and trials are often quite distant from actual applications — what really matters is that people try and evolve ideas in realistic contexts, and see how they feel about them there.

Twenty-five years ago, as a thirty year old wondering how to devote my life to pushing prediction markets, a mentor I respected basically suggested a low road – I should write a popular book to get lots of people excited. Instead I mostly chose a high road, going back to school to get a Ph.D., doing math models, lab experiments, etc.

Today I have reached a notable milestone along that road; my paper arguing for futarchy, a form of governance based on decision markets, is now published in the leading academic journal in the field of political philosophy: the Journal of Political Philosophy. This would be the abstract, if that journal had them:

Shall We Vote on Values, But Bet on Beliefs?

Democracies often fail to aggregate information, while speculative markets excel at this task. I consider a new form of governance, wherein voters would say what we want, but speculators would say how to get it. Elected representatives would oversee the after-the-fact measurement of national welfare, while market speculators would say which policies they expect to raise national welfare. Those who recommend policies that regressions suggest will raise GDP should be willing to endorse similar market advice. Using a qualitative engineering-style approach, I consider twenty-five objections, and present a somewhat detailed design intended to address most of these objections.

Of course I might do even better someday, perhaps publishing top journal articles on math models or lab experiments. Even so, this seems a good time to ask: is the high road really better?

I have doubts. What futarchy and decision markets mainly need, and have long needed, are organizations to try them out on small scales, to work out the little details that general ideas need for practical application. Small scale successes might then lead to larger trials, perhaps eventually at very large scales. And I doubt that publishing this paper, or further top journal papers, will do much to induce such trials.

A pandering popular book might do much more, if it actually got people to try the idea. They wouldn’t have to do it for the right reasons, by correctly evaluating pro and con arguments. In fact, it would be fine if the book gave most folks much worse estimates, as long as it induced a thicker high tail of enthusiasm to actually do something. A better idea for reform, with a big pool of rational advocates, might add much less value to the world than a worse idea for reform, matched with fewer less rational advocates willing to actually try and evolve their idea.

After all, beliefs mainly matter for inducing relevant actions. The high road might produce more accurate beliefs, but the low road may often get more things done.

GD Star Rating
a WordPress rating system
Tagged as: , ,

Ask Questions That Matter

By Robin Hanson · January 22, 2013 9:50 pm · Comments (28)

I know a lot of people who think of themselves as intellectuals. That is, they spend a substantial fraction of their free time dealing in ideas. Most of these people are mainly consumers who take in ideas, but don’t seem to do much with them, at least as far as anyone else ever sees. But others are more outward facing, talking and writing about ideas, often quite eagerly.

Oddly however, most of these idea dealers seem to define themselves mostly in terms of the answers they want to promote, instead of the questions they want to answer. Most idea-oriented Facebook status updates seem like this – saying yay for some answer they agree with. The few that deal in questions also seem to be mainly promoting them, saying yay for the sort of people who like that question.

Now yes, in addition to question-answering the world also needs some answer indexing, aggregation, and yes, sometimes even promotion. And yes, sometimes the world needs people to generate and even promote good questions. But my guess is that most intellectual progress comes from people who focus on a question to which they do not currently know the answer, and then try to answer it. Yes, people doing other things sometimes stumble on a new answer, but in general it helps to be looking in order to find.

I also know lots of academics, and they all have one or more research topics. And if you ask them they can usually phrase these topics in terms of questions they want to answer. And this is a big part of what makes academics more intellectually productive. But alas, few academics are able to articulate in much detail why it is important to the world that their questions get answered. They usually just invoke some vague associations, apparently considering it sufficient that some journal is willing to publish their answers. They seem to think it is someone else’s job to decide what questions are important. Unfortunately, most academic journal articles are answering pretty uninteresting questions.

So the important intellectual progress comes down to the rather small fraction of intellectuals who both define their focus in terms of a question, rather than an answer, and who bother to think about what questions actually matter. To these, I salute, and bow. They are the sweet thirst-quenching fount of progress.

GD Star Rating
a WordPress rating system
Tagged as: ,

Not Science, Not Speculation

By Robin Hanson · December 19, 2012 2:15 pm · Comments (28)

I often hear this critique of my em econ talks: “This isn’t hard science, so it is mere speculation, where anyone’s guess is just as good.”

I remember this point of view – it is the flattering story I was taught as a hard science student, that there are only two kinds of knowledge: simple informal intuition, and hard rigorous science:

Informal intuition can help you walk across a street, or manage a grocery list, but it is nearly hopeless on more abstract topics, far from immediate experience and feedback. Intuition there gives religion, mysticism, or worse. Hard science, in contrast, uses a solid scientific method, without which civilization would be impossible. On most subjects, there is little point in arguing if you can’t use hard science – the rest is just pointless speculation. Without science, we should just each user our own intuition.

The most common hard science method is deduction from well-established law, as in physics or chemistry. There are very well-established physical laws, passing millions of empirical tests without failure. Then there are well-known approximations, with solid derivations of their scope. Students of physical science spend years doing problem sets, wherein they practice drawing deductive conclusions from such laws or approximations.

Another standard hard science method is statistical inference. There are well-established likelihood models, well-established rules of thumb about which likelihood models work with which sorts of data, and mathematically proven ways to both draw inferences from data using likelihood models, and to check which models best match any given data. Students of statistics spend years doing problems sets wherein they practice drawing inferences from data.

Since hard science students can see that they are much better at doing problem sets than the lessor mortals around them, and since they know there is no other reliable route to truth, they see that only they know anything worth knowing.

Now, experienced practitioners of most particular science and engineering disciplines actually use a great many methods not reducible to either of these methods. And many of these folks are well aware of this fact. But they are still taught to see the methods they are taught as the only reliable route to truth, and to see social sciences and humanities, which use other methods, as hopeless delusional, wolves of intuition in sheep’s clothing of apparent expertise.

I implicitly believed this flattering story as a hard science student. But over time I learned that it is quite wrong. Humans and their civilizations have collected a great many methods that improve on simple unaided intuition, and today in many disciplines and fields of expertise the experienced and studied have far stronger capacities than the inexperienced and unstudied. And these useful such methods are not remotely we’ll summarized as formal statistical inference or deduction from well-established laws.

In economics, the discipline I know best, we often use deduction and statistical inference, and many of our models look at first glance like approximations derived from well-established fundamental results. But our well-established results have many empirical anomalies, and are often close to tautologies. We often have only weak reasons to expect many common model assumptions. Nevertheless, we know lots, much embodied in knowing when which models are how useful.

Our civilization gains much from our grand division of labor, where we specialize in learning different skills. But a cost is that it can take a lot of work to evaluate those who specialize in other fields. It just won’t do to presume that only those who use your methods know anything. Much better is to learn to become expert in another field in the same way others do; but this is usually way too expensive.

Of course, I don’t mean to claim that all specialists are actually valuable to the rest of us. There probably are many fraudulent fields, best abolished and forgotten, or at least greatly reformed. But there just isn’t a fast easy way to figure out which are those fields. You can’t usually identify a criminal just by their shifty eyes; you usually have look at concrete evidence of crime. Similarly, you can’t convict a field of fraud based on your feeling that their methods seem shifty. You’ll have to look at the details.

GD Star Rating
a WordPress rating system
Tagged as: , ,

College Admission Markets

By Robin Hanson · December 2, 2012 2:50 pm · Comments (86)

This article by Ron Unz is long and rambles a bit, but deserves its provocative reputation. It offers data suggesting that over the last few decades the most elite US colleges have had systematically biased admissions, against asians and for jews, when measured against other standards, like tests and top math/sci competitions. Given the strong academic rhetoric against racial discrimination, you might expect this to cause a fervor, and to result in big changes soon. But I don’t expect much soon – most academics are from those schools, and benefited from those biases, loud complaining isn’t the asian style, and the larger society doesn’t much care because this discrimination is mostly limited to these schools.

The problem comes mainly from granting discretion to admissions personal to make subjective judgements. One solution is to just use objective features like test scores. But Unz worries about ambitious kids wasting their youth in mostly useless test prep. Also, application packets may contain other useful but harder to read clues about promising students. Unz instead prefers to admit “qualified” students at random, at least for most of the slots. But once everyone knew for sure that the elite schools didn’t actually have much better students, it isn’t clear why they would remain the elite schools.

As usual, my solution involves prediction markets. As I posted here five years ago, we could hide clearly identifying info about students, post their application packets to the web for all to see, and let anyone bet on the consequences of each student going to each school. Students might care about their chance of graduating, their income later, and some measure of satisfaction. Elite schools might care more about the chances of students being “successful” someday. Different schools might use different measures of success, such as with different weights for achievement in sports, politics, business, arts, etc. Schools could admit the students with the best chance to succeed by their measure, and students could apply to and then go to the school giving the best chance if achieving their goals. Or students could not go to school at all, if that was estimated to be best.

Of course speculators will favor students showing concrete signs of future success, and so ambitious students would spend their youth trying to achieve such signs. But instead of locking in particular limited metrics like standard test scores, where prep efforts are mostly wasted, this process would create an open competition to find signs of future success where efforts to gain them are more useful. After all, your chance of success later should be higher the more the signs you pursue push you to gain useful skills and habits in the process.

Yes it would be hard to get people to accept that such markets are accurate and hard-to-manipulate enough for this purposes. But equally hard, I expect, would be getting elite schools to say explicitly what sort of success they most want from students. They probably pretend to care more about admirable success, like being a famous writer, than they actually do.

Added 8p: Regarding anonymity, an obvious solution is for the official application to be completely public. Usually only a small fraction of the relevant application info will be things that are better kept private. Regarding that info, the applicant can just reveal that extra private info to a few trusted folks who are willing to trade in these markets. Markets do not need all traders know all relevant info to work well.

GD Star Rating
a WordPress rating system
Tagged as: , ,

Respectable Resentment

By Robin Hanson · October 19, 2012 8:00 am · Comments (22)

Assume for the purpose of this post that used car sales folks are exploitive and socially unproductive – they mainly trick buyers into spending more than they need. I don’t actually believe this, but I don’t want this post to be distracted by the issue of which professions are or are not socially productive.

So, imagine that you are competing to be a successful used car salesperson. But you find that you face real biases. Buyers are unfairly less willing to buy from you because you are female, or young, or the wrong ethnicity, or the wrong personality type. Or perhaps it is managers at used car sales firms who are biased against hiring your people. In any case, you have a legitimate complaint of bias, and you can legitimately resent that bias.

Even so, I don’t feel very sympathetic to your cause. Oh, on the margin I’d prefer that you win your battle against such biases. Its just that I don’t see it is as a high priority. Why? Because your cause is mostly selfish. Oh sure, the used car sales industry might be slightly more efficient if they weren’t unfairly biased against your sort. But by assumption what they’d get more efficient at is mostly exploiting ignorant buyers. Not a cause I can get behind.

Now imagine that you run a charity, and that while your charity is especially effective at its cause, e.g., reducing African poverty, it suffers from the bias that donors care more about using their donations to seem to help, than to actually help. You resent the fact that your charity doesn’t do so well because it isn’t as good at helping donors look caring. This time, I’m a huge supporter of your cause. Why? Because the bias you oppose is hurting us all, a lot.

So if you face gender bias getting hired as a cancer doctor, but for a type of cancer where doctors actually do little to help patients live longer, then I’m only mildly sympathetic. But If you suffer as a doctor because patients are biased to “do something,” and dislike your correctly telling them they are better off doing nothing, then I’m a huge fan and supporter.

If you suffer bias in academia because you are religious, but your chosen research area is mostly a pointless exercise in showing off math skills, I’m not going to get too worked up for you. But if your academic career suffers because your research is focused on a way to actually making important intellectual progress, which doesn’t happen to be a good way to show off math skills, I’ll shout your cause from the rooftops.

If you suffer from a bias based on the kind of person you are, you have a legitimate complaint. But it may not be an especially noble cause. However, if you suffer because of a common bias against doing a sort of thing that is especially useful, you may have a very noble cause. I can much more respect your resentment of a bias against doing good, than a bias against who you are.

GD Star Rating
a WordPress rating system
Tagged as: , ,

We Add Near, Average Far

By Robin Hanson · October 2, 2012 8:30 am · Comments (20)

Quick, what is the best gift you ever got from a woman? From your parents? From a left-handed person? From a teacher? These aren’t easy questions to answer. But they seem easier than these questions: What is the total value of all the gifts you ever got from women? From your parents? From left-handed folks? From teachers?

For the first set of questions you can try to think of examples of particular people in those categories, and then think of particular gifts you got from those particular people. That can help you guess at the best gift from those categories. But to estimate the total value of gifts from people in categories, you’ll have to also estimate how many gifts you ever got from folks in each category.

Note that it also seems easy to estimate the average value of gifts from each category. To do this, you need only remember a few gifts that fit each category, and then average their values.

As another example, imagine you are looking at building entrance laid out in multi-colored tiles. Some tiles are blue, some red, some green, etc. You are looking at it from a distance, at an angle, in variable lighting. In this situation it will be much easier to estimate if there is more blue than red area in the tiles, than to estimate how many square inches of blue tile area is in that entrance. This later estimate requires you to additionally estimate distances to reference points, to estimate the total surface area.

These examples suggest that when we think in far mode, without a structured systematic representation of our topic, it is usually easier to average than to add values. So averaging is what we’ll tend to do. All of which I mention to introduce to a fascinating paper that I just noticed, even though it got a lot of publicity last December:

This analysis introduces the Presenter’s Paradox. Robust findings in impression formation demonstrate that perceivers’ judgments show a weighted averaging pattern, which results in less favorable evaluations when mildly favorable information is added to highly favorable information. Across seven studies, we show that presenters do not anticipate this averaging pattern on the part of evaluators and instead design presentations that include all of the favorable information available. This additive strategy (“more is better”) hurts presenters in their perceivers’ eyes because mildly favorable information dilutes the impact of highly favorable information. For example, presenters choose to spend more money to make a product bundle look more costly, even though doing so actually cheapened its value from the evaluators’ perspective. (more)

The authors attribute this to a near-far effect:

Presenters face many pieces of potentially relevant information and need to determine, in a bottom-up fashion, which ones to include in a presentation. This presumably draws attention to each individual piece of information as a discrete entity and a focus on piecemeal processing. If a given piece of information exceeds a neutrality threshold, the presenter will conclude that it is compatible with the message he or she seeks to convey and will include it. This results in presentations that would fare better under an adding rather than averaging rule. In contrast, evaluators’ primary task is to make a summary judgment of the overall presentation, which fosters a focus on holistic processing and the big picture and results in an averaging pattern as observed in many impression formation studies.

Additional experiments confirm this near-far interpretation. Those who prepare presentations and proposals tend to focus on them in detail, and so add part values in near mode style, while those who consume such presentations or proposals tend to pay much less attention, and so average their values in far mode style.

This result seems to me quite pregnant with interesting implications, none of which were mentioned in the dozen blog posts on the subject that have appeared since last December. So I guess it’s up to me.

First, this result predicts the usual academic advice to delete publications from low ranked journals from your vita. Yes those extra publications took extra work, and show more total intellectual contribution, but distracted readers evaluate you by averaging your publications, not adding them.

Second, this also predicts that academia will tend in general to neglect conclusions suggested by lots of weak clues, relative to conclusions based on a single strong theory or empirical comparison. People with a practical understanding of particular areas will correctly complain that academics tend too much to latch on to a few easy to explain and justify arguments, at the cost of lots of detail that practitioners appreciate.

Third, this predicts that in morality and politics, which are especially far sorts of topics, arguments tend to be won by those who push simple strong principles, even though people privately tend to choose actions that deviate from such principles. For example, while laws say no one can get medical advice from non-doctors, on the grounds that docs know best, but given a private choice most of us would often let other considerations convince us to listen to non-docs. While actions tend to be chosen in a near mode where lots of other weaker considerations get added, people know their best chance for winning an argument with a distracted audience is to focus on their one strongest point.

Fourth, this predicts Tetlock’s hedgehog vs. foxes result. Foreign policy is an especially far view sort of subject, and experts who focus on one strongest consideration get the most respect and attention, but experts who rely on many considerations, which are on average weaker, are more accurate.

Futurism is probably the most far view sort of topic, so I’d guess that all this holds there the most strongly. That is, while the most futurists who get the most attention from distracted audiences are those who harp endlessly on one clear plausible idea, the most accurate futurists are probably those who know and use hundreds of clues, many of them weak. Alas this is a problem for those of us who want to consider some aspect of the future in detail, since we quickly run out of strong principles, and then have to rely more on many weak clues.

Added Nov 25, 2012: This post gives data showing people donate money based more on the average than the total sympathy of the recipients. So you are better off asking for donations to help a particular especially sympathetic recipient, than to help many such folks.

GD Star Rating
a WordPress rating system
Tagged as: , , ,

Rude research

By Katja Grace · September 27, 2012 6:00 am · Comments (32)

Bryan Caplan says intelligence research is very unpopular because it looks so bad to call half of people stupider than average, let alone stupid outright. Calling people stupid is rude.

But if this is the main thing going on, many other kinds of research should be similarly hated. It’s rude to call people lazy, ugly bastards whose mothers wouldn’t love them. Yet there is little hostility regarding research into conscientiousness, physical attractiveness, parental marriage status, or personal relationships. At least as far as I can tell. Is there? Or what else is going on with intelligence?

 

GD Star Rating
a WordPress rating system
Tagged as: , ,

What Is Your Sermon?

By Robin Hanson · July 22, 2012 12:00 pm · Comments (18)

Here is a simple model of intellectuals. It is wrong, but insightful:

  1. Intellectuals discover insights, and better intellectuals find more and better insights.
  2. Insights can be ranked by net value of personal effort. This net value includes how much it would matter if the idea was taken more seriously, and a person’s marginal ability to achieve this.
  3. Insight value falls with the number of others well placed and motivated to pursue an idea, and by the effectiveness of effort to sustainably change opinions on the subject. Value tends to fall with costs to explain and explore a subject, and with the previous efforts already made.
  4. Insights vary greatly in value. Topics vary in how unequally distributed are their insight values, and with the potential for very large values.
  5. Usually, insight value is distributed so unequally that one’s best insight is a substantial fraction, even most, of the value of all one’s lifetime insights.
  6. When value is very unequal, intellectuals best help the world by following a career plan of first mostly searching for their one best insight, and then mostly promoting and developing that insight.
  7. Intellectuals following this search-then-promote career plan would do most preaching in the second half of their career. At least if a lot of promotion is possible.
  8. Greatly diminishing returns to promotion efforts might justify ending promotion of your best insight, and returning to a search for more, or promoting your second best insight.
  9. An intellectual uncertain about which insight is best might pursue several while studying them, but the world is better off if they soon focus on their one best guess.
  10. If “preaching” differs from “teaching” in focusing more on fewer bigger insights, then intellectuals promoting their one best insight are roughly “preaching” “sermons”.

In reality, the tendency to preach as a function of status seems to have a roughly slanted-N shape: /\/. While non-intellectuals preach little, amateur intellectuals preach a lot, and even more early in their career. Among academics, high status folks often find a new angle very early in their career, and then publish variations on that for the rest of their career. In contrast, low status folks tend to produce a steady stream of publications, which don’t much build on each other.

Getting the Nobel prize often triggers a burst of preaching, but mostly on the subject where the prize shows that their insight has long since won out, and doesn’t need promoting. The rest of their preaching is usually spread across many topics, rather than a single second best topic.

A lot of intellectual preaching is about the “insight” that one political ideology is better than others. Given the number of others motivated to promote each such idea, the marginal value of these promotion efforts by any one person must be very low.

Most of these deviations from the simple model can be seen as puzzles – we can ask what needs to be added to the simple model to explain them. Some possible missing elements:

  • Status – The freedom to preach may be seen as high status, with low status academics punished when they presume to take on this high status mark. Amateurs can more freely ignore such pressures.
  • Signaling – Common forms of preaching may be seen as too easy. If academics are mainly selected for being impressive, they need to do other things to show off.
  • Power – Academics tied to powerful social institutions could make such institutions uncomfortable by endorsing disruptive ideas. Preaching by such academics needs to be channeled into institutionally approved directions.

If you are an intellectual, occasionally ask yourself:

  1. Of all your insights so far, which is most worth devoting a life?
  2. If you are young and promoting, shouldn’t you keep looking instead?
  3. If you are old and not yet focused on promoting your best, isn’t it time?

That is: What is your sermon?

GD Star Rating
a WordPress rating system
Tagged as: ,

Teacher Remembered

By Robin Hanson · May 26, 2012 10:00 pm · Comments (7)

Andrew Lo remembers an inspiring teacher from his childhood: “The best third-grade teacher ever.” So I paused to recall memorable teachers from my childhood.

One does stand out, though I can’t remember her name. Somewhere about eighth grade my English teacher arranged a special English class for me. It was very simple. Every day I was to go to the library, sit in an isolated booth, and just write. About anything. Which I did. She’d quickly look over what I’d written, and give me some feedback. I don’t recall much about the feedback – it may not have mattered much. What mattered is that I wrote and wrote, with a learned audience in mind.

This is a story both about the difference a teacher can make, but also about how teaching may not matter much. If you want to learn to write, well, just write and write.

GD Star Rating
a WordPress rating system
Tagged as: ,

Schools Are For War

By Robin Hanson · May 13, 2012 11:00 am · Comments (31)

The main reason we had rules to force kids to attend primary school was to make obedient soldier citizens to support their nation in time of war. This effect was even stronger for democracies:

Using data from the last 150 years in a small set of countries, and from the postwar period in a large set of countries, we show that large investments in state primary education systems tend to occur when countries face military rivals or threats from their neighbors. By contrast, we find that democratic transitions are negatively associated with education investments, while the presence of democratic political institutions magnifies the positive effect of military rivalries. …

We study historical panel data on education spending and enrollment – for Europe since the 19th century and a larger set of countries in the postwar period – to assess the correlation between military rivalry (or war risk) and primary education enrollment (or the occurrence of educational reforms). … [Our models] show a positive and significant effect of rivalry on primary enrollment, a negative direct effect of democracy, and a positive and significant interaction term between the two. Overall, our empirical results indicate a causal relationship from rivalry to primary educational enrollment. …

An economic literature … finds robust correlations between past wars and current state capacity in international panel data. … [A study] shows that military rivalry raises fiscal capacity in postcolonial developing states. … [Others] find that democracy does seem to have a systematic influence on top rates of estate taxation, whereas wars with mass mobilizations do significantly raise those rates. …

[Prussia pushed schools] to arouse a moral, religious, and patriotic spirit in the nation, to instill into it again courage, confidence, readiness for every sacrifice. …

[France pushed schools to] teach Frenchmen to be confident of their nation’s superiority … It should … eliminate disruptive conflicts and promote the unity of the classes. … The new teaching program … was … designed to teach the child that it was his duty to defend the fatherland, to shed his blood or die for the commonwealth, to obey the government, to perform military service, to work, learn, pay taxes, and so on.

In Prussia, France and Japan … military defeats and/or perceived military threats appear to have prompted an otherwise reluctant ruling class to invest in mass primary education. …In most countries of the sample a war preceded the educational reform, while a democratic transition rarely occurs before the education rise … Most often, the democratic transition instead takes place *after the education reform period. (more)

GD Star Rating
a WordPress rating system
Tagged as: , , ,