Popularity Is Random

Sunday’s New York Times told of an experiment on music popularity randomness:

In our study, … 14,000 participants … were asked to listen to, rate and, if they chose, download songs by bands they had never heard of.  Some of the participants saw only the names of the songs and bands, while others also saw how many times the songs had been downloaded by previous participants. This second group – in what we called the "social influence" condition – was further split into eight parallel "worlds" such that participants could see the prior downloads of people only in their own world. …

In all the social-influence worlds, the most popular songs were much more popular (and the least popular songs were less popular) than in the independent condition. At the same time, however, the particular songs that became hits were different in different worlds, just as cumulative-advantage theory would predict. …

In fact, intrinsic "quality," which we measured in terms of a song’s popularity in the independent condition, did help to explain success in the social-influence condition. …. But the impact of a listener’s own reactions is easily overwhelmed by his or her reactions to others. The song "Lockdown," by 52metro, for example, ranked 26th out of 48 in quality; yet it was the No. 1 song in one social-influence world, and 40th in another. Overall, a song in the Top 5 in terms of quality had only a 50 percent chance of finishing in the Top 5 of success.

No doubt this also applies to many other kinds of popularity, including in blogs or academia.  Beware of overconfidence about how good is the popular, or bad the unpopular. 

GD Star Rating
loading...
Tagged as:
Trackback URL:
  • http://econlog.econlib.org/archives/2007/04/i_coulda_been_a.html EconLog

    I Coulda Been a Contender

    For every winner, there are ten people saying (perhaps under their breath) that “I’m as good as him. It could…

  • http://profile.typekey.com/halfinney/ Hal Finney

    The marketing phenomenon of “first-mover advantage” is another manifestation of this effect. In fact I’ll bet marketing literature is a good place to look for ideas on bias. In a way, marketing is a classic science of “applied bias-ology”.

    This also relates to the importance of institutional design in producing collective opinions. Having a group of people make private estimates tends to produce different results than having them make public estimates one at a time. Early estimates wield disproportionate influence in that case.

  • http://cob.jmu.edu/rosserjb Barkley Rosser

    Regarding the question of academic papers, there is no question that there are networks and unofficial, even sometimes official, groups that cite each other, often linked with who studied with whom, and so forth and so on. As a journal editor, I get very tired of seeing paper after paper that is just another manifestation of this in-group or that in-group, littered with the mutual self-citations of a narrow subset of groupies in their social/intellectual network. I have a lot more respect, ceteris paribus (because just because a paper comes out of some narrow group does not necessarily make it a bad paper), for papers whose citations are broader and to a wide range of literature and perspectives.

  • B.S.

    Does this happen in politics? I’ve always wondered why news programs insist on giving poll numbers constantly. Perhaps there is an attempt to steer opinion toward a candidate through selective poll citations?

  • http://wcw.bignose.org/ wcw

    Sure.

    But Elvis and the Beatles and the Stones and Dean Martin and June Taylor and Beethoven’s 9th and holy-mother-of-poo Shakespeare really are all that.

    If you can’t figure out what art is actually good, and which is just social influence, I pity you.

    Granted, I pity most people.

    -wcw

  • http://profile.typekey.com/robinhanson/ Robin Hanson

    wcw, most people are overconfident about their personal ability to tell quality from social influence, so why are you so confident that you are actually so much better than most people?

  • TGGP

    Hal, shouldn’t we expect people to revise the things that they state in light of of what they heard earlier? I expect that you mean to say that the random factor of which person speaks first carries too much weight relative to the opinions that never got spoken, but it also seems a bit like you are suggesting that in the case where people only give their opinions privately they will be more accurate than when they make public estimates one at a time (which means they are not random-walking toward accuracy Aumann-style).

    wcw, I have to say I side with Hayek against you here, although being a philistine I would have done so without Friedrich’s help.

  • Rue Des Quatre Vents

    Why is there a bias against imitation as valuable source of information? Google searches are a form of imitation. The whole wisdom of crowds is an instance of imitaiton. So why think it’s a bad thing? If lots of people like it, then maybe I should pay better attention because who am I to know.

  • Ed

    Wisdom of crowds actually depends on peoples ability NOT to imitate each other. This is what makes it fallible in the face of a powerful media.

  • http://profile.typekey.com/halfinney/ Hal Finney

    Several years ago I read a physics paper proposing an alternative to the standard cosmology picture of the origin of the universe, the Big Bang followed by a period of “inflation” in which the universe grew very quickly. It proposed a cyclic model in which no inflation is necessary:

    http://arxiv.org/abs/astro-ph/0204479

    The author was painfully aware of the dominance of the inflationary view and that he had an uphill battle to get his ideas accepted. Here is his comment:

    “The cyclic model could have been proposed at the same time as inflation, in which case it would have been interesting to see which proposal would have emerged as the more appealing idea. As it is, the cyclic model has arrived twenty years late, after cosmologists have become attached to the inflationary paradigm, so there can be no fair measure.”

    One wonders if there are many cases like this in physics and other sciences, where being first is more important than being right, in terms of becoming the dominant paradigm.

    I might mention BTW that I read this paper in 2003 and have not followed the topic much since (it is just a hobby for me), but upon searching I see that this theory, given the sexier name “the ekpyrotic universe” referring to a fiery wavefront that sweeps the entire universe, is being worked on relatively actively today with a number of groups involved, and is still considered (by them) to be a strong competitor to inflation.

  • Keith Elis

    On Yahoo! News, like a newspaper, all news stories are presented by way of headlines. However unlike a newspaper, Yahoo! News presents this headline as a link and can record the number of times readers follow each link. This allows Yahoo! to generate a news section called ‘Most Viewed News Stories’. One of the things I have noticed over the last 6 or 7 years of checking this section is that the stories in this section have some of the most interesting headlines of any stories on the site. I almost can’t help clicking on them, even though I know better.

    Here’s an example from just a few days ago. William F. Buckley’s nationally syndicated column is widely read, but his columns are literally *never* seen under Most Viewed News stories on Yahoo!. Now, I don’t check this section every day, but I check it enough to find this next part interesting. For the first time ever in my experience, I saw William Buckley’s column under Most Viewed News Stories with the headline ‘Imus Dead’.

    http://news.yahoo.com/s/ucwb/20070413/cm_ucwb/imusdead

    Now, just imagine users seeing this headline as a link so soon after the recent events with Imus. Even though I knew better, I clicked the link imagining that Don Imus had committed suicide or something. Apparently, many others also thought the headline ‘Imus Dead’ was worth clicking on.

    Now, everyone who clicked the link might have hated the column. I personally can’t stand Buckley’s stilted prose, so, as usual, I hated the column. But I had already voted. My preference for the headline was aggregated with everyone else’s. The headline worked.

    Hal is quite right from my perspective. I would take his statement that marketing is ‘applied bias-ology’ one step further and say it’s mass manipulation. Effective manipulation can not only cause popularity to be entirely uncorrelated with quality, it can also cause popularity to be uncorrelated with *preferences*, other things being equal.

  • http://cob.jmu.edu/rosserjb Barkley Rosser

    Hal,

    Regarding the inflationary vs cyclic debate, this is one of those that is or should remain open simply because there has been no clear empirical test or strong evidence. Certainly an existing paradigm has great strength of inertia, as Kuhn argued long ago. It is existing paradigms generating contradictions or failing to explain empirical observations that lays the groundwork for their fall and replacement. But with no clear test, the door is open for a competitor to at least compete.

  • http://www.saunalahti.fi/~tspro1/ Kaj Sotala

    A song in the Top 5 of quality has a 50% chance of finishing in the Top 5 of success. I’m not sure that’s as bad as the original post makes it sound like, especially considering all the random things that must influence whether or not a thing gets to the top. Personally, I think I might have expected the chance to be even lower.

    Some people seem to be reading this as a post that disproves popularity giving any indication of a thing’s quality. I’m interpreting it to suggest that popularity is actually a decent estimator of quality, though obviously not a perfect one. (Or is there any better way to objectively measure the quality of art?)

  • http://strangeknight.wordpress.com/2007/04/23/the-unpredictability-of-cumulative-advantage/ walking without rhythm

    The unpredictability of cumulativeadvantage

    I suppose this article from the NYT will be locked away behind a subscription soon, but Duncan J. Watts describes his teams experiment and suggests that its never possible to predict swings in aggregate behaviour (e.g. how popular somethi…

  • TGGP

    Kaj, art doesn’t have objective quality, no matter what Paul Graham might tell you. If it was we would have machines rather than fallible critics review it for us.

  • Pingback: Yearly Blogging Roundup #1 | Twisted Oak Studios Blog