Sunday’s New York Times told of an experiment on music popularity randomness: In our study, … 14,000 participants … were asked to listen to, rate and, if they chose, download songs by bands they had never heard of. Some of the participants saw only the names of the songs and bands, while others also saw how many times the songs had been downloaded by previous participants. This second group – in what we called the "social influence" condition – was further split into eight parallel "worlds" such that participants could see the prior downloads of people only in their own world. …
Kaj, art doesn't have objective quality, no matter what Paul Graham might tell you. If it was we would have machines rather than fallible critics review it for us.
I suppose this article from the NYT will be locked away behind a subscription soon, but Duncan J. Watts describes his teams experiment and suggests that its never possible to predict swings in aggregate behaviour (e.g. how popular somethi...
A song in the Top 5 of quality has a 50% chance of finishing in the Top 5 of success. I'm not sure that's as bad as the original post makes it sound like, especially considering all the random things that must influence whether or not a thing gets to the top. Personally, I think I might have expected the chance to be even lower.
Some people seem to be reading this as a post that disproves popularity giving any indication of a thing's quality. I'm interpreting it to suggest that popularity is actually a decent estimator of quality, though obviously not a perfect one. (Or is there any better way to objectively measure the quality of art?)
Regarding the inflationary vs cyclic debate, this is one of those that is or should remain open simply because there has been no clear empirical test or strong evidence. Certainly an existing paradigm has great strength of inertia, as Kuhn argued long ago. It is existing paradigms generating contradictions or failing to explain empirical observations that lays the groundwork for their fall and replacement. But with no clear test, the door is open for a competitor to at least compete.
On Yahoo! News, like a newspaper, all news stories are presented by way of headlines. However unlike a newspaper, Yahoo! News presents this headline as a link and can record the number of times readers follow each link. This allows Yahoo! to generate a news section called 'Most Viewed News Stories'. One of the things I have noticed over the last 6 or 7 years of checking this section is that the stories in this section have some of the most interesting headlines of any stories on the site. I almost can't help clicking on them, even though I know better.
Here's an example from just a few days ago. William F. Buckley's nationally syndicated column is widely read, but his columns are literally *never* seen under Most Viewed News stories on Yahoo!. Now, I don't check this section every day, but I check it enough to find this next part interesting. For the first time ever in my experience, I saw William Buckley's column under Most Viewed News Stories with the headline 'Imus Dead'.
Now, just imagine users seeing this headline as a link so soon after the recent events with Imus. Even though I knew better, I clicked the link imagining that Don Imus had committed suicide or something. Apparently, many others also thought the headline 'Imus Dead' was worth clicking on.
Now, everyone who clicked the link might have hated the column. I personally can't stand Buckley's stilted prose, so, as usual, I hated the column. But I had already voted. My preference for the headline was aggregated with everyone else's. The headline worked.
Hal is quite right from my perspective. I would take his statement that marketing is 'applied bias-ology' one step further and say it's mass manipulation. Effective manipulation can not only cause popularity to be entirely uncorrelated with quality, it can also cause popularity to be uncorrelated with *preferences*, other things being equal.
Several years ago I read a physics paper proposing an alternative to the standard cosmology picture of the origin of the universe, the Big Bang followed by a period of "inflation" in which the universe grew very quickly. It proposed a cyclic model in which no inflation is necessary:
The author was painfully aware of the dominance of the inflationary view and that he had an uphill battle to get his ideas accepted. Here is his comment:
"The cyclic model could have been proposed at the same time as inflation, in which case it would have been interesting to see which proposal would have emerged as the more appealing idea. As it is, the cyclic model has arrived twenty years late, after cosmologists have become attached to the inflationary paradigm, so there can be no fair measure."
One wonders if there are many cases like this in physics and other sciences, where being first is more important than being right, in terms of becoming the dominant paradigm.
I might mention BTW that I read this paper in 2003 and have not followed the topic much since (it is just a hobby for me), but upon searching I see that this theory, given the sexier name "the ekpyrotic universe" referring to a fiery wavefront that sweeps the entire universe, is being worked on relatively actively today with a number of groups involved, and is still considered (by them) to be a strong competitor to inflation.
Why is there a bias against imitation as valuable source of information? Google searches are a form of imitation. The whole wisdom of crowds is an instance of imitaiton. So why think it's a bad thing? If lots of people like it, then maybe I should pay better attention because who am I to know.
Hal, shouldn't we expect people to revise the things that they state in light of of what they heard earlier? I expect that you mean to say that the random factor of which person speaks first carries too much weight relative to the opinions that never got spoken, but it also seems a bit like you are suggesting that in the case where people only give their opinions privately they will be more accurate than when they make public estimates one at a time (which means they are not random-walking toward accuracy Aumann-style).
wcw, I have to say I side with Hayek against you here, although being a philistine I would have done so without Friedrich's help.
wcw, most people are overconfident about their personal ability to tell quality from social influence, so why are you so confident that you are actually so much better than most people?
Does this happen in politics? I've always wondered why news programs insist on giving poll numbers constantly. Perhaps there is an attempt to steer opinion toward a candidate through selective poll citations?
Regarding the question of academic papers, there is no question that there are networks and unofficial, even sometimes official, groups that cite each other, often linked with who studied with whom, and so forth and so on. As a journal editor, I get very tired of seeing paper after paper that is just another manifestation of this in-group or that in-group, littered with the mutual self-citations of a narrow subset of groupies in their social/intellectual network. I have a lot more respect, ceteris paribus (because just because a paper comes out of some narrow group does not necessarily make it a bad paper), for papers whose citations are broader and to a wide range of literature and perspectives.
The marketing phenomenon of "first-mover advantage" is another manifestation of this effect. In fact I'll bet marketing literature is a good place to look for ideas on bias. In a way, marketing is a classic science of "applied bias-ology".
This also relates to the importance of institutional design in producing collective opinions. Having a group of people make private estimates tends to produce different results than having them make public estimates one at a time. Early estimates wield disproportionate influence in that case.
Kaj, art doesn't have objective quality, no matter what Paul Graham might tell you. If it was we would have machines rather than fallible critics review it for us.
Kaj, art doesn't have objective quality, no matter what Paul Graham might tell you. If it was we would have machines rather than fallible critics review it for us.
The unpredictability of cumulativeadvantage
I suppose this article from the NYT will be locked away behind a subscription soon, but Duncan J. Watts describes his teams experiment and suggests that its never possible to predict swings in aggregate behaviour (e.g. how popular somethi...
A song in the Top 5 of quality has a 50% chance of finishing in the Top 5 of success. I'm not sure that's as bad as the original post makes it sound like, especially considering all the random things that must influence whether or not a thing gets to the top. Personally, I think I might have expected the chance to be even lower.
Some people seem to be reading this as a post that disproves popularity giving any indication of a thing's quality. I'm interpreting it to suggest that popularity is actually a decent estimator of quality, though obviously not a perfect one. (Or is there any better way to objectively measure the quality of art?)
Hal,
Regarding the inflationary vs cyclic debate, this is one of those that is or should remain open simply because there has been no clear empirical test or strong evidence. Certainly an existing paradigm has great strength of inertia, as Kuhn argued long ago. It is existing paradigms generating contradictions or failing to explain empirical observations that lays the groundwork for their fall and replacement. But with no clear test, the door is open for a competitor to at least compete.
On Yahoo! News, like a newspaper, all news stories are presented by way of headlines. However unlike a newspaper, Yahoo! News presents this headline as a link and can record the number of times readers follow each link. This allows Yahoo! to generate a news section called 'Most Viewed News Stories'. One of the things I have noticed over the last 6 or 7 years of checking this section is that the stories in this section have some of the most interesting headlines of any stories on the site. I almost can't help clicking on them, even though I know better.
Here's an example from just a few days ago. William F. Buckley's nationally syndicated column is widely read, but his columns are literally *never* seen under Most Viewed News stories on Yahoo!. Now, I don't check this section every day, but I check it enough to find this next part interesting. For the first time ever in my experience, I saw William Buckley's column under Most Viewed News Stories with the headline 'Imus Dead'.
http://news.yahoo.com/s/ucw...
Now, just imagine users seeing this headline as a link so soon after the recent events with Imus. Even though I knew better, I clicked the link imagining that Don Imus had committed suicide or something. Apparently, many others also thought the headline 'Imus Dead' was worth clicking on.
Now, everyone who clicked the link might have hated the column. I personally can't stand Buckley's stilted prose, so, as usual, I hated the column. But I had already voted. My preference for the headline was aggregated with everyone else's. The headline worked.
Hal is quite right from my perspective. I would take his statement that marketing is 'applied bias-ology' one step further and say it's mass manipulation. Effective manipulation can not only cause popularity to be entirely uncorrelated with quality, it can also cause popularity to be uncorrelated with *preferences*, other things being equal.
Several years ago I read a physics paper proposing an alternative to the standard cosmology picture of the origin of the universe, the Big Bang followed by a period of "inflation" in which the universe grew very quickly. It proposed a cyclic model in which no inflation is necessary:
http://arxiv.org/abs/astro-...
The author was painfully aware of the dominance of the inflationary view and that he had an uphill battle to get his ideas accepted. Here is his comment:
"The cyclic model could have been proposed at the same time as inflation, in which case it would have been interesting to see which proposal would have emerged as the more appealing idea. As it is, the cyclic model has arrived twenty years late, after cosmologists have become attached to the inflationary paradigm, so there can be no fair measure."
One wonders if there are many cases like this in physics and other sciences, where being first is more important than being right, in terms of becoming the dominant paradigm.
I might mention BTW that I read this paper in 2003 and have not followed the topic much since (it is just a hobby for me), but upon searching I see that this theory, given the sexier name "the ekpyrotic universe" referring to a fiery wavefront that sweeps the entire universe, is being worked on relatively actively today with a number of groups involved, and is still considered (by them) to be a strong competitor to inflation.
Wisdom of crowds actually depends on peoples ability NOT to imitate each other. This is what makes it fallible in the face of a powerful media.
Why is there a bias against imitation as valuable source of information? Google searches are a form of imitation. The whole wisdom of crowds is an instance of imitaiton. So why think it's a bad thing? If lots of people like it, then maybe I should pay better attention because who am I to know.
Hal, shouldn't we expect people to revise the things that they state in light of of what they heard earlier? I expect that you mean to say that the random factor of which person speaks first carries too much weight relative to the opinions that never got spoken, but it also seems a bit like you are suggesting that in the case where people only give their opinions privately they will be more accurate than when they make public estimates one at a time (which means they are not random-walking toward accuracy Aumann-style).
wcw, I have to say I side with Hayek against you here, although being a philistine I would have done so without Friedrich's help.
wcw, most people are overconfident about their personal ability to tell quality from social influence, so why are you so confident that you are actually so much better than most people?
Sure.
But Elvis and the Beatles and the Stones and Dean Martin and June Taylor and Beethoven's 9th and holy-mother-of-poo Shakespeare really are all that.
If you can't figure out what art is actually good, and which is just social influence, I pity you.
Granted, I pity most people.
-wcw
Does this happen in politics? I've always wondered why news programs insist on giving poll numbers constantly. Perhaps there is an attempt to steer opinion toward a candidate through selective poll citations?
Regarding the question of academic papers, there is no question that there are networks and unofficial, even sometimes official, groups that cite each other, often linked with who studied with whom, and so forth and so on. As a journal editor, I get very tired of seeing paper after paper that is just another manifestation of this in-group or that in-group, littered with the mutual self-citations of a narrow subset of groupies in their social/intellectual network. I have a lot more respect, ceteris paribus (because just because a paper comes out of some narrow group does not necessarily make it a bad paper), for papers whose citations are broader and to a wide range of literature and perspectives.
The marketing phenomenon of "first-mover advantage" is another manifestation of this effect. In fact I'll bet marketing literature is a good place to look for ideas on bias. In a way, marketing is a classic science of "applied bias-ology".
This also relates to the importance of institutional design in producing collective opinions. Having a group of people make private estimates tends to produce different results than having them make public estimates one at a time. Early estimates wield disproportionate influence in that case.
I Coulda Been a Contender
For every winner, there are ten people saying (perhaps under their breath) that "I'm as good as him. It could...
Kaj, art doesn't have objective quality, no matter what Paul Graham might tell you. If it was we would have machines rather than fallible critics review it for us.