To answer my own question, I vote for category 2 below:

1. Studies purporting to prove something that is totally obvious;2. Studies purporting to prove something that is surprising at first but that has a plausible explanation; or3. Studies purporting to prove something that is both surprising and implausible.

In category 1, false studies might not survive as easily -- if it's an obvious result, there will probably be other valid studies out there that will rise to the top.

In category 3, enough people will be motivated to disprove the implausible result that the false studies will be exposed more readily.

I suspect it's category 2 that is potentially the most dangerous. If the false study is surprising but plausible, it will be spread by people who like to think of themselves as clever ("You wouldn't think that X is the case, but it actually is, and boy have I got the clever explanation for it!").

Expand full comment

In the HR environment the Hawthorne effect is usually represented as being that 'just taking an interest in people' (leading them to feel important) will improve their morale. So the effect is not random, and increases in productivity are mediated by imrovements in morale. What is understood to be an independant variable, in the standard representation, is the form of intervention in which 'taking an interest' is demonstrated.All that is ancient history of course, I believe the last major corporations who still applied the humanist HR practices which the Hawthorne experiment and the Palo Alto people inspired threw them out in the early 90's.

Expand full comment