Why Not Heterodox Research?

The following claim seems both roughly true and pregnant with implications:

The vast majority of statements that appear in natural human conversation, such as at dinner parties or on social media, are clearly unsuitable for inclusion as content (not study object) in the vast majority of academic journal articles.

Similar but weaker claims apply to academic books and lectures. Yes, there may be rare weird journals with quite different standards that some call “academic”, but they are exceptional, and most academics pause before even calling them “academic”.

This unsuitability seems clear to article authors, reviewers, and editors. It’s not just that many statements don’t intend to persuade or to argue for general claims of general interest; my claim above holds even when limited to such statements. And it’s not the form of such statements; no rephrasing of them that keep the same meaning would be suitable. And it’s not just their topics; even statements within the topic categories of a journal would not be seen as suitable for that journal. Nor is it their tone or purpose; my claim applies even when speakers clearly present themselves as do academics, as serious people trying to think seriously and informatively about serious things. It even applies in larger conversation contexts where this is the consistent tone.

Note: not only do such journal articles not include any such statements, they almost never cite articles that include such statements. And as authors are supposed to cite any sources that substantially influenced their article, their lack of citations seems to be them saying that the entire world of ordinary conversation has no substantial influence on the entire world of academic research. 

I conclude that each typical academic journal not only sees itself as covering a limited topic area, it also sees itself as being willing to consider only a limited range of concepts and argument types. Furthermore, these limits are quite strongly selective; the vast majority of statements that ordinary people actually generate on such topics, even when they are trying to talk seriously, are seen as unsuitable. 

Another way to say this is that typical academic disciplines (really sub-disciples) are organized around a very limited set of acceptable concepts and methods. Students of each discipline learn to comprehend, and then to apply, its accepted concepts and methods, and also to evaluate such applications. They become insiders by proving such abilities via submitting conforming articles, and having those submissions be positively evaluated by prior insiders. 

Plausibly we now find it quite hard to organize academic disciplines on any other basis. So if we want a world where articles within a discipline can be cheaply and reliably read and evaluated by others in that discipline, perhaps this is how we must organize them. 

But notice a dramatic implication: individuals who master the concepts and methods of many disciplines, or who apply the concepts and methods that ordinary people naturally apply when talking in ordinary conversation about a topic, should be able to generate a much larger range of valid relevant arguments on that topic. So our usual academic practices must be excluding a vast space of possible valid and relevant but “heterodox” arguments. (And people seduced by the insight-potential of this larger space will tend to lose academic prestige contests.)

Now it might happen to be that this larger set of valid relevant arguments doesn’t actually allow one to draw any better conclusions on some topic. One might be able to argue for any valid conclusion using a much more limited set of accepted concepts and methods, and to do this at no higher cost. In this case, nothing is lost by limiting articles to these few concepts and methods. 

To justify the further habit of academic articles almost never citing ordinary conversation, we’d need to make the further assumption that the very large practice of ordinary conversation almost never makes substantial contributions to topics of academic interest. 

In contrast, if the typical strong limits on discipline concepts and methods actually do prevent the use of many more cost-effective ways to generate valid relevant arguments, then we face a tradeoff between the costs (and benefits) of generating articles, versus the costs (and benefits) of evaluating articles. The habit of only considering articles that use (or cite) a very limited set of concepts and methods would have be justified by saying that the added cost (relevant to benefit) of evaluating heterodox articles is almost always greater than the added benefit (relative to cost) that such articles could provide via drawing relevant valid conclusions. 

That is, discipline authorities must claim:

To make progress on our topics, our discipline’s concepts and methods are quite sufficient. Sure others might in principle use other concepts and methods to draw relevant conclusions on our topics more easily than do we, but the chance of that usually seems so low that we just habitually ignore all purported candidates of this sort. There are just not usually clues that could plausibly indicate such a scenario well enough to get us to consider including heterodox articles in our journals, or to consider citing them in our articles. The enormous costs to us of evaluating the quality of such heterodox contributions completely swamps any value they might have to offer. 

Note that this claim, and academic citation practices, seem to suggest that ordinary people just couldn’t possibly be generating much useful insight from their vast practice of non-academic conversation. They might bond with each other, impress each other, and have fun, but they couldn’t possibly be learning much.

For me this issue highlights the great potential of innovations in how we evaluate contributions. Today, a reviewer typically takes an hour or two to review a dense 4-8K word research paper, where anyone in a discipline needs to be qualified to evaluate any article in that discipline (again, really sub-discipline). In this case, yes, everyone in a discipline must know well the same concepts and methods, and so each discipline can’t accept many such concepts and methods. 

But, it should be possible to instead have different people review different aspects of a paper, so that the concepts and methods of a paper don’t have to be limited to just one sub-discipline. Some academic reviewers could specialize in evaluating the concepts and methods of ordinary conversation, to make those available to paper authors. And it should be possible to get quick less-expert less-formal evaluations from betting markets, with bettor incentives tied to much-rarer more expert and expensive evaluations. Using such methods, academic journals should be able to consider submissions using a much wider range of concepts and methods. 

Note, however, that such innovations have long been possible, and I have personally seen such proposals enthusiastically rejected. Turns out disciplinary authorities who have risen to the top of their fields via their mastery and control over acceptable concepts and methods may not be eager to invite competition for their prestigious positions from a wider range of people using strange concepts and methods. And as long as no parties near the academic world are able to defy the power of their prestige, this is how things will remain.