Simply distributing simple, empirically robust findings regarding supply and demand, along with public choice theory, would upend vast networks of academic prestige. No matter what the LLMs say, established networks of academic prestige will resist these insights for as long as they possibly can. This is why at the most banal level I've encouraged prediction markets/forecasting as a complement to the system of academic prestige through established high reputation journals. Many prestigious academics in the social sciences and humanities routinely make claims that are easily falsifiable. But at present the relative independence of the status networks in various disciplines and subdisciplines prevent the simplest standards of prediction/forecasting from carrying any reputational weight.
Are there really vast networks of academic prestige that can be easily upended? It might be tough to convince me that any discipline that is easily upended could be prestigious.
I'm not a mathematician so I can't judge from myself, but from reports of LLMs solving """real math problems""" - and in particular the recent unit distance problem breakthrough from OpenAI - it seems that at least within a field (Math) models are already connecting distant dots. So how long before this crosses over fields?
"Oddly, few people plan when young to adopt such a polymath life strategy."
I think modern people have become cocooned in privilege, terrified of losing comfort and status, and risk-averse. Feminization obviously contributes to all of this mightily. Consequently, more than the workers of previous generations, modern people tend to see success as a series of steps on a pre-laid path: graduation, credentials, internships, grad school, job, promotions, etc. This can be a profoundly negative sequence for society when many of the organizations that people are using to structure their life paths are either sclerotic and bureaucratized (and generating little net social value) or profit-driven but financialized and parasitic (generating little net social value). Our best and brightest often go into academia and finance and medicine... and end up weighed down in rule-bound, perverse structures, doing little other than burnishing their careers at the expense of intellectual inquiry, economic production, and patient well-being.
Agreed. I'm not sure what has caused it - partly I suspect it's increasing wealth and unfamiliarity with hardship and struggle. Partly it may be smaller family sizes - when you only have one child you over-protect it ("helicopter parenting") and train it to be terrified of even tiny risks.
Anyway the result seems to be people who are overly risk-averse. In crude terms - no balls.
The stand-out successful people seem to be those who are exceptions - Robin has accepted a huge reputational risk to pursue truth and importance. People like Elon Musk take crazy risks. That's the only way to get crazy payouts. (Obviously much more than just accepting risk is necessary - you need to judge which risks to take and be good at dealing with things going downhill.)
LLMs really are reducing the friction for doing interdisciplinary work. A common situation is that you have expertise in some field A and shallow understanding of another field B, and some intuition that it should be useful to combine them. Today you can talk to an LLM about your intuition, learn a lot and get many useful ideas. In the past, you'd have to either invest an enormous amount of time and effort to master field B, or to find a person with expertise in field B and enough understanding of field A to be able to communicate with you.
One of the best use cases for LLMs is as an infinitely patient tutor to ask questions of and bounce ideas off – especially on topics I don't know much about. I find their breadth more useful than their depth, at least in their current form.
Isn't this just a function of the fact that Academia generally prioritizes hyperspecialization, or rather, that it is the safest bet to get your place under the academic sun. Polymathy is personally rewarding and a very Humanistic endeavor, but it feels both high risk and high reward.
Indeed. We've had lip service for interdisciplinary studies, but from what I have seen, academic success is proportional to hyperspecialization. I've seen tenure and promotion cases fail when an individual's body of work appears to be too unfocused.
I think you have a good idea here. AIs are trained on human data so it's perhaps more natural for them to connect the dots on ideas "in distribution", than it is to extrapolate outside the distribution.
> ask if what they know about those two areas are in conflict, and if so substitute new more consistent views
Though insights are to be had not just by identifying conflicts between fields, but also gaps in one field that can be filled by another, or overlaps that show wider unidentified similarities.
No doubt there are various examples between the sciences. But also in the arts.
Eg despite many noting the connections between music and mathematics, they have been hardly explored by composers, as though some like the idea, few know any math. (Notable exception was the avant garde composer Xenakis, who had also worked as an architect, and wrote many pieces based on statistics, physics, etc, albeit not too successfully IMO; others could do better.)
Eg also in the arts, the connections between music and fine art (eg painting) have been oddly unexplored, despite their obviousness (eg Impressionism having been a simultaneous France-centred movement in both fields ).
"We might get a huge burst of progress soon if only we could get LLMs to look carefully at pairs of distant areas, ask if what they know about those two areas are in conflict, and if so substitute new more consistent views."
This is an institutional problem. We have interdisciplinary academics, but other academics don't listen to them. I don't think it will it be easier to get academics to listen to LLMs than to their fellow academics.
<EDIT>
I originally wrote a long section below, but now I think it's a tangent. I think the problem isn't so much that academics won't listen to people using interdisciplinary problem, as that the entire journal system is set up in a way that makes contributions from truly interdisciplinary people extremely difficult. Journals today have standardized guidelines for different types of articles to ensure every article is "rigorous", and these guidelines generally require a paper of many pages which describes one complete "study data > model > hypothesis > formulate test > run study" package all in a single paper. (This is thanks to Popper.) Whereas the interdisciplinary person could often make a very valuable contribution in a single paragraph, or even a single sentence, like "You can use information theory to just compute that."
</EDIT>
We could get that huge burst of progress right now if only we could:
- get journals and academics to listen to /humans/ with expertise in multiple fields, but prestige in none. Having interdisciplinary knowledge must of itself lend prestige.
----- Luis Alvarez, a physicist, was told to mind his own business when he found a global layer of iridium at the K-Pg boundary and proposed that an asteroid impact killed the dinosaurs.
----- Alfred Wegener's expertise was in astronomy, meteorology, and climatology when he proposed the theory of continental drift using evidence from geography and climatology. His Wikipidia page includes this quote from a critique of his theory: "One can only ask for the necessary distance to be maintained and the request to stop honouring geology in the future, but to visit specialist areas that have so far forgotten to write above their gate: "Oh holy Saint Florian [saint for protection against fire], spare this house, set others on fire!" (Max Semper, 1917)"
- get academics within a field who are stuck in a thesis / antithesis battle to be willing to consider a synthesis.
----- My 1997 PhD dissertation was on reconciling symbolic and neural network AI by embedding words in Hopfield networks (as LLMs do today). I showed that doing so produced symbolic networks which conformed to the predictions of priming experiments on humans, whereas the existing purely symbolic networks were complete failures at doing so unless you picked your examples very carefully, owing to the power-law distribution of node connectivity in symbolic networks versus the constant node connectivity in neural networks. I'd expected that people on both sides would be excited by this; instead, people on both sides refused to read it. My first dissertation advisor strung me along for 3 years before admitting that he had no intention of letting me graduate, because he "didn't understand it, and didn't want to."
- get practitioners in a single field to stop listening so much to prestigious people.
----- The use of antiseptics in surgery and obstetrics was obstructed for decades by the prestigious obstetrician Charles Meigs.
----- Freud. 'Nuff said.
----- Marvin Minsky & Seymour Papert killed funding for connection AI for decades with one book. This is a peculiar case: Minsky & Papert were quite right that a perceptron couldn't solve classification problems when the classes weren't linearly separable, because the perceptron used a linear activation transfer function. It took 17 years IIRC before somebody said, "Hey, what if we use a nonlinear transfer function?" It should have taken 17 seconds.
----- Noam Chomsky devastated linguistics, and even psychology (which he is in no way an expert in). Chomsky was IMHO obviously wrong from the start about the poverty of the stimulus (which can be proven to be false on the back of a napkin, yet wasted an entire generation of linguists), about behaviorism, and about probabilistic grammar. (And about politics, but let's not go there now.)
I deliberately got an interdisciplinary education. I majored in math, AI, bioinformatics, biotechnology, and English Lit, and minored in creative writing, linguistics, neurology, psychology, and cognitive science. I don't recall any potential employer ever seeing this as a good thing.
A digression: If we had an arXiv edited by AI, which allowed papers by AIs, which would inevitably be readable only by AI, what would happen? The near future may be one in which governments continue to make regulations based on research by humans and for humans, all hopelessly behind the research frontier being pushed forward by AI and used by businesses.
I agree we should try. I have weak evidence, from how great AI forecasters now are at finding where the community of humans is wrong, and then being proven right. Plausibly the main advantage these LLM systems have is breadth, not depth, of reasoning on these prediction market style questions.
I'm not sure what experiment would more directly test this on academic questions.
If LLMs could reason, wouldn't they be doing better at prediction markets than they are now? https://blog.zgp.org/money-bots-talk/ Their role in research right now might be as a super-featureful deck of "Oblique Strategies" cards...generating enough information that a person who can already recognize useful information can pick it out of their output. That's certainly useful, otherwise the original cards wouldn't have been ported to so many platforms
Human intelligence is largely staying on top of a situation. It's why AI can easily parallel any two things you ask it to, but can't hold a crying baby.
And you're a selective polymath just like everyone else.
I'm an amateur polymath because I'm curious across a wide range of topics, with the primary topic being mind-brain-consciousness-psychology, which I studied in college 50 years ago, and have continued to this day. I've never been in academia. Instead, I started my own school with now more than 800 graduates in 45 countries across the world. My first peer-reviewed academic research article will be published this year, so NOW I'm interested in academic research and the credibility it brings. I have used ChatGPT as a thinking partner, offering up bits of different knowledge bases and asking it to help me think through topics of intersection. It has brought unique insights I never would have conceived of myself. It knows me well, so it offers to do this even when I don't ask. I think LLM prompts (and informing the LLM about you and your interests) are the key to its innovative ability to put two disparate concepts together. Ask, and Ye Shall Receive.
You note each field has an expert version and a public version. Most of the N(N-1)/2 intersections are someone carrying the public version of A across, stripped of the boundary conditions the experts keep in mind.
You want to look at pairs of distant areas and resolve the contradictions. But the transfers that produced real knowledge came from misfit: a field supplies a problem people care about, the borrowed tool almost works, and forcing it to work yields a new abstraction (Shannon, Fisher, game theory). [LLMs are not going to do this yet unaided - agree]
Prestige shows up in your piece once, in who gets invited. It also shapes which imports get accepted: the recipient field reaches upward and adopts the high-status apparatus to borrow renown, sometimes without the substance (cargo-cult science, "mathiness").
[I guessed on reading that you were imagining a prompt directing an LLM to come back with the contradictions, gaps, etc; I doubt you only think contradictions are reasons to borrow across fields]
Also, the simplest explanation of why techniques flow from physics/math/philosophy->economics->softer sciences is actually that smarter-people fields produce things that work.
>> Most of my contributions have been applying stuff we know in some areas to other areas.
Its an essay about abstractions, but would have loved a concrete example or two here.
Yes please, because my mind keeps going to the social sciences, where from Sokal to replication, there seems to be a bigger problem...
Simply distributing simple, empirically robust findings regarding supply and demand, along with public choice theory, would upend vast networks of academic prestige. No matter what the LLMs say, established networks of academic prestige will resist these insights for as long as they possibly can. This is why at the most banal level I've encouraged prediction markets/forecasting as a complement to the system of academic prestige through established high reputation journals. Many prestigious academics in the social sciences and humanities routinely make claims that are easily falsifiable. But at present the relative independence of the status networks in various disciplines and subdisciplines prevent the simplest standards of prediction/forecasting from carrying any reputational weight.
Are there really vast networks of academic prestige that can be easily upended? It might be tough to convince me that any discipline that is easily upended could be prestigious.
I'm not a mathematician so I can't judge from myself, but from reports of LLMs solving """real math problems""" - and in particular the recent unit distance problem breakthrough from OpenAI - it seems that at least within a field (Math) models are already connecting distant dots. So how long before this crosses over fields?
"Oddly, few people plan when young to adopt such a polymath life strategy."
I think modern people have become cocooned in privilege, terrified of losing comfort and status, and risk-averse. Feminization obviously contributes to all of this mightily. Consequently, more than the workers of previous generations, modern people tend to see success as a series of steps on a pre-laid path: graduation, credentials, internships, grad school, job, promotions, etc. This can be a profoundly negative sequence for society when many of the organizations that people are using to structure their life paths are either sclerotic and bureaucratized (and generating little net social value) or profit-driven but financialized and parasitic (generating little net social value). Our best and brightest often go into academia and finance and medicine... and end up weighed down in rule-bound, perverse structures, doing little other than burnishing their careers at the expense of intellectual inquiry, economic production, and patient well-being.
https://jmpolemic.substack.com/p/job-search-part-6
Agreed. I'm not sure what has caused it - partly I suspect it's increasing wealth and unfamiliarity with hardship and struggle. Partly it may be smaller family sizes - when you only have one child you over-protect it ("helicopter parenting") and train it to be terrified of even tiny risks.
Anyway the result seems to be people who are overly risk-averse. In crude terms - no balls.
The stand-out successful people seem to be those who are exceptions - Robin has accepted a huge reputational risk to pursue truth and importance. People like Elon Musk take crazy risks. That's the only way to get crazy payouts. (Obviously much more than just accepting risk is necessary - you need to judge which risks to take and be good at dealing with things going downhill.)
LLMs really are reducing the friction for doing interdisciplinary work. A common situation is that you have expertise in some field A and shallow understanding of another field B, and some intuition that it should be useful to combine them. Today you can talk to an LLM about your intuition, learn a lot and get many useful ideas. In the past, you'd have to either invest an enormous amount of time and effort to master field B, or to find a person with expertise in field B and enough understanding of field A to be able to communicate with you.
One of the best use cases for LLMs is as an infinitely patient tutor to ask questions of and bounce ideas off – especially on topics I don't know much about. I find their breadth more useful than their depth, at least in their current form.
Isn't this just a function of the fact that Academia generally prioritizes hyperspecialization, or rather, that it is the safest bet to get your place under the academic sun. Polymathy is personally rewarding and a very Humanistic endeavor, but it feels both high risk and high reward.
Indeed. We've had lip service for interdisciplinary studies, but from what I have seen, academic success is proportional to hyperspecialization. I've seen tenure and promotion cases fail when an individual's body of work appears to be too unfocused.
I think you have a good idea here. AIs are trained on human data so it's perhaps more natural for them to connect the dots on ideas "in distribution", than it is to extrapolate outside the distribution.
> ask if what they know about those two areas are in conflict, and if so substitute new more consistent views
Though insights are to be had not just by identifying conflicts between fields, but also gaps in one field that can be filled by another, or overlaps that show wider unidentified similarities.
No doubt there are various examples between the sciences. But also in the arts.
Eg despite many noting the connections between music and mathematics, they have been hardly explored by composers, as though some like the idea, few know any math. (Notable exception was the avant garde composer Xenakis, who had also worked as an architect, and wrote many pieces based on statistics, physics, etc, albeit not too successfully IMO; others could do better.)
Eg also in the arts, the connections between music and fine art (eg painting) have been oddly unexplored, despite their obviousness (eg Impressionism having been a simultaneous France-centred movement in both fields ).
"We might get a huge burst of progress soon if only we could get LLMs to look carefully at pairs of distant areas, ask if what they know about those two areas are in conflict, and if so substitute new more consistent views."
This is an institutional problem. We have interdisciplinary academics, but other academics don't listen to them. I don't think it will it be easier to get academics to listen to LLMs than to their fellow academics.
<EDIT>
I originally wrote a long section below, but now I think it's a tangent. I think the problem isn't so much that academics won't listen to people using interdisciplinary problem, as that the entire journal system is set up in a way that makes contributions from truly interdisciplinary people extremely difficult. Journals today have standardized guidelines for different types of articles to ensure every article is "rigorous", and these guidelines generally require a paper of many pages which describes one complete "study data > model > hypothesis > formulate test > run study" package all in a single paper. (This is thanks to Popper.) Whereas the interdisciplinary person could often make a very valuable contribution in a single paragraph, or even a single sentence, like "You can use information theory to just compute that."
</EDIT>
We could get that huge burst of progress right now if only we could:
- get journals and academics to listen to /humans/ with expertise in multiple fields, but prestige in none. Having interdisciplinary knowledge must of itself lend prestige.
----- Luis Alvarez, a physicist, was told to mind his own business when he found a global layer of iridium at the K-Pg boundary and proposed that an asteroid impact killed the dinosaurs.
----- Alfred Wegener's expertise was in astronomy, meteorology, and climatology when he proposed the theory of continental drift using evidence from geography and climatology. His Wikipidia page includes this quote from a critique of his theory: "One can only ask for the necessary distance to be maintained and the request to stop honouring geology in the future, but to visit specialist areas that have so far forgotten to write above their gate: "Oh holy Saint Florian [saint for protection against fire], spare this house, set others on fire!" (Max Semper, 1917)"
- get academics within a field who are stuck in a thesis / antithesis battle to be willing to consider a synthesis.
----- My 1997 PhD dissertation was on reconciling symbolic and neural network AI by embedding words in Hopfield networks (as LLMs do today). I showed that doing so produced symbolic networks which conformed to the predictions of priming experiments on humans, whereas the existing purely symbolic networks were complete failures at doing so unless you picked your examples very carefully, owing to the power-law distribution of node connectivity in symbolic networks versus the constant node connectivity in neural networks. I'd expected that people on both sides would be excited by this; instead, people on both sides refused to read it. My first dissertation advisor strung me along for 3 years before admitting that he had no intention of letting me graduate, because he "didn't understand it, and didn't want to."
- get practitioners in a single field to stop listening so much to prestigious people.
----- The use of antiseptics in surgery and obstetrics was obstructed for decades by the prestigious obstetrician Charles Meigs.
----- Freud. 'Nuff said.
----- Marvin Minsky & Seymour Papert killed funding for connection AI for decades with one book. This is a peculiar case: Minsky & Papert were quite right that a perceptron couldn't solve classification problems when the classes weren't linearly separable, because the perceptron used a linear activation transfer function. It took 17 years IIRC before somebody said, "Hey, what if we use a nonlinear transfer function?" It should have taken 17 seconds.
----- Noam Chomsky devastated linguistics, and even psychology (which he is in no way an expert in). Chomsky was IMHO obviously wrong from the start about the poverty of the stimulus (which can be proven to be false on the back of a napkin, yet wasted an entire generation of linguists), about behaviorism, and about probabilistic grammar. (And about politics, but let's not go there now.)
I deliberately got an interdisciplinary education. I majored in math, AI, bioinformatics, biotechnology, and English Lit, and minored in creative writing, linguistics, neurology, psychology, and cognitive science. I don't recall any potential employer ever seeing this as a good thing.
A digression: If we had an arXiv edited by AI, which allowed papers by AIs, which would inevitably be readable only by AI, what would happen? The near future may be one in which governments continue to make regulations based on research by humans and for humans, all hopelessly behind the research frontier being pushed forward by AI and used by businesses.
I agree we should try. I have weak evidence, from how great AI forecasters now are at finding where the community of humans is wrong, and then being proven right. Plausibly the main advantage these LLM systems have is breadth, not depth, of reasoning on these prediction market style questions.
I'm not sure what experiment would more directly test this on academic questions.
If LLMs could reason, wouldn't they be doing better at prediction markets than they are now? https://blog.zgp.org/money-bots-talk/ Their role in research right now might be as a super-featureful deck of "Oblique Strategies" cards...generating enough information that a person who can already recognize useful information can pick it out of their output. That's certainly useful, otherwise the original cards wouldn't have been ported to so many platforms
Human intelligence is largely staying on top of a situation. It's why AI can easily parallel any two things you ask it to, but can't hold a crying baby.
And you're a selective polymath just like everyone else.
This is what postfoundational philosopher of science Wentzel van Huyssteen called "transversality".
I'm an amateur polymath because I'm curious across a wide range of topics, with the primary topic being mind-brain-consciousness-psychology, which I studied in college 50 years ago, and have continued to this day. I've never been in academia. Instead, I started my own school with now more than 800 graduates in 45 countries across the world. My first peer-reviewed academic research article will be published this year, so NOW I'm interested in academic research and the credibility it brings. I have used ChatGPT as a thinking partner, offering up bits of different knowledge bases and asking it to help me think through topics of intersection. It has brought unique insights I never would have conceived of myself. It knows me well, so it offers to do this even when I don't ask. I think LLM prompts (and informing the LLM about you and your interests) are the key to its innovative ability to put two disparate concepts together. Ask, and Ye Shall Receive.
You note each field has an expert version and a public version. Most of the N(N-1)/2 intersections are someone carrying the public version of A across, stripped of the boundary conditions the experts keep in mind.
You want to look at pairs of distant areas and resolve the contradictions. But the transfers that produced real knowledge came from misfit: a field supplies a problem people care about, the borrowed tool almost works, and forcing it to work yields a new abstraction (Shannon, Fisher, game theory). [LLMs are not going to do this yet unaided - agree]
Prestige shows up in your piece once, in who gets invited. It also shapes which imports get accepted: the recipient field reaches upward and adopts the high-status apparatus to borrow renown, sometimes without the substance (cargo-cult science, "mathiness").
[I guessed on reading that you were imagining a prompt directing an LLM to come back with the contradictions, gaps, etc; I doubt you only think contradictions are reasons to borrow across fields]
Also, the simplest explanation of why techniques flow from physics/math/philosophy->economics->softer sciences is actually that smarter-people fields produce things that work.