Chip Away At Hard Problems

Catherine: And your own research.
Harold: Such as it is.
C: What’s wrong with it?
H: The big ideas aren’t there.
C: Well, it’s not about big ideas. It’s… It’s work. You got to chip away at a problem.
H: That’s not what your dad did.
C: I think it was, in a way. I mean, he’d attack a problem from the side, you know, from some weird angle. Sneak up on it, grind away at it.
(Lines from movie Proof; Catherine is a famous mathematician’s daughter.)

In math, plausibility arguments don’t count for much; proofs are required. So math folks have little choice but to chip away at hard problems, seeking weird angles where indirect progress may be possible.

Outside of math, however, we usually have many possible methods of study and analysis. And a key tradeoff in our methods is between ease and directness on the one hand, and robustness and rigor on the other. At one extreme, you can just ask your intuition to quickly form a judgement that’s directly on topic. At the other extreme, you can try to prove math theorems. In between these extremes, informal conversation is more direct, while statistical inference is more rigorous.

When you need to make an immediate decision fast, direct easy methods look great. But when many varied people want to share an analysis process over a longer time period, more robust rigorous methods start to look better. Easy direct easy methods tend to be more uncertain and context dependent, and so don’t aggregate as well. Distant others find it harder to understand your claims and reasoning, and to judge their reliability. So distant others tend more to redo such analysis themselves rather than building on your analysis.

One of the most common ways that wannabe academics fail is by failing to sufficiently focus on a few topics of interest to academia. Many of them become amateur intellectuals, people who think and write more as a hobby, and less to gain professional rewards via institutions like academia, media, and business. Such amateurs are often just as smart and hard-working as professionals, and they can more directly address the topics that interest them. Professionals, in contrast, must specialize more, have less freedom to pick topics, and must try harder to impress others, which encourages the use of more difficult robust/rigorous methods.

You might think their added freedom would result in amateurs contributing proportionally more to intellectual progress, but in fact they contribute less. Yes, amateurs can and do make more initial progress when new topics arise suddenly far from topics where established expert institutions have specialized. But then over time amateurs blow their lead by focusing less and relying on easier more direct methods. They rely more on informal conversation as analysis method, they prefer personal connections over open competitions in choosing people, and they rely more on a perceived consensus among a smaller group of fellow enthusiasts. As a result, their contributions just don’t appeal as widely or as long.

I must admit that compared to most academics near me, I’ve leaned more toward amateur styles. That is, I’ve used my own judgement more on topics, and I’ve been willing to use less formal methods. I clearly see the optimum as somewhere between the typical amateur and academic styles. But even so, I’m very conscious of trying to avoid typical amateur errors.

So instead of just trying to directly address what seem the most important topics, I instead look for weird angles to contribute less directly via more reliable/robust methods. I have great patience for revisiting the few biggest questions, not to see who agrees with me, but to search for new angles at which one might chip away.

I want each thing I say to be relatively clear, and so understandable from a wide range of cultural and intellectual contexts, and to be either a pretty obvious no-brainer, or based on a transparent easy to explain argument. This is partly why I try to avoid arguing values. Even so, I expect that the most likely reason I will fail is that that I’ve allowed myself to move too far in the amateur direction.

GD Star Rating
loading...
Tagged as: , ,
Trackback URL:
  • Romeo Stevens

    Social science (of dubious merit obviously) suggests that speed of progress on a goal affects construal level. So it makes sense that amateurs who have the luxury of rediscovering low hanging fruit and in general doing more breadth first search in an area would be less interested at chipping away at fiddly details in near mode. As you mention, such a construal works better in new areas where a wider variety of imported methods are likely to bear fruit. So I guess I would be less tempted to say that there is a balance point that is optimal and instead say that we should like to fit the correct methods to the current state of a domain which is changing over time. Since people specialize in methods, aiming the wrong people at an area will cause underperformance, even if they are plenty smart.

  • J

    If you’re measuring intellectual progress in terms of published papers, then that obviously selects strongly for academic norms of rigor. But I guess I’m not sure how else I’d measure progress.

  • Michael Arc

    Citation needed.
    Preferably one with demanding statistical methods.

  • DanielHaggard

    I believe there was a dude Socrates who did fairy well for himself in terms of renown…

    I see a lot of what you talk about here as hypothesis formation work. Too much experimental work just fishes for results without a model of their domain that is used in hypothesis formation. Or worse concoct their model post hoc and pass it off as a priori. I think it would be massively naive to think that this behaviour isn’t widespread.

    And why? Because this sort of discourse has little value to the professional academic given their incentive structures. Prestige comes from the published result. And it just remains a fact that one can get away with doing a lot of experimental work in various fields without taking the time to engage in the sort of informal discourse that can do much to clarify logical space before a single experiment is run.

    The fact that such discourse is accorded no institutional value has pushed various disciplines to the point of absolute absurdity. Philosophy would be the natural home of this kind of crucible discourse. But instead they chase results just like everyone else… Except they do everything a priori, while occasionally updating their views in light of new empirical research..
    For all the professed rigour of analytic metaphysics, it is arguable whether it has made any sort of contribution to human knowledge whatsoever. Rigor itself becomes abberant where it yields so little.

    I think institutions and the rigor and heterogeneity they can provide are great things. I’m no radical. But they are so much in need of reform I don’t think your claims about are necessarily true in all academic contexts. An academic environment that incentivised this sort of discourse as a precursor to the rigorous application of method would offer the best of both worlds imo

  • Pingback: Overcoming Bias : Beware Futurism As Political Allegory

  • Garrett Lisi

    Near vs far, yes?

  • BT

    The largest missing piece of your story is access to resources to collect novel data. The benefit of being a professional is that other people provide you with resources to collect data on topics of interest. The cost of being a professional is that the gatekeepers of those resources require those more robust methods. The more successful you are at answering questions, the greater resources you are given in the form of tenure, grants, etc.

    When a new field arises, amateurs are well-positioned to make advances in those topics, but they can only grab low-hanging fruit, by aggregating whatever little data is relevant to the topic at hand. After that, further investigation requires resources to gather genuinely novel data and so the field necessarily professionalizes.

    I would suspect that the first intellectuals that address a new topic use informal methods even if they are professional academics, because of the limitations on data. They need to make a case for why professional investigation is worth the resources, and informal methods are more effective at aggregating disparate types of data through “narrative” arguments. The first argument for relativity was a gedankenexperiment, and only later was it confirmed via experiment.

    • http://overcomingbias.com RobinHanson

      In many areas the main resource to get data is time. In those areas, amateurs still make less progress, even proportional to the time they spend.

  • http://www.sanger.dk Pepper

    Even non-rigorous arguments can provide Bayesian updates. It seems like even if you are dealing with a mass of non-rigorous arguments, you can compile them in to a big pro-con list and build on non-rigorous work that way.

  • Pingback: Links for December 2016 – foreXiv