An alternative hypothesis to the 'burden of knowledge' theory of the slowdown
Hall did not notice (deliberately or otherwise) the huge and flourishing field of Black poetry that is now widely recognized. He was pretty siloed. But the point stands: innovation comes often from unexpected (to some, to others expected and long delayed) quarters.
"Surely, the burden of knowledge is more burdensome in science than in writing." Quite the opposite. The burden of Dante, Milton, Shakespeare etc already existing is far more problematic to poets than the existing scientific knowledge. Harold Bloom and The Anxiety of Influence is a good place to start for that.
Just a fascinating read. Thank you for all of it--I will need to read it again. I do have an example for you about “divorcing” fields--the field of mechanotransduction. Happy to chat offline or here about it!
Working my way back through earlier posts here. The big question this wonderful work brings me to is obvious- why did the systems and incentives change so dramatically in the 1970s? This is also the turning point for job security of the middle classes, plus a whole lot of other social and economic trends that changed direction (usually for the worse). I would hazard a guess that the ultimate cause was the peak in US conventional oil production. If this is the case, then simply instructing people to organise systems the old way and realign incentives may not be enough. I am of the school that sees industrialisation as being a temporary blip in the longer churn of history. The complexity and fragility of industrial-style research science may not be sustainable for that much longer if this is the case. Looking forward to reading more of your back catalogue!
Thanks so much for writing this. The poetry section in particular is an excellent piece of evidence.
Another element you've surely thought about, but for the benefit of the audience: From Robin Hanson I learned how new peer review was, and how Einstein was skeptical of it--that's another part of the institutionalization of science that may end up slowing down science.
I noticed some overlap here with Adam at Experimental History and his piece about peer review. When the older, established professors begin acting as gatekeepers to grant funding and publications you get more citations of older work and more incremental improvement type work.
There is also the explosion in number of researchers during this time. This raises the expectations of number of publications to get noticed for the next role. This leads to more ‘minimum publishable unit” papers and fewer big jumps. It’s tough to spend three years on a postdoctoral project that might not work at all.
I’ll need to read your post a second time to understand it better. But I have a tangential question about this quote from Freeman Dyson: “and it was made clear that they didn’t want me at their meetings. So, they regarded me, at first, as being on their side, but then afterwards they found I wasn’t.” This is very strange to me because we’re talking about mathematics, not something more “political.” What does “their side” represent?