Jeremy Howard (former President and Chief Scientist of Kaggle) and Eric Ries (creator of The Lean Startup movement and Long Term Stock Exchange) have teamed up to found a new applied R&D lab: Answer.AI.

When speaking with Jeremy, he made it clear that many details of Answer.AI's structure are still being worked out. Only announced a month ago, the org is still in its early development stages. But the founders have conviction on certain principles. The most prominent of them is one extremely relevant to this Substack: The founders seem to be particularly inspired by Edison’s Menlo Park Lab and the early days of commercial electric research.

Previously, I’ve covered Edison’s Menlo Park Lab, the early GE Research Laboratory, and many other industrial research shops. Since FreakTakes is explicitly written for individuals founding new science orgs, has a particular focus on applied R&D, and has covered the lab that partially inspired Answer.AI, I thought it would be worthwhile to put together a piece specifically for Jeremy and Eric. The goal is to distill into a single piece the takeaways from prior FreakTakes pieces that are most useful to their nascent company. If they take even one nugget of information and use it to run their organization slightly differently, I’ll consider that a win.

In the piece, I’ll briefly examine the (working) plans for the lab and do some historical analysis, detailing:

1. What the earliest electrical R&D labs can teach Answer.AI

2. Useful rules-of-thumb from other historically great applied R&D labs

3. Potential pitfalls to keep in mind as they move forward

You can find more thorough historical evidence in my prior pieces for any of the lab details I mention, listed below:

• Thomas Edison, tinkerer published in Works in Progress

• Tales of Edison's Lab (podcast)

• Irving Langmuir, the General Electric Research Laboratory, and when applications lead to theory

• How did places like Bell Labs know how to ask the right questions?

• “The Third University of Cambridge”: BBN and the Development of the ARPAnet

• An Interview with Chuck Thorpe on CMU: Operating an autonomous vehicle research powerhouse

• ILLIAC IV and the Connection Machine

• A Progress Studies History of Early MIT — Part 2: An Industrial Research Powerhouse

• How Karl Compton believed a research department should be run

Each of the orgs listed has lessons to teach Answer.AI. But none are a perfect analog. So, as the piece progresses, I’ll explain which lessons I think most strongly apply to Answer.AI. With that, let’s get into it!

Subscribe now

Share

Answer.AI in a Nutshell

Jeremy’s blog post announcing Answer.AI makes it clear that the org is, to a large degree, inspired by the field of electricity’s path of progress in the 1800s. He believes the current state of the AI field is similar to the state of the electricity field between the work of Michael Faraday and Edison’s lighting projects. This was an era in which new electrical findings were being pieced together, but few had made any progress in turning the potential of electricity into great applications.

I don’t find this comparison crazy. So far, I don’t believe AI has come close to the level of breakthrough that electricity proved to be. Electricity brought the sunlight indoors for a negligible cost and powers so many of our modern conveniences— refrigeration, TVs, central heating, etc. That’s a high bar. However, given that human ingenuity created the breakthrough that was electricity and each of those applications, it is surely worth considering that AI could grow to be the most impactful field of them all. Whether AI does reach that level of promise, to me, is a question of human ingenuity. So, I have no issue with Jeremy comparing the AI field to the electrical field c. 1830 to 1910.

With that elephant out of the way, let’s briefly examine what sets Answer.AI apart from AI labs like OpenAI and Anthropic. From a funding perspective, Answer.AI seems much, much cheaper. The founders have initially raised $10 million. This stands in stark contrast to the gargantuan initial rounds of OpenAI and Anthropic. Also, Answer.AI's research agenda is more application-centric. The following excerpt from Jeremy’s blog post highlights what he thinks differentiates the lab’s approach:

At Answer.AI we are not working on building AGI. Instead, our interest is in effectively using the models that already exist. Figuring out what practically useful applications can be built on top of the foundation models that already exist is a huge undertaking, and I believe it is receiving insufficient attention.

My view is that the right way to build Answer.AI’s R&D capabilities is by bringing together a very small number of curious, enthusiastic, technically brilliant generalists. Having huge teams of specialists creates an enormous amount of organizational friction and complexity. But with the help of modern AI tools I’ve seen that it’s possible for a single generalist with a strong understanding of the foundations to create effective solutions to challenging problems, using unfamiliar languages, tools, and libraries (indeed I’ve done this myself many times!) I think people will be very surprised to discover what a small team of nimble, creative, open-minded people can accomplish.

At Answer.AI we will be doing genuinely original research into questions such as how to best fine-tune smaller models to make them as practical as possible, and how to reduce the constraints that currently hold back people from using AI more widely. We’re interested in solving things that may be too small for the big labs to care about — but our view is that it’s the collection of these small things matter a great deal in practice.

It would be unfair to say that an application-centric research agenda is necessarily less ambitious than AGI. Those biased toward basic research might say so, but I don’t think that opinion is very historically-informed. Edison himself was application-centric above all else. His deep belief in market signals is fascinating when juxtaposed with the market indifference of many great academic physicists. In the book From Know-How to Nowhere, a history of American learning-by-doing, Elting Morison described the interesting nature of Edison’s motivations:

If the means by which he [Edison] brought off his extraordinary efforts are not wholly clear, neither is the cause for his obsessive labors. No diver into nature's deepest mysteries carrying next to nothing for the advancement of knowledge and even less for the world's goods, he would become absorbed in making something work well enough to make money. The test in the marketplace was for him, apparently, the moment of truth for his experiments.

Edison built his god-like reputation by dreaming in specific applications. He kept market, resource, and manufacturing constraints in mind from the earliest stages of his projects. Edison dreamed practical, realizable dreams. And when the limitations of component technologies stood in the way of his dreams, he often had the talent to invent new components or improve existing materials. Edison’s biggest dream, the light bulb, mandated that  Edison solve a much broader set of problems. The following excerpts from my Works in Progress piece on Edison paint a clear picture of his ambitious but practical dreams:

After Edison’s bulb patent was approved in January 1880, he immediately filed another for a ‘System of Electrical Distribution’. Filing for these so close together was no coincidence. To Edison, it was never just a bulb project. It was a technical business venture on a possibly unprecedented scale. Edison wanted to light up homes all over the world, starting with lower Manhattan.

Bringing the project from dream to mass-market reality would require solving over a hundred technical problems. His was a new bulb that needed to be powered by a generator that did not yet exist at the start of the project, strung up in houses that had no electricity, connected via underground street wiring that was only hypothetical, and hooked up to a power station that had never existed before.

Yet, at the end of two years’ time, Edison would do it. And, just as importantly, the entire venture was profitable by the end of the project’s sixth year.

Edison was clearly doing a different kind of dreaming than those who do basic research. His lighting work embodies what extreme ambition looks like in application-centric research. Answer.AI making this kind of ambitious, applied work their North Star is an extremely interesting goal.

This goal has the potential to give Answer.AI a comparative advantage in the growing space of for-profit AI labs. For example, the most ambitious aspects of OpenAI are considered to be in its research, not its work on applications. Answer.AI’s particular setup can also set it apart from AI startups and academic labs. New AI startups do some research on how to commercialize new AI models in new ways, but they generally have short runways. In this kind of environment, only specific types of research projects can be pursued. Academic labs — for many reasons covered elsewhere on this Substack (such as in the ARPA series) — don't have the right combination of incentives, experience, and staffing to build new technologies in most problem areas. The main incentive of the profession, in a simplified form, is producing many paper studies that get cited many times. Answer.AI has the chance to let its alternative focus lead it to areas under-explored by academics, companies with brief timelines to hit revenue benchmarks, and more AGI-focused R&D labs.

Legally, Answer.AI is a company. But in practice, it might hover somewhere between a lab and a normal “profit-maximizing firm” — as was the case with Edison's lab. The founders seem perfectly content to pursue high-risk projects that might lead to failures or lack of revenue for quite a while. In saying this, I do not mean to imply they are content to light money on fire doing research with no chance of a return. Rather, they hope to fund a body of research projects that ideally have positive ROI in the long term. They are just not overly concerned with short-term revenue creation.

(Making the pursuit of research agendas like this easier is actually one of the founding goals of Ries’ Long Term Stock Exchange — which I address later.)

There is apparently no pressure to produce a product that can hit software VC-style revenue goals within 12-24 months, or anything similar. This is good. Seeking to satisfy these types of metrics does not traditionally permit a company to act like a truly ambitious R&D lab. I’m not saying it can’t happen — DeepMind seems to have made it work in its early years — but it does require pushing against investor pressure quite strongly. The VC money raised for Answer.AI has left the founders with enough voting shares that investors can’t veto founders’ decisions. Additionally, Howard says the company’s investors understand what they are trying to build is, first and foremost, a lab. This is a great step towards building an organization focused on building very useful, very new things rather than the most profitable thing possible — which often comes with bounded technical novelty.

Interestingly, Answer.AI will also keep a small headcount. Jeremy built Fastmail up to one million accounts with only three full-time employees. He hopes to keep the Answer.AI team exceptionally talented and “ruthlessly small” in a similar way; he believes keeping teams small is important to building new, technically complex things.

Now that I've outlined some important pieces of Answer.AI’s vision, I'll dive into the historical analysis. In the first section, I detail lessons that Answer.AI can draw from both Edison’s Menlo Park laboratory and the Early GE Research Laboratory. In the following section, I'll share useful lessons from other historically great industrial R&D labs. Lastly, I’ll highlight the bureaucratic details that explain why the operational models of the great industrial R&D labs have not been replicated often.

Learning from the First Electrical R&D Labs

I find it exciting that Edison’s Menlo Park lab is a North Star for Answer.AI. I covered Edison’s work in several pieces because I think evergreen lessons can be drawn from his work. But I think a more complete way to incorporate lessons from the 1870-1920 electrical space is to draw on the work of both Edison’s Menlo Park Lab and the young GE Research Lab. The latter operated as a more traditional industrial R&D lab. GE Research’s history holds many lessons to help steer Answer.AI’s problem selection and work on its standard projects. However, exceptionally ambitious projects may draw more heavily on the lessons of Edison’s lab.

(As a note, while Edison General Electric was one of the two companies that merged to become GE — along with Thomson-Houston Electric — Edison had essentially nothing to do with the formation of the iconic GE Research Laboratory.)

Different types of projects characterized the work of the two electrical labs. When it came to electrical work, for years, Edison’s lab and mental efforts were focused on doing everything necessary to bring a single, revolutionary product to market. On the other hand, GE Research usually had many separate courses of research underway at once. These projects all sought to improve the science and production of existing lighting systems, but they were otherwise often unrelated to each other. Additionally, GE’s work could be categorized as more traditional “applied research.” The lab was not actively looking to create a field of technology from scratch as Edison did. GE Research's projects were often novel and ambitious, but in a different way than Edison's.

Later, I will explore the types of novelty the GE Research Lab pursued. First, I’ll give the reader a more fine-grained idea of how Edison’s lighting project actually operated.

Lessons from Edison’s Work on Electricity

Edison’s lighting work provides great management lessons for those looking to direct a large chunk of a lab’s efforts toward a single, big idea.

Edison’s major contribution to the field of electricity was not inventing each of the components in his lighting system, but in turning a mass of disparate gadgets, scientific principles, and academic misconceptions into a world-changing system. The burden of doing “night science” — as Francois Jacob refers to it — largely fell on Edison. In the late 1870s, nobody knew much about electricity yet. The existing academic literature had more holes than answers, and many of its so-called “answers” turned out to be wrong or misleading. From this shaky starting point, Edison proceeded. He combined his unique mix of attributes and experience to deliver a world-changing system. These included: knowledge of several adjacent scientific fields, deep knowledge in then-overlooked experimental areas, market knowledge, manufacturing knowledge, and the ability to adequately operate a small research team.

In large part, Edison created his lab as a way to scale himself. As a result, to understand how his lab operated, one needs to know how Edison himself carried out his explorations. Edison was one of the more stubborn experimentalists of all time. He spent most of his waking hours carrying out one experiment or another. While he did pore over scientific literature, for him, nothing was settled until he proved it for himself at the lab bench.

I write in my Works in Progress piece:

Edison respected scientific theory, but he respected experience far more. In Edison’s era of academia as well as today’s, many professors had a certain preference for theory or ‘the literature’ over hands-on improvement. Because of this Edison did not care much for professors. He was even known to go on long diatribes, during which he had assistants open up textbooks, locate scientific statements that he knew to be untrue from experience, and quickly rig up lab demonstrations to disprove them. ‘Professor This or That will controvert [dispute with reasoning] you out of the books, and prove out of the books that it can’t be so, though you have it right in the hollow of your hand and could break his spectacles with it.’

Contained in his head was a database of countless experiments and results that made it seem as if his “intuition” was far beyond his contemporaries. This left him with an unparalleled skillset and body of knowledge. If anyone could feel comfortable pursuing a project that others had previously failed at, it was Edison. Edison’s confidence in his skills was never more on display than when he chose to pursue his lighting work. Many in the scientific establishment knew electric bulb lighting was technically possible, but claimed they had proven that it could never be economical. Edison disagreed.

On top of Edison’s admirable approach to experimentation, he brought a high level of practicality to his process. He knew his inventions needed to make commercial sense in order to make it out of the lab. So, even in early courses of experimentation, he kept factors like manufacturability in mind. He wouldn’t commit much time to something that didn’t make commercial sense. With that being said, Edison wanted to change the world with his technologies more than he wanted to get rich. So, the practical factors he paid aggressive attention to were primarily treated as constraints. He did not optimize for profitability, but he knew his ideas needed to be profitable. Nobody who wanted to optimize for profit would have pursued lighting in the way Edison did. The technical risks were too great.1

Edison was able to imagine an ambitious system that required many technical advances. It was so futuristic that maybe only he was capable of coming up with it. But just as impressively, he was able to do it profitably and on schedule. His dogged commitment to experimentation seems to be largely responsible for this. Edison and “the boys” constantly experimented on every piece of the process to improve and learn more about all the sub-systems in Edison’s grand system. They wanted to know how every piece of every sub-system performed in all conditions. I’ll share just two excerpts from my Works in Progress piece as examples.

The first is from Edmund Morris' biography of Edison. It recounts how thoroughly Edison and his trusted aid, William Batchelor, were in carrying out round after round of filament experiments:

For week after week the two men cut, planed, and carbonized filaments from every fibrous substance they could get — hickory, holly, maple, and rosewood splints; sassafras pith; monkey bast; ginger root; pomegranate peel; fragrant strips of eucalyptus and cinnamon bark; milkweed; palm fronds; spruce; tarred cotton; baywood; cedar; flax; coconut coir; jute boiled in maple syrup; manila hemp twined and papered and soaked in olive oil. Edison rejected more than six thousand specimens of varying integrity, as they all warped or split…

In the dog days, as heat beat down on straw hats and rattan parasols, the idea of bamboo suggested itself to him. Nothing in nature grew straighter and stronger than this pipelike grass, so easy to slice from the culm and to bend, with its silicous epidermis taking the strain of internal compression. It had the additional virtue, ideal for his purpose, of being highly resistant to the voltaic force. When he carbonized a few loops sliced off the outside edge of a fan, they registered 188 ohms cold, and one glowed as bright as 44 candles in vacuo.

This approach went far beyond bulb filaments. The following excerpt describes the work of one of Edison’s lead mechanics in turning the Menlo Park yard into a 1/3 scale model of what they would later install in Lower Manhattan. I write:

[Kruesi, Edison’s mechanic] along with a group of engineers and a team of six diggers, turned the excess land of the lab in Menlo Park, New Jersey…into a one-third-scale model of Edison’s first lighting district in lower Manhattan. This team tested and re-tested the electricity delivery system, digging up Menlo Park’s red clay to lay and re-lay an experimental conduit system. The team carried out countless tests to ensure that they found materials to efficiently carry the electric current while also keeping the delicate materials safe from water and ever-present New York City rats.

The entire process was marked by the classic trial-and-error of the Edisonian process. The first subterranean conducting lines and electrical boxes the group laid were completely ruined by two weeks of rain — despite being coated with coal tar and protected with extra wood. While the diggers dug up the failed attempt so the damage could be examined, Kruesi and a young researcher…studied and tirelessly tested unbelievable numbers of chemical combinations — making full use of the laboratory library and chemical room — until, finally, a blend of ‘refined Trinidad asphaltum boiled in oxidized linseed oil with paraffin and a little beeswax’ was found that protected the electrical current from rain and rats.

Edison built his own style of dogged experimentation into the culture of his lab. Since the lab was meant to scale Edison, this makes perfect sense; he was a man with far more ideas than hands. So, he hired more hands. Edison did not search far and wide to hire the world’s best research minds, and many of those he employed did not even have scientific backgrounds. This didn’t matter much to Edison because most of them were employed to undertake courses of research that he had directed them to pursue. A couple of his Menlo Park employees had advanced scientific degrees, but far more did not. For the most part, the lab and its activities were steered by Edison and his ideas. As a result, the productivity of his lab followed wherever his attention went. After some time working on a project area, Edison would often grow antsy and wish to move on to the next thing — he craved novelty. The lab’s resources and extra hands would move with him. As we’ll see in the next section, this stands in stark contrast to how the GE Research Lab recruited and chose problems.

Menlo Park's electrical activities provide a great management playbook for what it looks like to direct a lab’s efforts toward a single, major system. If Answer.AI does not want to go all-in on one thing, it can still find a way to apply this playbook to a certain focused team of employees while leaving the others to tinker around with exploration-stage ideas. In Edison’s less-focused experimentation periods, his lab served as more of an “invention factory,” doing this sort of fiddling. Additionally, Edison's preference for application and commitment to experimentation over theory in a young area of science can surely provide Answer.AI some inspiration.

Of course, Edison did some things better than others. Edison’s most easily-spottable “deficiency” is that his lab was largely dependent on him. Without him and his big ideas, the lab would have probably ground to a halt. While Edison’s technical vision, practicality, and experimental approach are absolutely worthy of emulation, the lessons of GE Research should probably be added into the mix as well. GE operated as more of a prototypical industrial R&D lab with an approach quite suited to the fact that the science of electricity was beginning to mature in the early 1900s.

Lessons from the Young GE Research Laboratory

The young GE Research lab took a different approach to electricity research than Edison. The lab worked on many unrelated projects at once, recruited more talented researchers, and allowed these talented researchers more freedom to exert the scientific method on commercializable projects. The lab did not undertake projects that were as purposely futuristic as Edison did. Nobody from the lab earned nicknames like “the Wizard of Menlo” or “the Wizard of Recorded Sound.” But early GE Research was still responsible for a Nobel Prize and making the light bulb a much-improved, more cost-effective technology.

Elting Morison wrote the following on the lasting impact of GE Research’s early decades:

There seems little doubt that…much that was done in Schenectady in electrical engineering and some parts of physics was both better done and more interesting than what was being done in those fields in any American university.

In its heyday, even great researchers like Karl Compton hoped to shift their academic departments to operate more like GE Research.

While GE did simultaneously pursue diverse projects, there was a unifying thread holding all of the projects at GE Research together. Each project aimed to improve the quality and profitability of GE’s products and manufacturing. Under that unifying theme, all kinds of projects were encouraged. Much of the research was very applied, particularly in the early years when the lab was still proving itself.

William Coolidge was one of the lab’s most talented applied researchers in its early years. Coolidge joined the lab in 1905, part-time while teaching courses at MIT. Coolidge had the kind of toolkit typical of many MIT professors in that era. He had a far greater grasp of the science of physics and metallurgy than somebody like a blacksmith; he was simultaneously far closer to a blacksmith than one would ever expect a university researcher to be. With this differentiated toolkit, he did science in a way that was not typical of academics. In describing the process that led to his successes at GE, he claimed that he was, “guided in the main by experiment itself rather than by metallurgical knowledge.”

Coolidge expertly applied practical skills in concert with scientific knowledge to pursue the problem. Elting Morison described a small sample of Coolidge’s workflow:

He suspended tungsten powder in an amalgam of bismuth, cadmium, and mercury. He then passed the resulting substance through tiny dies — drawing it — and obtained a silvery pliable wire. At that time, he thought he had reached ductility and the search was over. But when a current was passed through this wire the mercury, cadmium, and bismuth distilled out, leaving, unfortunately, a nonductile tungsten. But it also proved to be tungsten in the purest state he had yet produced.

I continue in my FreakTakes piece, writing:

He eventually iterated his way to a workable process where…the more pure tungsten was put through a specific combination of metal-working processes at a temperature that worked that produced rods of tungsten about 1 mm in diameter. These 1mm rods could then be drawn and re-drawn through rods of decreasing size until you were left with wires of tungsten .01 mm in diameter. When put in the vacuum-sealed bulb, electricity ran through the tungsten filaments and demonstrated an efficiency of 1 watt per candle — extending the life of a bulb up to 27x.

Within 5 years, 85% of all lamps would be made from tungsten. As the project went on, more and more research chemists and technical assistants grew to be involved in the wide-ranging steps and combinations involved in Coolidge’s experiments. But it worked. GE had the factories re-fit and deployed the new bulb. Coolidge moved on to other research.

The success of Coolidge’s hybrid work style, not dissimilar to Edison’s, is surely a useful data point to Answer.AI. But GE Research also did work that went far beyond Coolidge’s technically adept, applied science. The lab was fantastic at making use of talented individuals who were very academic. Irving Langmuir was a prime example. I described his interests in my original piece:

It should be noted…Langmuir did not even care about lightbulbs. Well, I guess that is not technically true. The bulb interested him because 1) he thought a metal like tungsten was cool because it could accept really high temperatures which opened up options to the scientist working with it and 2) these vacuum-sealed bulbs provided a pristine environment for controlled scientific investigations.

To Langmuir, light bulbs were primarily a playground in which to do his science. But Willis Whitney knew how to take an individual like that and direct his energy towards productive ends. The lab deployed a principle that I call extending a “long leash within a narrow fence” to basic researchers like Langmuir.

The way the lab facilitated this was rather simple. On his first day, Langmuir was told to walk around the applied end of the lab and ask people about their projects. Whitney permitted him to undertake any course of investigation of any phenomenon he wanted, but it had to be directly related to an existing problem/limitation/constraint that the applied folks were working through. These applied folks were working on projects that rather directly plugged into GE’s operations, so there was minimal risk of Langmuir’s work not amounting to anything useful if he succeeded and found answers. With that assurance of applicability, Langmuir was given extensive timelines to find answers to open questions.

Langmuir’s first course of research focused on the constant bulb-blackening problem common to bulbs at the time. The problem was generally attributed to a bulb’s imperfect vacuum. Langmuir found this problem to be a great excuse to carry out a course of experimentation he found interesting. Morison described Langmuir’s thought process as follows:

If residual gases — imperfect vacua — produced a bad effect — blackening — here was a fine opportunity to study the effects produced by different gases introduced one by one into the bulb. What he wanted to do, he told Whitney, was simply to plot the interactions of various gases exposed at low pressures to very high temperatures in the filament. Nobody knew very much about this phenomena and he wanted to look into it simply “to satisfy [his] own curiosity.”

Langmuir carried out this course of research over three years. There were many gases and temperatures to test, which took time. But unforeseen results constantly took Langmuir off in different directions. Exploring these unforeseen results often entailed new courses of experiment altogether. With his long leash, Langmuir was able to figure out that imperfect vacua were not what caused bulb blackening at all. Rather, it was that tungsten vapor particles were finding their way onto the wall of the bulb. Temperature was the issue.

He also discovered that different gases markedly changed the rate of evaporation. One extreme example was nitrogen, which reduced the evaporation rate by 100-fold. However, adding nitrogen to the bulbs caused the electrical efficiency of the system to decrease drastically. So, the existing bulb design with nitrogen added was less cost-efficient than the normal bulbs. But Langmuir was undeterred. This was progress.

Existing fundamental research in this area led him to believe that this efficiency issue could be alleviated by increasing the diameter of the filament. Further experimentation proved this to work. He also found that coiling the filament in a certain way could mitigate the heat loss issue. The final result was a novel bulb that used an inert gas instead of a vacuum to reduce bulb blackening. Along with the coiled tungsten filament, this new bulb only required .5 watts per candle and lasted three times longer than any other bulb.

Once he passed the bulb project onto the engineering team at GE Research, Langmuir set his sights on an anomaly he had come across talking with the lab’s more applied staff. The bulbs in the lab had a design that depended on only a few milliamperes of current flowing across the space between one end of the filament and the other. Langmuir noted this anomaly in a letter to Scientific Monthly, writing:

This fact seemed very peculiar to me, for the work of Richardson and others had indicated that at temperatures as high as those used in the tungsten-filament lamp, currents of many amperes should flow across the space. In other words, according to the then-accepted theory of the electron emission from hot filaments, a serious difficulty should have been encountered in the construction of tungsten-filament lamps. The fact that we did not meet any such difficulty therefore seemed to me a peculiar fact that should be investigated.

In the brief course of exploration that followed from Langmuir, he discovered what is now known as the space-charge effect. This work combined with follow-on work from Coolidge to produce an entirely new kind of GE X-ray tube.

Under this “long leash within a narrow fence” guideline, Irving Langmuir would go on to be partially responsible for a handful of new and improved product lines at GE. Additionally, the knowledge he created with his tungsten filament work went far beyond padding GE’s balance sheet. Over the course of his project, he noted that the way tungsten vapor condensed did not gel with existing academic theory. His subsequent exploration of this phenomenon led Langmuir to be credited with founding the field of surface chemistry. Langmuir earned himself a Nobel Prize for his efforts.

There was a symbiosis in the GE lab between Langmuir types and the Coolidge types — the latter skillset being more standard in the lab. I imagine Answer.AI will have no shortage of Coolidge-like individuals: bright, Kaggle Grandmaster-type individuals who understand academic theory but whose specialty is in expertly applying their craft in dirty, practical situations. Someone like Jeremy Howard will likely have great intuition about how to utilize these individuals. The GE playbook — with its “long leash within a narrow fence” principles — can help Answer.AI think through how to deploy basic researchers in its operations

Langmuir’s career at the GE Research Lab provides a clear roadmap for how to optimally leverage a basic researcher’s energies in an applied context. Langmuir getting paid to investigate any anomalies would likely have satisfied his curiosity. However, it was his investigation of the right anomalies that made this a beneficial arrangement for GE Research.

In general, there is a time and place to apply insights from either Edison’s playbook or GE’s. The maturity of a given research field or technology area has a strong hand in dictating which set of principles is more applicable. Edison came first and had to shoulder the burden of developing an extensive technical system to power the “killer app” that was his bulb. GE Research had the benefit of working on an existing technology area with moderately developed science and existing user technology (thanks to Edison), but the technology still needed a lot of work to become reliable and economical.

A lab can simultaneously employ both playbooks. Even most of Edison’s projects were modest in relation to his lighting work. When inventing for existing fields, such as telephony, Edison contained his inventive streak to working within existing technical systems. He knew nobody would rebuild entirely new telephone infrastructure just because the young inventor had rigged up a moderately improved but completely different version. When adding to Bell’s telephone, he simply invented a carbon transmitter that could plug directly into the system. This device made voices come through much clearer. That was it: one gadget that cleanly plugged into the existing system. Technologies like these may not be as earth-shattering as Edison’s lighting system, but they were still enough to make him a world-famous inventor in his own time.

It was about impact. In optimizing impact, I thoroughly suspect Answer.AI to make great use of the playbooks of both of these small industrial research giants.

Learning From Other Historically Great Industrial R&D Labs

I’d now like to highlight applicable lessons from other research operations covered on this Substack. I’ll cover the orgs in no particular order.

Striking the Balance of BBN and CMU’s Autonomous Vehicle Group

FreakTakes recently covered two historically great DARPA contractors who expertly balanced the competing pulls of project novelty and deployable technology. The first was Bolt, Beranek, and Newman (BBN), the contractor primarily responsible for the ARPAnet. The second was Carnegie Mellon’s autonomous vehicle groups.

BBN embodied what it meant to be a “middle ground between academia and the commercial world.” The firm was initially set up by MIT acoustics professors to pursue their contracting work more ambitiously. In its early decades, the firm gradually expanded its contracting efforts into the computing space, initially under the leadership of BBN VP J.C.R. Licklider. BBN soon became a common home for the best researchers in Cambridge, abandoning their academic positions to work for BBN. The firm's growing reputation even earned the monicker the “third university of Cambridge.”

The firm’s revenue was primarily sourced from research contracts given out by orgs like DARPA, research grantmakers, and aerospace firms. BBN’s positioning was somewhat unique; when compared to industry, the firm emphasized novelty and cutting-edge technology work. This insistence on novelty helped the firm recruit individuals who felt a bit too talented to waste away working on derivative projects at Westinghouse. When compared to academia, BBN emphasized working on real technology that people would use in the near term. J.C.R. Licklider is just one prominent example of an individual who left a tenureship at MIT to work on more useful technology down the road at BBN. Leveraging this positioning, the firm was able to recruit the best talent.

BBN also provided its most talented individuals latitude to ply their minds broadly. Many projects at BBN showcased the extreme potential of small teams of talented individuals with broad technical knowledge. Only eight BBNers were primarily responsible for pushing the early ARPAnet into existence. The size of the team was no accident; Frank Heart, the engineering lead of the project, described why he preferred a team of this size in his oral history:

I tend to believe important things get done by small groups of people who all know all about the whole project. That is, in those days all the software people knew something about hardware, and all the hardware people programmed. It wasn't a group of unconnected people. It was a set of people who all knew a lot about the whole project. I consider that pretty important in anything very big. So I suppose if you call it a management style, that would be something I'd state. I think also that they were a very, very unusually talented group. I think things tend to get done best by small groups of very, very good people — if you can possibly manage that. You can't always manage it. So if you again want to call it a management style, it is to get the very, very best people and in small numbers, so they can all know what they're all doing.

Lockheed Skunk Works legend “Kelly” Johnson also held quite similar beliefs when putting together teams to build experimental aircraft. Particularly in the early stages of novel projects, there is a strong case for keeping things small, with specialists who understand the fields that touch theirs. To me, Jeremy’s belief in small teams seems well-validated by technical history.

BBN demonstrates the ideal case of a research firm that wholly embraces technical novelty. CMU can be thought of as the flipside of that coin: a university that wholly embraced systems-building and used firm-like management practices to do so. The highlight of CMU’s later-1900s systems work was its autonomous vehicle projects. The academic group staffed itself with researchers responsible for technical integration and management-style work to effectively carry out novel technological systems building.

Similar to BBN, CMU’s positioning was differentiated from both industry and academia. This fact became very clear as DARPA’s mid-1980s autonomous vehicle work progressed. CMU was seemingly the only contractor excited about technical novelty and systems integration. Martin Marietta — the defense prime in charge of DARPA’s Autonomous Land Vehicle — obsessed over ways to hit DARPA’s demo benchmarks while using unambitious, dated technologies. Simultaneously, the academic vision research groups cared more about using the camera data to write papers than helping directly contribute to building a functional driving system. CMU was the only contractor involved in the project who truly cared about building a novel, functional system. DARPA eventually recognized this and gave them ample funds to build successive generations of autonomous vehicles. The rest was history.

CMU carried out this work with a management structure that was more firm-like than most academic labs. For example, Chuck Thorpe did project management-style work for the team with firm-like incentives — he was a researcher promoted based on vehicle performance, not his h-index. While this was a firm-like position, the group also had academic-style positions. Its use of graduate students on the projects is one prominent example. Each student on the project had to own a piece of the project that was all their own and could be written up as a thesis.

(I explore how the team mitigated the risks of theses not panning out in my interview with Chuck Thorpe.)

These academic incentives partially enabled the CMU team to continually innovate. Oftentimes these students’ theses perfectly plugged into existing systems, such as a thesis on reducing the processing time of an existing sensor’s data from 15 minutes to 90 seconds. But on the most extreme occasion, in 1988, this incentive structure led a grad student named Dean Pomerleau to successfully train a neural net to steer the vehicle. In that particular case, the requirement to allow each grad student to try something new changed the world.

Answer.AI similarly cares about deep technical novelty and building deployable technology. As such, Answer.AI could benefit from emulating BBN and CMU’s strategies to balance the two. The success of BBN and CMU should hopefully embolden Answer.AI’s founders to trust in the priorities they have set. This balance of goals is uncommon today, but orgs from history have expertly balanced the two to world-changing effect.

With that said, time elapsing without world-changing results might be unnerving. This ambiguity is partially what pushed academia to rely on near-term outcome variables that incentivize the incremental. Fear of wasting time and money is real. The Answer.AI founders would surely like some way to ensure that they are spending theirs on good problems. To deal with that, I think the Bell Labs’ approach to problem selection has a lot to add to the approaches I’ve already covered.

The Bell Labs Approach to Problem Selection

Bell Labs’ management of researchers in its golden era is famous — and it should be. However, Bell researchers were not left to their own devices to pursue whatever they wished, despite what many think. Bell managed their researchers with an approach similar to the “long leash within a narrow fence” approach of GE — which one long-time Bell chemist called “circumscribed freedom.” The most effective tool it used to do this was its corp of excellent systems engineers.

Bell had an expansive product line with massive scale — even more so than GE. Even modest improvements from the research team could have outsized returns. This, of course, is not the case with Answer.AI. However, Bell’s use of systems engineers can still be extremely instructive to Answer.AI, even if Answer.AI may deploy them in different ways.

As I covered extensively on FreakTakes, Bell’s systems engineers often combined several knowledge bases to expose the right researchers to the problems that most needed solving. Within one mind systems engineers often combined STEM backgrounds, knowledge of the nitty-gritty details of Bell’s manufacturing, an understanding of Bell’s implementation problems, detailed knowledge of Ma Bell’s expenses, and familiarity with the researchers at Bell Labs.

Bell knew these systems engineers were a massive part of their secret sauce, ensuring Labs deployed its limited resources on the right kinds of problems with sufficient upside. I wrote in the conclusion of my Bell Labs piece:

Finding a problem in these systems is not so hard for those familiar with the systems. That’s why many researchers and engineers do not feel the need to bring in help. But finding a set of good problems is not finding the best problems. Finding the best problems is a profession in and of itself. A systems engineer is worth it when, under the right scrutiny, it might turn out that the best problem is 10X as financially valuable, does 50X the social good, or is 2X as likely to work as just some run-of-the-mill good problem.

My prior piece delves deeper into the specific problems towards which Bell’s systems engineers led Bell’s researchers. For now, suffice it to say that I think that any new applied science org that can dedicate an (ideally full-time) individual to doing the work of a systems engineer should strongly consider it.

Of course, these systems engineers would need somewhat clear marching orders on what sorts of technologies they should be exploring. Nowadays, many existing roles in industry and academia train individuals to equate revenue or potential citation count with impact. Answer.AI will not be satisfied with these metrics as proxies of impact, and they shouldn’t be. What to direct these systems engineers toward instead should surely be up to Jeremy and Eric.

As a firm that pursued novelty and was not attached to a large industrial operation, BBN might be an interesting source of inspiration. Several of its hallmark projects started with systems-engineer-style contributions. Three examples are DARPA PM Larry Roberts putting out the ARPAnet contract, J.C.R. Licklider’s visionary Libraries of the Future Project, and BBNer Jordan Baruch’s early-1960s pitch to the NIH on a system to build a computer system to facilitate modern hospital operations. All three project initiators had Bell systems-engineer-like exposure to the people and problems of their field — technical backgrounds, regularly spoke with the best academic researchers, knew modern industry’s issues, were able to project the costs and complications of potential projects, etc.

Few have used systems engineers as effectively as Bell Labs, with Bell-style goals, since the great lab was broken up. It would be amazing to see a lab like Answer.AI commit significant staff time to this purpose.

The Cautionary Tale of Thinking Machines Corporation’s Funding

The case of the Thinking Machines Corporation (TMC) is not as directly instructive as the examples above, but TMC made one key mistake that makes it worth mentioning. TMC put itself in the unfortunate position of raising some of its funds from investors whose incentives were not aligned with theirs.

For those who don’t know, TMC was a complete failure as a commercial firm. As a result of its bankruptcy, many write the firm off as a holistic failure. But the firm did accomplish many of the technical goals it set out to achieve. Since the company was conceived with technological goals in mind, rather than a specific market, this was no small feat. The company was founded by Danny Hillis, a PhD from Marvin Minsky’s lab at MIT. Through TMC, he sought to build the machine he conceptualized in his graduate thesis: a truly parallel computer to improve the capacity of all scientists. The young company recruited the best researchers — including scoring Richard Feynman as its “intern” for several summers — and achieved many technical goals that helped pave the way for the modern field of parallel computing. Jeremy, who knows far more about the technical aspects of parallel computing than I do, sang TMC’s technical praises in our first conversation. He emphasized how shocking it was that TMC seemed to be the first to employ so many methods the field still uses today.

However, TMC’s high-powered team and great technical work were not enough to overcome their management follies. The firm spent money as if its financial standing was in line with its technical reputation, which it was not. In retrospect, some of these management decisions — such as a comically expensive long-term lease — could have been avoided without modifying the company’s general approach. However, on a deeper level, there was a dissonance between the company’s technical goals and the funding it raised.

TMC’s two major funders were DARPA’s computing office and private investors. The goals of Hillis and his top-flight technical staff were only aligned with DARPA’s goals. In looking to build the technically most ambitious parallel computer possible, DARPA funding was ideal. The DARPA computing office also felt that TMC’s work was progressing exceptionally well for most of its early years. However, the level of enthusiasm Hillis and the technical staff had for building a machine for science did not bleed into the most profitable areas — like deploying the machine on banking databases or managing logistics for Walmart. As time wore on, it seems that pressures were beginning to mount for TMC to pursue work more in line with those areas. It seems highly likely that TMC would have had to disappoint one of its funders sooner or later, even if it spent funds more wisely in its early years.

Had TMC just raised funds from DARPA and spent them much more modestly, the company might still exist today; it may have even earned a reputation beyond that of NVIDIA.

Copying mid-20th C. Industrial R&D Models is Hard for Incumbents

The middle 20th Century saw both the rise and fall of ambitious American industrial R&D labs. It is not just nostalgia that makes modern researchers look back on these labs with fondness. Not long after these labs were formed, it was becoming clear to many top researchers that the model was special.

In 1927, Karl Compton wrote a prescient letter to Science that praised these labs. At the time of writing the letter, Compton was the head of Princeton’s Physics Department and a part-time GE contractor. His letter asserts that these organizations were doing some of the best science in the country, even though the top universities often had the best men. A portion of the letter, in which Compton praises the labs’ management of scientific projects, reads:

This has been strikingly demonstrated in some of the big industrial research laboratories, from which the output has greatly exceeded the individual capacities of the research workers.

Compton believed there were many lessons that university departments should steal from these exceptional industrial labs. The first was the need to specialize when building a portfolio of researchers and projects. Why should every department attempt to loosely approximate the makeup of researchers and research questions in the field as a whole? No company would ever do such a thing. He believed that, “Such a policy dissipates effort, and if every institution followed it we should have the spectacle of a great many universities all very much alike and all with struggling, mediocre departments.”

On an organizational level, Compton believed the equilibrium of N autonomous professors with N separate budgets and a few grad students in their lab under their control was just not efficient for most projects. It would be silly for all departments to function that way. In the letter, Compton proposes what I’ve taken to calling a “Compton Model” research department. This model is far more structured than a department of mostly autonomous professors doing ad-hoc research with their own separate funds. Compton describes it as follows:

There is another direction in which more effective organization is possible within the universities themselves! Departments of a somewhat more flexible nature than those to which we are accustomed and which could, more than now, be built around one or two outstanding men in the department, could give these men an opportunity for organization and concentration of effort which is now rarely possible.

Another way to describe this model is as a “fund department heads, not projects” model. The model allows one or two individuals to largely shape the research vision, hiring, project selection, capital purchasing, etc. of an entire department at once. In addition, it allows these individuals the latitude to replace salaries spent on additional professors or grad students with full-time engineers or discretionary capital expenses as needed.

The model makes perfect sense. However, it’s remarkably difficult to make happen at a real university. Compton couldn’t succeed in doing so when he took over as MIT President. The existing stakeholders and structures are just too hard to shift in this direction. Absent some special circumstance, most university administrators wouldn’t find the idea even worth considering. However, CMU’s President Cyert was able to build a department that loosely resembled a Compton Model department. And its results were exceptional! But this was the exception. Pivoting an existing department to run like a Compton Model department has proven infeasible in almost all cases.2

The great, old models of managing labs didn’t disappear from industry because they lacked scientific merit. These labs largely began to disappear in the 1970s and 1980s. The 1970s saw a deep recession, which usually hurt R&D budgets. In the 1980s, new corporate management trends surfaced that led to companies being managed more myopically than ever before or since. These, along with other non-scientific trends, were largely responsible for the labs going away when they did. With them, the operational know-how that had slowly been built up throughout the century dissipated. Now, to learn how these orgs operated, one must read oral histories or talk to now-retired engineers.

The first thing these retired engineers will often tell you is that we should bring the old models back! Bell’s long-time researchers had confidence in the Bell model even as a late-1900s court case ended Bell Labs as we know it. In the 1920s, Compton dreamt of copying much of the GE model to use within a university. One should not be afraid to put their time and resources behind bringing these models back. Great engineers largely maintained confidence in these models from their inception until they faded into the background.

Many in academia know it should operate more like BBN, CMU, or GE research. They just can’t change the structures to make it happen. Many R&D leaders know that their company should think more in 10-20-year time horizons when planning research expenditures, but shareholders and shareholder-wary CEOs often do not find this view actionable. Answer.AI can be free from all this. The org can apply the great old models in a new era on new technology.

(Ries being a cofounder of Answer.AI should be comforting. Ries has railed against the types of myopic management trends that make activities like maintaining an expansive industrial R&D lab difficult for firms. He founded the young Long Term Stock Exchange, in part, to help mitigate issues like this.)

Conclusion: Running Your Own Races

It’s up to Answer.AI’s two-headed management team to stick to the organization’s comparative advantages. There is no need to race academic CS researchers or corporate R&D departments in races those two groups feel incentivized to run. Corporate R&D, NSF-funded CS researchers, and AGI-focused labs all have areas in which they clearly will and won’t operate. And those areas don’t come close to covering 100% of the good ideas somebody should obviously be working on. Answer.AI is free to run its own races, uncontested.

Many will feel the org is using an untested model. However, Answer.AI’s founders — like me — believe that this model is proven, but has simply gone away for a while. So, shouldering this “organizational risk” that others are seemingly not willing to do, they have the chance to work on problems without much competition. If their $10 million experiment works, it has the chance to spark a rush of emboldened researchers and engineers to found small research firms, leveraging the models of the once-great dragons of American industrial R&D.

I wish Jeremy and Eric luck in the early stages of their mission. To any researchers and engineers — across all areas — reading this and wishing a BBN, CMU, TMC, or Answer.AI existed in your area, please reach out to me on Twitter. I’d love to see if there’s anything I can do to help.

Thanks for reading:)

Subscribe now

Share

Don would become known for developing computing methods to take advantage of the increasingly online nature of scientific papers — leading to discoveries in a field he knew little about, biology.

The first paper in a series that would come to be his career-defining work was released in the April 1986 edition of The Library Quarterly. The paper, titled “Undiscovered Public Knowledge,” is a fascinating piece of first principles thinking that is even more interesting today than it was in 1986 — Swanson’s big idea was not dependent on the technology of his time, but limited by it.

Subscribe now

Share

This post is a much longer writeup to an abbreviated response I gave to Michael Retchin when, at coffee, he asked I had any ideas for incorporating LLMs into a research workflow. It is also a bit of a break from my pieces on industrial R&D labs and applied research — which are my primary focus and will be resuming in the coming weeks.

In the first section, I tell the story of Don Swanson and his use of computers to identify undiscovered public knowledge. In the second section, I detail how Mullainathan and Ludwig, recently, have devised a pipeline — enabled by modern ML — to generate novel social science hypotheses.

Then, in the third and final section of this piece, I enlist the help of an actual life scientist — Corin Wagen. Corin gives a practical breakdown of how one could deploy the general ideas covered in the first two sections of this piece to the field of organic chemistry.

The first two sections and the conclusion are written by myself; the third section was written by Corin. Back to the action!

In spite of Swanson's 1986 paper being the first in a very practical series, 22% of the citations in the article cite a non-scientific researcher: Karl Popper. Popper, the great philosopher of science, outlined a model of how science works in practice defined by an interplay of three worlds. Popper's model of these three worlds is what set the stage for Swanson’s simple idea which, in Popper’s era, would not have been at all feasible.

Popper’s model, as Swanson saw it, saw the truth-seeking that made up the real scientific process as a messy and failure-prone process. This messy process is what Gerald Holton referred to as ‘science-in-the-making’ or S₁. While science-in-the-making is much messier than the clean write-ups that appear in journals, it is how great science is actually done.

Swanson further broke down his reading of Popper and how he viewed the body of scientific knowledge contained in existing research, saying:

The question of what is or is not real can in part be fought out on the battleground of critical argument, with various scholars corroborating or correcting the perceptions of others.

Knowledge begins therefore with conjecture, hypothesis, or theory, all of which mean about the same thing. Scientific knowledge grows through testing and criticizing theories and through replacing theories with better ones that can withstand more severe tests and criticism. Thus knowledge is constructed of conjecture, and, though filtered through reality, remains forever conjectural. The sine qua non of science is not objectivity or even "truth," as is often thought, but a systematically self-critical attitude. Scientists are expected to propose testable theories and to be diligent in seeking evidence that is unfavorable to those theories. If they do not criticize their own work, someone else will.

Public criticism and published argument are crucial in helping the scientist create a product that can rise above his own prejudices and presuppositions. Although scientists and scholars may never be objective, the published products they create, shaped and weeded by criticism, can move ever closer to objectivity and truth.

Popper’s model of the three worlds of science contained:

• World 1, the physical world. This is the actual world which scientists and engineers concern themselves with.

• World 2, researchers’ swirling thoughts about the physical world. This is the subjective world of “mental states and mental processes” that happen in individuals’ heads. World 2 can be taken to mean many things. But, for the sake of this piece, we can take World 2 to mean the collection of personal experiences, ideas, and individual thought processes that pertain to the physical world (World 1) but are not exactly formal scientific theories or fully fleshed-out ideas. The beginnings of theories are based on some vast collection of experiences with the physical world present in any individual researcher’s head. This collection is contained in World 2.

• World 3, the (mostly) solidified body of scientific knowledge. This world contains problems, theories, and other products of the human mind. As Swanson noted, “World 3 is real in that, through interacting with World 2, it can influence World 1.”

Popper pointed out the interesting implication that World 3 was created by man, but it could give rise to problems unforeseen by its creators. As one example, man created a number system, but with the number system came “an infinity of unintended and unforeseen consequences, including prime numbers.” Each of these unforeseen consequences awaited discovery. Many of these relevant unforeseen consequences have been discovered, but a countless number of them have also not been.

Swanson reiterated this point, saying:

World 3, while created by man, contains far more than man has ever thought of or dreamed about. World 3 indeed must contain ever increasing quantities of undiscovered knowledge.

The objective state of World 3, and what we subjectively know about World 3, are quite different concepts. So far as either certainty or "truth" is concerned, I shall show that World 3 must be in principle unknowable in the same sense that World 1 is unknowable. In either case, we cannot know, we can only guess. Only a small portion of World 3 is known to any one person. Some things we can perhaps know reasonably well, such as a theory that we ourselves have invented. But, once our invention becomes public knowledge-a bona fide resident of World 3 — it takes on a life of its own.

There are many implications of this conclusion, but the one most relevant to this piece (and Swanson’s work) is that many great, new ideas can be found by a researcher primarily concerning themself and their research process with only the observations in World 3 — rather than in World 1. In Swanson’s eyes, only so much of World 3 is ever knowable to any one researcher or research team’s brain. Computers and new methods of information retrieval, in 1986, were starting to change that. This led to practical implications that Popper's model did not explore — but which Swanson's work would.

Swanson had learned in his now two decades in his adopted world of library science that the role of librarians and their indexing methods were vital to the scientific process. Individual researchers were only familiar with certain aspects of the scientific literature, and librarians had developed methodologies to, as best they could, help researchers identify and gain access to the public scientific knowledge which they did not know how to find. When in this position, a researcher often does not know exactly what they're looking for, how to explore the literatures of related fields, how to account for the fact that some concepts go by multiple terms, etc.

Swanson — and anyone who has ever used a library index — knew the librarians' methodologies were better than nothing, but also extremely limited. Interestingly, he noted, “Information retrieval is necessarily incomplete, problematic, and therefore of great interest — for it is just this incompleteness that implies the existence of undiscovered public knowledge.”

He then went on to explain several hypothetical examples in which advancements in information retrieval could have extremely practicable scientific implications. One of these examples was titled, ‘A Missing Link in the Logic of Discovery:’

A more complex, and perhaps more interesting, example will further illuminate the idea of undiscovered public knowledge. Suppose the following two reports are published separately and independently, the authors of each report being unaware of the other report: (i) a report that process A causes the result B, and (ii) a separate report that B causes the result C. It follows of course that A leads to, causes, or implies C. That is, the proposition that A causes C objectively exists, at least as a hypothesis. Whether it does or does not clash with reality depends in part on the state of criticism and testing of i and ii, which themselves are hypotheses. We can think of i and ii as indirect tests of the hidden hypothesis "A causes C."

In order for the objective knowledge "A causes C" to affect the future growth of knowledge-that is, in order for it to be tested directly, or to reveal new problems, solutions, and perhaps new conjectures, the premises i and ii must be known simultaneously to the same person, a person capable of perceiving the logical implications of i and ii. If the two reports, i and ii, have never together become known to anyone, then we must regard "A causes C" as an objectively existing but as yet undiscovered piece of knowledge — a missing link. Its discovery depends on the effectiveness with which information can be found in the body of recorded knowledge. Notice that "A causes C" is not something once known but forgotten; it is genuinely new knowledge that awaits discovery by explorers of World 3.

Swanson, in this paper, proceeds to put this theory to the test, highlighting an A→B→C relationship which he used his own computing pipeline to identify. In this case, A was fish oil, B was the reduction of platelet aggregability and blood viscosity, and C was the risk of Raynaud’s disease. In the literature, it seemed that A caused B. In the literature, it seemed that B might counteract C. But there were no papers implying that A counteracted C.

Maybe it had been tried, failed, and not published due to publication bias. Maybe it was a silly idea for a reason that Swanson, not a biologist, was unaware of. Or maybe this was an idea that was brand new to World 3 and worthy of empirical testing by scientists who concerned themselves with World 1.

So, Swanson wrote the idea down and proposed that any interested life sciences research team test the hypothesis. It turned out that he was somewhat onto something! A paper published in the following years concluded:

We conclude that the ingestion of fish oil improves tolerance to cold exposure and delays the onset of vasospasm in patients with primary, but not secondary, Raynaud's phenomenon. These improvements are associated with significantly increased digital systolic blood pressures in cold temperatures.

Swanson’s most well-known finding using this A→B→C methodology, published two years after this initial paper, pertained to magnesium and the alleviation of migraines. The paper, titled “Migraine and Magnesium: Eleven Neglected Connections,” explored eleven possibly unnoticed connections between the pair of literatures on migraine and magnesium.

In the introduction of this paper, Swanson breaks down the necessity of this style of work even more clearly:

Undocumented connections arise neither by chance nor by design but as a result of the inherent connectedness within the physical or biological world; they are of particular interest because of their potential for being discovered by bringing together the relevant noninteractive literatures, like assembling pieces of a puzzle to reveal an unnoticed, intended, but not unintelligible pattern. The fragmentation of science into specialties makes it likely that there exist innumerable pairs of logically related, mutually isolated literatures.

Swanson then goes on to point out a litany of A→B→C style connections that connected A, magnesium levels, to C, the presence/severity of migraines. In spite of a number of connections, the number of papers in the literature that mentioned A and C together was vanishingly small. The eleven factors relevant to both migraine physiopathology and the physiological effects of magnesium were: type A personality, vascular tone and reactivity, calcium channel blockers, spreading cortical depression, epilepsy, serotonin, platelet activity, inflammation, prostaglandins, substance P, and brain hypoxia.

So, naturally, Swanson recommended that researchers in the medical community that concern themselves with World 1 look into this relationship to see if there was causality. And, once again, it seems that Swanson and his methods were vindicated. While magnesium treatments did not prove to be a silver bullet for migraine sufferers — obviously migraines still exist — magnesium has since become a remarkably common treatment to help alleviate migraine symptoms. A paper as recently as 2012, titled “Why all migraine patients should be treated with magnesium,” argued, “The fact that magnesium deficiency may be present in up to half of migraine patients, and that routine blood tests are not indicative of magnesium status, empiric treatment with at least oral magnesium is warranted in all migraine sufferers.”

Even if the reader is not impressed with those findings in and of themselves, the findings should surely be taken seriously as an extremely convincing proof of concept of the idea that undiscovered ideas are out there and can be practically discovered with the right scientific area and computationally-augmented workflow. These new hypotheses done using data from World 3 can then be tested and verified/rejected using traditional scientific methods on World 1.

These pipelines will never be perfect. And using them well may always be — as is the case for much of computing — more of an art or messy engineering process than a pure science. As Swanson pointed out, the problem of information retrieval is inevitably incomplete and is destined to not be perfectly efficient. But the areas of computing relevant to this problem area have improved by many orders of magnitude in the years since Swanson…and, as readers have surely noticed, to a shocking degree in the past year or two.

New functionality exists, and much is possible now that was not before. The pipeline which Swanson used to generate his insights was remarkably primitive by modern standards.

I'll (briefly) break down his process to give the reader an idea. For various diseases, such as migraine, Swanson scanned a Medline file in which 2,500 article titles contained the word “migraine.” One of those titles referred to a possible relation between spreading depression to the visual scotomata of classic migraine. Then, as Swanson detailed:

The phrase “spreading depression” is next explored on its own, by examining titles in which it occurs and reading the literature itself is necessary. One can discover among other things that magnesium in the extracellular cerebral fluid can prevent or terminate spreading depression. Combining this new fact with the above connection, one is led to the conjecture that magnesium deficiency might be a causal factor in migraine.

A surmise that migraine might be a vasospastic disorder led to another connection with magnesium. By examining several hundred titles containing the word “vasospasm,” one can notice a few titles indicating that magnesium deficit can cause vasospasm. That discovery can be made more efficiently if one guesses at the outset that a deficiency of some kind might be implicated. By requiring the Medline subheading “deficiency” to appear in the descriptor field, one can narrow the several hundred titles with “vasospasm” down to four, of which three are about magnesium.

Swanson points out that, of course, this methodology did not lead directly or exclusively to magnesium. It led to many other guesses, most of which were discarded. The biggest reason for hypotheses being weeded out was simply that two terms tended to be mentioned in the same article frequently. Swanson’s heuristic took that to mean the hypothesis had already been examined.

Swanson later put together a computer system called Arrowsmith which attempted to make carrying out these steps easier, but it never became widely used. This system was inevitably quite manual. And the use of computers in day-to-day knowledge exploration tasks was both not as common today and the machines themselves were much less capable.

But Swanson’s core idea should live on! Yes, his methods were simple. And, yes, his findings were not Nobel-winning (or anything close). But one can only imagine what he may have attempted had modern text processing, tf-idf scoring, or cosine similarity existed in 1986. Swanson only had keyword searching, manual skimming, and the faint idea that this should be possible available to him.

We now live in a different world — one Swanson only could have dreamed of. The task of rooting out undiscovered public knowledge and aiding human researchers in assembling this information into real-world scientific discoveries is absolutely something that, now, should be very seriously re-visited.

With the improvements in LLMs, it seems we are living in a world where language models — whether or not they ‘understand’ in a literal sense — have capacities that often mimic human capacity to understand conceptual phenomena beyond the literal words on a page. Creative minds can likely find ways to make use of this fantastic new tool in the hypothesis-generating process. Surely, at the minimum, Swanson's A→B→C methodology to find undiscovered public knowledge in World 3 is one viable way to do this. But I imagine even more fruitful frameworks for deploying modern computing tools to accomplish similar goals are out there to be discovered as well.

This will be explored in more detail in Corin's section which contains some ideas on how this could apply to modern organic chemistry research. But, first, I'd like to explore a recent paper on machine learning as a tool for hypothesis generation. The paper seeks to generate new hypotheses in a common area of computational social science, causes of bias in bail decisions, using non-LLM methods in modern ML — image recognition, morphing, and GANS.

Ludwig & Mullainathan’s “Machine Learning as a Tool for Hypothesis Generation”

Jens Ludwig and Sendhil Mullainathan recently released a working paper on machine learning as a tool for hypothesis generation. The paper demonstrates that a very different future of hypothesis generation could already be upon us — in very specific areas at least.

The authors open with a beautiful description that outlines how messy, human, and creative hypothesis generation usually is. It is a description that does a good job of unpacking the kind of activities that tend to reside in Popper’s World 2. The authors open the paper, writing:

Science is curiously asymmetric. New ideas are meticulously tested using data, statistics and formal models. Yet those ideas originate in a notably less meticulous process involving intuition, inspiration and creativity. The asymmetry between how ideas are generated versus tested is noteworthy because idea generation is also, at its core, an empirical activity. Creativity begins with “data” (albeit data stored in the mind), which are then “analyzed” (albeit analyzed through a purely psychological process of pattern recognition). What feels like inspiration is actually the output of a data analysis run by the human brain. Despite this, idea generation largely happens off stage, something that typically happens before “actual science” begins. Things are likely this way because there is no obvious alternative. The creative process is so human and idiosyncratic that it would seem to resist formalism.

That may be about to change because of two developments. First, human cognition is no longer the only way to notice patterns in the world. Machine learning algorithms can also notice patterns, including patterns people might not notice themselves. These algorithms can work not just with structured, tabular data but also with the kinds of inputs that traditionally could only be processed by the mind, like images or text. Second, at the same time data on human behavior is exploding: second-by-second price and volume data in asset markets, high-frequency cellphone data on location and usage, CCTV camera and police “bodycam” footage, news stories, children’s books, the entire text of corporate filings and so on. The kind of information researchers once relied on for inspiration is now machine readable: what was once solely mental data is increasingly becoming actual data.

We suggest these changes can be leveraged to expand how we generate hypotheses. Currently, researchers do of course look at data to generate hypotheses, as in exploratory data analysis (EDA). But EDA depends on the idiosyncratic creativity of investigators who must decide what statistics to calculate. In contrast, we suggest capitalizing on the capacity of machine learning algorithms to automatically detect patterns, especially ones people might never have considered. A key challenge, however, is that we require hypotheses that are interpretable to people. One important goal of science is to generalize knowledge to new contexts. Predictive patterns in a single dataset alone are rarely useful; they become insightful when they can be generalized. Currently, that generalization is done by people, and people can only generalize things they understand. The predictors produced by machine learning algorithms are, however, notoriously opaque — hard-to-decipher “black boxes.” We propose a procedure that integrates these algorithms into a pipeline that results in human-interpretable hypotheses that are both novel and testable.

The problem area on which the authors chose to test their new method is one they know well: biased bail decisions.

Contextualizing the Problem

In recent years, Mullainathan, Jens Ludwig, and a handful of other researchers who straddle the worlds of social science and computer science have been producing a widely-followed series of papers on algorithmic bias. Many of the papers in this area focus on bail decisions. It is a somewhat ideal environment in which to do this kind of research, all things considered. The outcome variables are public, there is a lot of (bias-prone) human discretion from judges’ decisions, statistical algorithms are sometimes used to determine risk scores to jail or release defendants, both numerical data and text are available on the crimes, and additional data is available about the defendants themselves — even their photos in some cases.

Researchers have been utilizing this data to ask and answer questions such as: Can we produce bail algorithms that are more accurate than judges?, Can we produce bail algorithms that minimize seemingly unfair racial bias?, or What features seem to be most associated with unfair treatment from judges?

The latter question is the one that Ludwig and Mullainathan chose to be their subject of investigation in this paper: Can we use machine learning to help generate new theories for what features of a defendant seem to bias a judge one way or another? This question is particularly interesting because, as in many areas with large datasets available, the machine learning algorithms far outperform the hypothesis-based statistical models put together by people like economists — which are often regression models.

The machine learning algorithms are clearly seeing some patterns that are just not captured by existing hypotheses. So, Ludwig and Mullainathan set out to see if they could get similar neural net-based approaches to unpack what these complex models were seeing.

And…it worked. Their methods discovered hypotheses that were interpretable, novel, and seemingly right.

The disparity between ML models and regression models in this area

When the researchers built deep learning models to predict judges' decisions, the defendants mugshot photos proved to be extremely predictive of judge behavior. This, on its own, accounted for a quarter to half of the predictable variation in detention — depending on whether one uses R-squared or AUC. Contextualizing this for the readers, the authors write, “Defendants whose mugshots fall in the bottom quartile of predicted detention are 20.4 percentage points more likely to be jailed than those in the top quartile. By comparison, the difference in detention rates between those arrested for violent versus non-violent crimes is 4.8pp.”

Of course, some percentage of what the algorithm was ‘seeing’ in the photo was inevitably going to be things that have already been hypothesized and accounted for in the existing social science literature and their regression models. Some of these prior hypotheses include things like: gender, race, race, skin color, and other facial features suggested by existing research.

Panel A of Figure I in the paper demonstrates that the authors' ML algorithm is finding far, far more than what the causally-focused models account for. Even if one adds in a column that accounts for human guesses of judge behavior based on the photos, only statistically accounting for those factors does not come close to the algorithm’s prediction of judge behavior. In the figure below, the panel on the left indicates which variables are being controlled for in the rightmost panel where the R-squared achieved by the existing hypotheses in the literature is compared to the R-squared of a machine learning algorithm using that information plus the mugshot pixel information.

Now, while impressive, that result is not in and of itself groundbreaking. We have known for a while that basic statistical methods such as OLS often cannot touch the performance of machine learning algorithms when dealing with high-dimensional data such as cell phone data, online clickthrough data, images, text, etc.

What has vexed us is the ability to interpret what’s happening in a human-explainable way. Even attempting to understand what neural nets are doing as they learn primitive tasks such as basic addition is a handful for remarkably intelligent researchers. Explainable hypotheses on applications like this, many feel, have seemed quite out of reach in the near term.

But that is exactly what the authors succeed in doing a convincing job of. In the end, they are able to produce a very similar chart to the one above — demonstrating marked improvements based on new, human-interpreted, ML-based hypotheses rather than just a black box algorithm.

Their new pipeline, in part, leverages M-Turkers learning what the algorithm was seeing. The M-Turkers' descriptions of what the algorithm was seeing — in word form — became hypotheses. These hypotheses were fed to separate M-Turkers who added those hypotheses — as text labels — to the test set. And the basic statistical model — similar to the simple ones often used in social science literature — was able to achieve substantial increases in performance by controlling for these new machine-discovered and human-interpreted features.

Let’s explore how exactly they did this in more detail. (Feel free to only skim the steps of the pipeline section if you don’t care to know the details.)

The Pipeline

This section contains a condensed version of the steps that made up the authors' workflow for generating human-interpretable hypotheses leveraging machine learning methods. Please check out the paper if you’d like to know the minutiae of how certain individual steps were performed. (The paper plus appendix came out to 130 pages, so the authors really do get into a lot of detail on all the steps for those curious.)

Step 1: Identification of a statistical trend of interest

In this case, specifically, the authors built a supervised learning algorithm — with an outcome variable based on judges' bail decisions — and identified that a surprising amount of the predictable variation in judges’ decision-making came from the defendants' faces. They did this using an ensemble approach with gradient-boosted decision trees on the structured administrative data (current charge, prior record, age, gender, etc.) and a convolutional neural net trained on the raw pixel values from the mugshots.

Step 2: Checking if the trend of interest is ‘unknown to the literature'

This is the step displayed in the graphic in the preceding section — comparing the R-squared of the ML approach to the simpler models which control for hypotheses from the social science literature. The authors ran an OLS regression controlling for the variables that represent positive hypotheses in the existing literature to see how much of the variation, if any, that the ML ensemble method identifies that is not accounted for by these variables. The answer was: a lot.

To do this, for this particular use case, the authors needed to employ people to label some images with things like race, skin tone, etc. What this step looks like will vary based on the application. But what matters for this high-level overview is that the authors did what they needed to do to label the data so previously proven hypotheses could be controlled for. With that done, the algorithm was still finding a ton of predictable variation that those variables did not account for.1

Step 3: Morphing

This step is the most pivotal (and technically complex) of the paper. The authors write:

The algorithm has made a discovery: something about the defendant’s face explains judge decisions, above and beyond the facial features implicated by existing research. But what is it about the face that matters? Without an answer, we are left with a discovery of an unsatisfying sort. We will have simply replaced one black box hypothesis-generation procedure (human creativity) with another (the algorithm).

The authors need to find a way to unpack this in a human-understandable way. That is the crux of the paper. First, they explain why common computer science methods like saliency maps do not do the job for cases like this.

A common solution in computer science is to forget about looking inside the algorithmic black box and focus instead on drawing inferences from curated outputs of that box. Many of these methods involve gradients…The idea of gradients is useful for image classification tasks because it allows us to tell which pixel image values are most important for changing the predicted outcome.

For example, a widely used method known as “saliency maps” uses gradient information to highlight which specific pixels are most important for predicting the outcome of interest (Baehrens et al., 2010; Simonyan et al., 2014). This approach works well for many applications like determining whether a given picture contains a given type of animal, a common task in ecology (Norouzzadeh et al., 2018). What distinguishes a cat from a dog? A saliency map for a cat detector might highlight pixels around, say, the cat’s head: what is most cat-like is not the tail, paws or torso, but the eyes, ears and whiskers. But more complicated outcomes of the sort social scientists study may depend on complicated functions of the entire image.

The authors elaborate:

In the cat detector example, a saliency map can tell us that something about the cat’s (say) whiskers are key for distinguishing cats from dogs. But what about that feature matters? Would a cat look more like a dog if its whiskers were, say, longer? Or shorter? More (or less?) even in length? People need to know not just what features matter but how they must change to change the prediction. For hypothesis generation, the saliency map under-communicates with humans.

For something like a human face’s age, age is a function of almost all parts of our faces. So, a saliency map highlights almost everything. That isn't helpful.

So, the authors pursue a “morphing” procedure. Instead of highlighting pixels, they change them in the direction of the gradient of the predicted outcome. This, essentially, creates a new synthetic face that has a different predicted outcome. They describe the idea as follows:

Our approach builds on the ability of people to comprehend ideas through comparisons, so we can show morphed image pairs to subjects [M-Turk workers] to have them name the differences that they see.

The top row of the following graphic shows how, for a factor like age, the saliency map covers the entire image. The bottom panel of the graphic demonstrates the authors' morphing procedure done in a way that seeks to increase predicted age — just as an example.

The authors constrain their morphing procedure in such a way that it attempts to change as little as possible about the images while maximizing the detention risk of the synthetic images. In addition, they also construct the procedure in such a way that the morph produces differentiation orthogonal to traits that are already accounted for by hypotheses in the existing literature. In short, these are new faces are minimally morphed to achieve a maximal increase in the probability of certain bail decisions. And they have been morphed in such a way that the traits that have been changed are not traits accounted for in the existing literature.2

Step 4: Humans prove they can learn new, predictive features from the morphs

The researchers then ask M-Turkers to look at image pairs — with one morphed image morphed to have a higher predicted harshness of sentence than the other. These subjects, after being given a short set of images and correct answers to have a chance to ‘learn’ the pattern, were then shown many pairs of images and asked to guess which image expresses a higher likelihood of a harsh bail decision. The M-Turkers, on average, improved from more or less 50/50 guessing at the start of the round to a 67% accuracy rate by the end of the round.

You can imagine the following image — with the caption under the photos showing the correct answer — representing what the training process looked like.

As a point of reference: the authors also generated the following ‘intermediate steps’ for that particular photo to give us an idea of what the image morphs look like for detention probabilities between .41 and .13

Step 5: Humans name the heuristics they were using to make more accurate predictions

Next, given a set of humans clearly just demonstrated that they were learning the differences between the morphs somehow and using the knowledge to make more accurate predictions, the researchers simply asked them to describe what they were doing.

The M-Turkers would have just looked at many pairs of images that look like those below and been asked to describe what traits separate one column from the other.

The authors gave these text descriptions to separate research assistants and asked them to create basic category labels that captured the mostly synonymous human descriptions. The humans' unstructured text comments were obviously a bit noisy, but the authors show that — even in a word cloud — a pretty clear first hypothesis starts to emerge. Some of the most common words are things like “cleaner,” “shaved,” “scruffy,” “moustache,” and “shorter.”

The researchers called this feature “well-groomed.” According to the humans who were making accurate predictions, being well-groomed helps a lot!

Step 6: Checking the new hypothesis' validity by adding it into the regression model

Now, with this new hypothesis in place, the next step is to check if it holds up in practice. So, in the next step, the researchers asked a new set of 343 humans to label over 32,000 mugshots in the training and validation set according to how well-groomed they were on a 9-point scale. Once they did this, the authors found that this well-groomed variable was very highly correlated with the algorithm's predictions. A 1-point increase in standard deviation was associated with a 1.74% decline in predicted detention risk. A 1.74% decrease in detention risk is a 7.5% reduction in the base rate.

The new variable also significantly increased the predictiveness of the new version of the full model. The authors break down the explanatory power of the new variable, saying: “Another way to see the explanatory power of this hypothesis is to note that this coefficient hardly changes when we add all the other explanatory variables to the regression…despite the substantial increase in the model’s R-squared.”

With that, it seems they've found a new, ML-enabled, human-interpretable hypothesis that is likely correct: being well-groomed seems to substantially bias judges in your favor.

Step 7: Iterating

Next, the authors did it again!

The cool thing about this procedure is that it's iterable. This well-groomed variable explains some of the variation in the algorithm’s prediction of the judge, but not all of it. So, the authors continue with the above steps — now with a defendant’s well-groomed-ness labeled in the dataset alongside things like race and charge — and attempt to find another novel hypothesis orthogonal to what has already been labeled and accounted for in the data.

The authors write:

Note that the order in which features are discovered will depend not just on how important each feature is in explaining the judge’s detention decision, but also on how salient each feature is to the subjects who are viewing the morphed image pairs. So explanatory power for the judge’s decisions need not monotonically decline as we iterate and discover new features.

To isolate the algorithm’s signal above and beyond what is explained by well-groomed, we wish to generate a new set of morphed image pairs that differ in predicted detention but hold well-groomed constant. That would help subjects see other novel features that might differ across the detention-risk-morphed images, without subjects getting distracted by differences in well-groomed.

At this point, they note that this iteration procedure raises several technical challenges because, if they used the same procedure as in the first step, they would be orthogonalizing against predictions of well-groomed rather than objective well-groomed-ness — and orthogonalizing against a prediction can be an error-prone process. So, instead, they built a new detention prediction algorithm curated on a training set that was limited to pairs of images matched on the features that they were looking to orthogonalize against. In this particular step, that meant that they could, as they put it, “use the gradient of the orthogonalized judge predictor to move in GAN latent space to create new morphed images that have different detention odds but are similar with respect to well-groomed.”

These new images, shown below, were then put through the same steps as before. The authors also note that — as you will see — the morph generator did not do a perfect job of orthogonalizing against well-groomed-ness. But what is important is that there is still what they believe to be “salient new signal.”

With these image pairs, a second new salient facial feature comes to light in the eyes of the M-Turkers: how “heavy-faced” or “full-faced” a defendant is.

In fact, heavy-faced seemed to be even more important than well-groomed was. A one standard deviation improvement in the well-groomed variable was associated with a 7.5% reduction in the base rate. A one standard deviation improvement in the full-faced variable was associated with a 9.3% improvement.

The authors, once again, have the M-Turkers go through and add this label to the relevant dataset so they could run the regression model controlling for this new variable as well. Once again, it yielded an improvement.

The authors conclude their section on ‘Iteration,’ saying:

In principle, the procedure could be iterated further. After all, well-groomed, heavy-faced plus previously known facial features all taken together still only explain 27% of the variation in the algorithm’s predictions of the judges’ decisions. So long as there is residual variation, the hypothesis-generation crank could be turned again and again. But since our goal is not to fully explain judges’ decisions but rather to illustrate that the procedure works and is iterable, we leave this for future work.

Step 8: Evaluating the new hypotheses on true outcome data — rather than predicted outcome data

Before this can be considered a success, one has to check if these new hypotheses are good predictors of actual judge decisions. Up until this point, the models were largely being validated on the (imperfect and noisy) ML predictions of judge predictions rather than actual judge decisions. So, this step is the one that really matters. Well-groomed and full-faced could be correlated with some part of the algorithm that is actually unrelated to actual judge behavior.

When the authors did this, building a regression model using all of the original variables as well as well-groomed and full-faced, they noted that “the explanatory power of our two novel hypotheses alone equals about 28% of what we get from all the variables together.”

The gray dot in the bottom-most row in the right panel demonstrates the improvement of the regression model when well-groomed and heavy-faced are controlled for. The magnitude of improvement when including both of these variables is similar to when skin tone was found to matter more than race in many cases — which was considered a very significant and not necessarily intuitive finding made by humans in this field of social science.

The implications of all this

After running an experiment at a public defender’s office and legal aid society, the authors determined that it did not seem that the practitioners previously understood the extent to which these newly-discovered characteristics correlated with outcomes. The authors noted that they were mentioned here and there by some of the practitioners, “but these were far from the most frequently mentioned features.”

The researchers also had the practitioners guess which of a pair of images was more likely to be detained — just as the authors did with M-Turkers. They found that the effect of the practitioners' guesses in the performance of the full model was four times as large as the M-Turker guesses. In fact, the practitioners' guesses were about as accurate as the algorithm itself. The researchers elaborated on the relationship between the practitioners' predictions and the algorithm's predictions, saying:

Yet practitioners do not seem to already know what the algorithm has discovered. We can see this in several ways in Table VI. First, the sum of the adjusted R-squared values from the bivariate regressions of judge decisions against practitioner guesses and judge decisions against the algorithm mugshot-based prediction is not so different from the adjusted R-squared from including both variables together in the same regression…We see something similar for the novel features of well-groomed and heavy-faced specifically as well. The practitioners and the algorithm seem to be tapping into largely unrelated signal.

This pipeline — leaning heavily on neural nets and a human-feedback loop leveraging basic human competencies — generated a new, good idea.

While Swanson’s idea was an example that used primitive technology in the general area to which LLMs can contribute — text-based idea discovery — Mullainathan and Ludwig demonstrate how modern neural nets can be leveraged to generate clever theories in a completely different realm of application.

Is what the computer is doing creativity? I’d say no…but I’d also say it doesn’t matter. The increased capacity of modern computing tools simply is. It's up to us to find ways to use the new capacities well. If researchers use them well, it should free up human minds to focus their efforts on entirely new areas of creativity than they have before. This won't impact all hypotheses. Many hypotheses will come to be in the same ways that they always have. But, hopefully, many hypotheses will also come to be through new combinations of humans and computing. Both Don Swanson's work from the 1980s and the morphing work of the more recent authors lay a fantastic base for understanding what is now possible in this realm.

Researchers should be taking advantage of these ideas wherever they can! In line with this goal, the following section contains one researcher's ideas for possible applications of these ideas which could be fruitful in his own field: organic chemistry.

The ideas are largely inspired by Swanson’s A→B→C methodology and the work that led to the (recent) second Nobel Prize in Chemistry for Karl Sharpless.

Can LLMs help enable 100 more Karl Sharplesses?

Tools that accomplish Don Swanson-like tasks are particularly useful in areas of science in which the capacity to read and remember a large volume of research papers — from all languages and time periods — is a distinct advantage. You could think of this as having a bunch of really dumb research assistants who are very well-read and have great memories — but they're still dumb. One current research area in which this would be a distinct advantage is the field of organic chemistry.

The rest of this section was written by Harvard Ph.D. organic chemist, blogger, and current founder, Corin Wagen. Corin was the first person to tell me about Karl Sharpless' idea-generation process. Corin, after reading a draft of the Don Swanson and Sendhil & Ludwig sections, set out to write a section describing how he thought those ideas could fit into his own field in the near-term and make a big difference.

Several factors make organic chemistry a promising domain for LLM-assisted hypothesis generation. Organic chemistry is a very old field, with almost two centuries of publications to sift through, and there are a lot of venerable results which have been forgotten and unearthed with great fanfare (vide infra). Additionally, it’s relatively easy to validate the accuracy of previous results in organic chemistry by reproducing the reactions—few organic reactions require bespoke instrumentation, and there’s an expectation that one lab’s reactions ought to be easily reproducible by other labs (e.g.).

The work of K. Barry Sharpless illustrates how knowledge of the old literature can allow for important breakthroughs today. Since around 2000, Sharpless has been advocating for a new paradigm in chemical synthesis, termed “click chemistry,” for which he was co-awarded the 2022 Nobel Prize. In his words:

The goal [of click chemistry] is to develop an expanding set of powerful, selective, and modular “blocks” that work reliably in both small- and large-scale applications. We have termed the foundation of this approach “click chemistry,” and have defined a set of stringent criteria that a process must meet to be useful in this context. The reaction must be modular, wide in scope, give very high yields, generate only inoffensive byproducts that can be removed by nonchromatographic methods, and be stereospecific (but not necessarily enantioselective).

This approach aims to avoid many of the problems that render chemical synthesis so inefficient and capricious today, and thus accelerate chemical discovery. But it’s not easy to find reactions with the desired characteristics: to date, Sharpless has focused on two reactions: copper-catalyzed azide–alkyne cycloaddition (CuAAC) and sulfur(VI) fluoride exchange (SuFEx). Importantly, both of these reactions were discovered long ago: CuAAC was developed by Rolf Huisgen in the 1960s (ref), while SuFEx dates from the 1920s German literature (ref).

These reactions, thus, are prime examples of undiscovered public knowledge. In both cases, the actual experimental results had been published for many decades, yet their relevance to modern synthesis wasn’t realized until someone (Sharpless) put the pieces together. And it wasn’t just the novelty of the click chemistry concept that led to these results being forgotten: the original click chemistry paper was published in 2001, yet SuFEx wasn’t rediscovered until 2014, meaning that there was about a decade in which anyone could have recognized that these 1920s sulfur(VI) reactions satisfied the demands of click chemistry, and yet no one did.

What makes Sharpless so successful? In part, it’s his vision for how chemical synthesis could be different — having the creativity and courage to be iconoclastic and push the field in a new direction. But it’s also his ability to discover and remember old results that fit with his vision that’s made him so successful. It’s not really possible to sit down and design a reaction that satisfies a list of desired properties; it’s much more effective to simply look through all previous reactions and find one that meets your criteria.

If we consider a cartoonish version of the Sharpless production function, then, we might arrive at the following list:

1. Have a bold vision for the future of the field.

2. Read a lot of old papers and remember them.

3. Speak German.

4. Have a good “eye” for promising old results that might fit into your vision.

Attributes 1 & 4 are likely to remain the domain of humanity for the foreseeable future — it’s silly to expect LLMs to develop good taste or a keen scientific vision. But attributes 2 & 3 are ripe for automation: an LLM can digest papers much more quickly than a human, and in far more languages than any scientist can learn. Perhaps there are future potential Sharpless-type scientists out there who have been rate-limited by the speed at which they read, or their inability to speak German!

Unfortunately, application of LLMs to these questions is hamstrung by the fact that ChatGPT and its ilk don’t have access to most research papers, which are stuck behind publisher paywalls. But one could imagine LLMs being very useful research assistants if this limitation were removed. To demonstrate, I asked ChatGPT to suggest a click reaction of allenes, another bench-stable but high potential energy functional group that (to my knowledge) no one has proposed a click reaction for. ChatGPT responded with a very promising lead, the “allene–amine coupling reaction”:

This answer is quite good: ChatGPT identifies that the reaction is highly chemoselective, displays excellent functional group tolerance, and proceeds in water as a solvent, all qualities that match the criteria for inclusion as a click reaction. Unfortunately, the reaction doesn’t exist: Google doesn’t return any results for “allene–amine coupling,” the closest examples I can find don’t match the descriptions given at all (poor functional group tolerance, not compatible with water), and the references ChatGPT gives me upon further interrogation are completely made up.

Despite this failure, the response is encouraging. ChatGPT clearly understands what it’s supposed to be looking for (type of reaction, behavior with different functional groups, solvent compatibility), but is hamstrung by the fact that it doesn’t actually seem to know chemistry beyond the level of Wikipedia. This is a pretty fundamental limitation of LLMs on their own, but not an insurmountable one: the use of retrieval augmentation and similar techniques has been demonstrated to allow ChatGPT and other LLMs to reason much more accurately based on extant data, and there’s no reason that scientific data ought to behave any differently.

While there are plenty of (proprietary) ways to search the literature based on chemical structures (like SciFinder and Reaxys), the ability of LLMs to perform semantic search opens the door to a wider variety of applications. A chemist who knows the properties they wish their reaction to have, but not the reagents involved, would be hard-pressed to use state-of-the-art tools to find an answer. But one could imagine a version of ChatGPT equipped with the ability to semantically query a database of patents and scientific publications (with the information stored in a machine-friendly way) being an extremely useful research assistant to the practicing organic chemist. Armed with such a tool, a chemist could ask plenty of useful questions:

• “I’m looking for photochemical cyclization reactions that generate piperidines, published before 1960.”

• “Robust and functional-group tolerant reactions that couple allenes to another molecule and work in water.”

• “Regioselective cycloadditions of bench-stable starting materials that occur at room temperature”

These prompts are presently very challenging to answer, limiting the ability of scientists to locate relevant literature results, but hypothetically ought to be easy enough to answer based on the data that exists. And while the answers to these particular questions probably won’t lead to any Nobel prizes, enabling scientists to query arcane corners of the literature quickly and easily would allow for more workflows like the one demonstrated by Sharpless, and thus be beneficial for scientific innovation.

What prevents this from happening today? Developing a suitable database of chemical data would entail scraping through tens of thousands of publications and patents and extracting the data contained therein, both structured (reaction conditions, yields, selectivities, etc.) and unstructured (the actual text of the papers). While efforts in this direction have been made (e.g. the Open Reaction Database for structured data), doing so for all the old journals that Sharpless and others know so well would be a sizable undertaking — but not insurmountable, contingent on IP issues being worked out.

What's next?

Eric taking over again.

In my eyes, the work of both Swanson and Mullainathan & Ludwig serve as massively important proofs of concept that computationally-augmented hypothesis generation is much more than sci-fi-inspired wishful thinking.

One ideal place to start might be a scientific philanthropy scraping together the resources to put together a minimum viable dataset in an area ripe for experimentation with the idea — say, organic chemistry — and getting some researchers to see just how much they can do with the basic tool.

If it works, you can scale the hell out of it. If it doesn't, we can iterate, learn, and hopefully try again. Building up resources and know-how in the area of computer-augmented hypothesis generation seems to be an area of significant promise that is obviously worth pursuing.

Thanks for reading:)

Share

Subscribe now

Some think it spells doom!

In my eyes, even if something like that happens, OpenAI and DeepMind are still the odds-on favorites to make the next big AI breakthroughs as well. Check out the piece to learn why! It draws on recent innovations in jiu-jitsu as well as the history of mid-20th-century industrial R&D labs.

Enjoy!

ChinaTalk

OpenAI: How Do They Do It? Lessons from Jiu-Jitsu Innovation and for S&T Policy

Satya Nadella didn’t want to hear it. Last December, Peter Lee, who oversees Microsoft’s sprawling research efforts, was briefing Nadella, Microsoft’s CEO, and his deputies about a series of tests Microsoft had conducted of GPT-4, the then-unreleased new artificial intelligence large-language model built by OpenAI. Lee told Nadella Microsoft’s researchers were blown away by the model’s ability to understand conversational language and generate humanlike answers, and they believed it showed sparks of artificial general intelligence—capabilities on par with those of a human mind…

Read more

3 years ago · 6 likes · Nicholas Welch and Jordan Schneider

For those who do not run research operations or are not really interested in diving further into the details of how Bell Labs ran, feel free to skip this bonus post.

To get a better idea of how Bell Labs’ rules of thumb came to life as (possibly) the greatest applied technology lab in human history, there is no better place to start than Mervin Kelly’s speech to the Royal Society: ‘The Bell Telephone Laboratories — an example of an institute of creative technology.’

Mervin Kelly, who in retrospect could be considered Bell’s most impressive manager in terms of big-picture thinking, gave this fantastic lecture in 1950. It is, of all the Bell documentation I’ve seen, the best single document at breaking down Bell’s system of managing its projects and engineers.

He concluded his opening statement, emphasizing that the secret sauce of Bell Labs and places like it was everything connecting the two endpoints of manufacture and basic research:

There has been so much emphasis on industrial research and mass-production methods in my country, that even our well-informed public is not sufficiently aware of the necessary and most important chain of events that lies between the initial step of basic research and the terminal operation of manufacture. In order to stress the continuity of procedures from research to engineering of product into manufacture and to emphasize their real unity, I speak of them as the single entity “organized creative technology.”

Then, Kelly highlighted the three different groups of employees that made up the living, breathing, prolific bundle of humanity that was Bell Labs — the driving force behind his nation’s telephone system.

The headcount of the organization was about 5,700. Of those, 2,200 were scientists and engineers and the rest were support staff of one kind or another. Kelly saw the breakdown of the headcount at his “mature institute of creative technology” as falling into three “interlaced” categories:

• Research and Fundamental Development

• Systems Engineering

• Specific Systems and Facilities Development.

We’ll first dive into how the research department worked, then the key role of facilities development, and, lastly, how systems engineering tied the whole thing together.

Research and Fundamental Development

Most of the Bell Labs researchers whose names have become lore in Silicon Valley — such as Shannon and Shockley — resided in this department. This portion of the Labs, which made up about 30% of Labs employees at the time of Kelly’s speech, was meant to be an area of Labs that provided “the coupling between the ever-advancing forefront of pure science and the forward march of our communications technology.”

In line with two two-headed purpose of this department 1) keeping abreast of and expanding on research frontiers and 2) pushing the communications network forward in a practical sense, there were separate “research” and “basic technology” ends of the department — each made up around half of the department.

Kelly noted in his speech that those in the research end of the department dove into topics like:

Solid state physics, including magnetism, piezo-electricity, dielectrics, and semiconductors; physical electronics; electron dynamics; acoustics; electro-magnetics; mathematics; organic and physical chemistry of synthetic plastics and rubber; corrosion chemistry; physical metallurgy; and fundamental mechanics are a typical but not comprehensive list of subjects.

And it was important, in Mervin Kelly’s eyes, that those on the research end of the department confine their efforts to the area of research as much as possible. He wanted them to work on useful problems and help out the development engineers, manufacturing staff, and systems engineers, but that help was meant to come from the point of view of a “research man.” That was their comparative advantage within the large organization. If maintaining this comparative advantage came at the expense of some lost engineering know-how from the researchers, so be it.

To work in research, one only needed to have the skillset of a good chemistry, math, physics, or electrical engineering researcher from a university. The Bell rules of thumb would ensure that their brains were folded into the operation in a practical way — as happened with John Pierce in the main article.

Employees who worked in fundamental development, the other end of the department, often had a slightly different educational background — or at least a different mindset.

When selecting individuals for this end of the department, Kelly said Bell selected “those having technologic and engineering aptitudes and interests.” Specifically, this was often done by:

Recruitment from among the most promising of the graduate students of our schools of applied science such as Massachusetts and California Institutes of Technology.

For those who have not read my series on the MIT of the early 1900s, that meant that these were graduate students of a sort of hybrid trade school + engineering research organization — rather than the MIT of today. Bill Jakes — the engineer from the main article who researched pieces of communication equipment, driving around in vans and trailers — is an example of a fundamental development researcher.

These individuals were ideal connectors of academic knowledge and manufacturable products because that was almost exactly what MIT, back then, was set up to do. Of course, you didn’t need to only have gone to MIT, but the MIT mindset was what they were looking for. And, in an era where many individuals found their way into science and engineering through industrial or practical interests, this was not a shallow talent pool.

Employees on this end of the department were managed differently from the more pure research group. The primary difference was there were proper bosses on this end of the department. When it came time to do development research, it began to make sense to have an organized team with a real team leader in charge. On the pure research end of the department, individuals still often folded into teams and research groups to work on various things, but it was generally the Bell rules of thumb and nudges from their informal supervisor that shaped their project selection.

Kelly said this of how the two ends of the department were, in practice, interlaced:

To make possible and to encourage this concentration of attention of the men of science to research, we provide, through organization and stimulated association, an intimate tie between research and fundamental development, the next step in the chain of events from research to manufacture and use. In this way the programmes of research are taken over at a well-considered point by fundamental development, where they are extended and enlarged upon to supply the body of basic technology for the specific development and design of systems and facilities. While the fundamental development work is done in the best research tradition, it has a large content of the technologic, and economic considerations begin to be a factor in its programmes.

“Fundamental development” is a carefully chosen phrase to describe what the development group was doing. It was not quite as far-ranging research as the research group, but it was still very much research, and work from these groups would still often end up in engineering-adjacent academic journals.

In the world of industrial technology, an idea itself is only worth so much. A new concept often does not become useful until a team of dedicated applied scientists and engineers have familiarized themself with the implications of the new area, developed methods for making practical use of it, learned how to overcome unfamiliar issues associated with the new idea, etc. Additionally, any impracticality of the crude idea that came out of research had to be worked through by fundamental development — the research team was meant to take into account things like cost and manufacturing capability, but, obviously, MVPs are always imperfect.

John Pierce once said in an interview, asserting the massive importance of development at Bell:

You see, out of fourteen people in the Bell Laboratories…only one is in the Research Department, and that’s because pursuing an idea takes, I presume, fourteen times as much effort as having it.

For this reason, the fundamental development and development budget for a project tended to dwarf the cost of the research for it. How basic this department’s research was made Bell quite unique, but it was development work that made sure ideas frequently bore fruit for actual Bell products, manufacturing facilities, and maintenance work. The impact of Labs’ development work on Bell’s fantastic outcomes is largely underrated.

Specific Systems and Facilities Development

60% of Bell Labs staff were the (mostly) engineers — electrical, mechanical, chemical, and metallurgical — from technical schools and universities that made up the Specific Systems and Facilities Development portion of Labs. This group was charged with testing and proving out a functional performance of the ideas originating in the Research and Fundamental Development portion of the org. This portion of Bell Labs ensured that anything that came out of Research and Fundamental Development was made to work at scale in Bell’s vast system — a different beast entirely.

Kelly noted that the projects of this group were handled in three stages.

In the first stage:

A laboratory model is evolved that meets the functional requirements of the systems engineering study.

In the second stage:

The design for manufacture and use is created. While retaining the functional performance of the laboratory model, it also meets the requirements of manufacture at lowest cost consistent with providing the specified service at lowest complete service cost…In achieving these economic objectives in the design, the development group is the focal point of a closely integrated trifurcated team with manufacturing engineering of Western Electric [Bell Telephone’s manufacturing arm] as interpreters of, and important contributors to, design form for lowest manufacturing cost and with systems engineering as interpreters of operating requirements that are essential to lowest complete service cost.

This step is finalized with a completed pre-production model. A small number of these models are placed under the observation of systems engineers, specific development groups, and the engineering groups of the Bell operating company relevant to the project.

In the third stage:

The findings of the service tests and Western Electric’s experience in producing the models are used as background for our final freezing of design and Western Electric’s planning and tooling for quantity production. Design drawings and specifications are then prepared for manufacture, and engineering practices for the technical operations involved in supplying telephone service.

Special facilities and development was very “interlaced” with not just fundamental development, but research as well. On this point, Kelly said the following in his speech:

While there are men with the necessary mathematical training and facility for the analytical work that normally is encountered in each of the areas of work, the mathematicians of our research area devote at least one-fourth of their time to consultation and to aiding the analysts of different areas of development in the solution of their problems. Such co-operation is informal and initiated by the men of development requiring the help.

Through the years our research and development leaders have developed patterns of informal co-operation and the habit of going to the expert, whether he be a mathematician, a metallurgist, an organic chemist, an electromagnetic propagation physicist, or an electron device specialist. This has made it possible to focus the full-power of our organization on a particular problem. The organization’s capacity for the solution of telecommunication problems is much greater than the sum of the capabilities of the individuals.

As Kelly continues to paint the picture of how vast the application areas of Bell Labs’ work were, it begins to feel quite unrealistic to believe that Bell Labs could collaborate with the rest of Bell Telephone properly via some intangible “culture of sharing and openness.”

It’s not hard to believe that the research and fundamental development teams, the two ends of the first department we explored, could keep up with each other through some culture of openness. After all, when you think about it, what that looked like, in practice, was some former Princeton grad student — in research — talking to some former MIT masters student — in fundamental development — about pieces of a shared problem. These groups, in another life, might share an office in some university even if they were slightly different kinds of researchers.

However, in many cases, serendipity would just not cut it. Take the case of a development engineer. Many development engineers had an absolutely unwieldy number of places and people they needed to be plugged into. Even if one was a university-trained engineer, one still might not fully understand the work of many of the researchers.

Additionally, even if the engineers knew a lot about particular aspects of Bell’s manufacturing and installation that were close to their areas of knowledge, Bell had an absolutely vast operation. Any piece of this vast operation could contain an unsettling amount of operational minutiae that the engineers might need to know for their work.

It would be completely unreasonable to expect the best problems to find the best people without a little more planning.

Enter systems engineers. Systems engineers did not technically do anything that you wouldn’t hope would happen with a culture of collaboration, but they were the people that made finding the “one or two in a thousand” problems relevant to Bell and converting them into successful business applications “essentially automatic.”

In Kelly’s description of this Specific Systems and Facilities Development group, comprising 60% of his organization, the guiding hand of the systems engineers is everywhere.1

Systems Engineers

The role of ensuring Bell ran “better, cheaper, or both” was the full-time responsibility of Systems Engineering.

Systems engineers helped coordinate and plan development work — as we saw with the mobile phone project in the main article. Additionally, they kept up with all work going on in Research and Fundamental Development. With this knowledge, they would both find ways to turn research ideas into profitable projects as well as bring the researchers great problems that could use their skills.

Kelly described the background of his systems engineers as follows:

[Systems Engineering] staff members must supply a proper blending of competence and background in each of the three areas that it contacts: research and fundamental development, specific systems and facilities development, and operations. It is, therefore, largely made up of men drawn from these areas who have exhibited unusual talents in analysis and the objectivity so essential to their appraisal responsibility

Or, as Jon Gertner put it, systems engineering was:

A discipline started by the Labs, where engineers kept one eye on the reservoir of new knowledge and another on the existing phone system and analyzed how to integrate the two. In other words, the systems engineers considered whether new applications were possible, plausible, necessary, and economical.

Systems engineers were (largely) trained as engineers and researchers, but also had a brain for the whole operational board when it came to scientific business and development. These systems engineers were often the ones who shepherded problems to the researchers and fundamental development folks at Bell Labs. Sometimes, today at least, researchers get a little territorial or defensive about others thinking they could come up with better research ideas than them. But, in a setting like Bell Labs, that was not much of an issue given the applied nature of the operation.

The basic researchers were hungry to be as useful as possible.

It’s not embarrassing to not come up with as good of an idea as a systems engineer. I’d imagine the thinking was something like, “Even if you are a great basic researcher who can come up with research ideas that have applications, this systems engineer is another former researcher with a Ph.D. in the same subject as you. But this guy now spends 100% of his time studying the phone system and its problems as well as the costs of various things. Why would he not almost always come up with an equally good or better idea? That’s his job.”

You’re still probably a better researcher than them. That’s just an efficient division of labor. If you’re both doing your job, he’s probably the better “operational idea guy” and you’re the better “scientific idea guy.” And, if you’re both doing a good job, you’re probably going back and forth with each other frequently.

As far as I can tell, these systems engineers were seen as good guys in the research portion of the labs. Not only did they bring in white-hot problems from the development and implementation end of the operation, but they were also tasked with finding ways to put the findings of the researchers to use for Bell. I believe that was the initial reason the position was created. Kelly writes:

One of the principal responsibilities of systems engineering is technical planning and control….The determination of the most effective use of new knowledge in the interest of the telephone user is the guiding principle of the [systems engineering] planning studies. The most effective use may be the creation of new services, the improvement of the quality of existing services, the lowering of their cost, or some combination of these three. As the technology of communication has broadened and become more complex, the choice of the technical paths to be pursued in the instrumentation of the new technology has become increasingly difficult. It is this situation that had led to the evolution of the systems engineering function as a mechanism of guidance and control.

Systems engineering has intimate knowledge of the telephone plant and its operation and maintains close contact with the engineers of the operating organizations. The teamwork of operating engineers and our systems engineers makes available to the laboratories in a most effective way the knowledge of the telephone system’s needs and the opportunities for economy and improvement.

Bell had tens of thousands of workmen involved in all variety of factory work — laying wires, installing phone lines, servicing homes, etc. How were they supposed to know what problems in their particular jobs could be solved by the researchers? How were the researchers supposed to know those problems were happening? When you think about it, in an applied research lab that works on a number of problems, it becomes almost weird not to have some kind of systems engineer around to help things run more smoothly.

It’s easy to see how the ROI of an applied research operation can obviously go up if researchers had available to them — and were excited to work on — problems that the systems engineering team had already brought to them as a veritable gold mine with the constraints known from the beginning — because systems engineering had looked into it.

(And I do mean gold mine. Bell tended to develop projects whose cost savings was something like 20x-30X the cost of development. Constraints like this ROI constraint as well as Bell’s longevity requirements — they hoped for new parts to be able to operate in the system for 20+ years — were all taken into account by systems engineers.)

Systems engineers couldn’t force the basic researchers to work on anything, but all things equal, most people would like their research to be used.

In Conclusion

Please reach out to me on Twitter if you’d like to ask any questions or implement the ideas in your own operation!

Subscribe now

Share

Thanks for reading the bonus post:)

I’ve finally updated the “About” section of this Substack. For months and months, I put it off because I was so busy working a job and writing at night. Then, I began writing full-time and concerned myself wholly with making sure I kept up producing novel pieces as a “professional.”

Today, I finally got to it. And I realized this was the perfect opportunity to do something I’ve never done: a subscriber push. Most of you have found the Substack through word-of-mouth, which has been great! I have no Twitter following to speak of (I think only like 200 of you follow me on there at most), but you found your way here anyway!

The only thing that I’ve found helps this Substack grow is you all liking what I write and sharing it. I’m trying to continue to do my part as we speak and working on a great piece (with Stuart Buck) on the golden-era of molecular bio.

In the meantime, it would be great if you could help me out! I’ve pasted my updated “About” section below! If there’s anyone you know who you’ve been meaning to send my Substack to/that you think would get a kick out of it, today is as good a day as any to send it over to them!

Share

What is Freaktakes about?

Helping builders in the metascience/progress studies space discover exciting, new ways forward.

For years, my hobby was reading about the lives of a great era of scientists, innovators, and builders (from about 1880-1965). Eventually, since I have always done economics-adjacent work, I began to immerse in the economics of innovation literature to understand some of the dynamics I was reading about. Most of the data in the literature is from around 1970 and onward. Some of it really helped explain what I was seeing in the older work; some of it clearly demonstrated that the era or two I was reading about was MUCH different than the one represented in most of the data.

I started writing this newsletter to better explore that dynamic. Because, if I was starting a new science org, I’d most heavily utilize the economics of innovation literature, but I’d also want to understand the extreme limitations of it. The limitations are mainly a function of the later 1900s ushering in the more modern (and bureaucratic) era of the scientific ecosystem that we know today. And that system has proven to help us do a lot of things well — for example, science is much more lavishly funded now, but it also changed a lot of things. The types of visions many new science funders describe are scrappy, moonshot-driven, decentralized, and weird. And if you’re looking for detailed examples of that, the mid-1900s is a much richer source of evidence than the back half of the 20th century.

But both sources of evidence should be heavily drawn upon. And, when you do that, you can uncover enlightening ways forward for new science initiatives.

So please subscribe to learn all about what the 1880-1965-ish history of innovation can teach us about how to build an exciting new science organization in the present day. (I think) This Substack is the perfect accompaniment to Matt Clancy’s New Things Under the Sun which does a world-class job of walking readers through what we do and don’t know about innovation using the economics of innovation literature.

Enjoy:)

Thanks so much!

I started the Substack with four piece ideas written on a white board and some free time. I’d start there and see where it went. Less than a year in, I’ve written almost twenty pieces, am working on several more, and have a list of at least a dozen more to start “at some point.”

And more importantly, I love it. I’d do this for free…I did do it for free for a while. FreakTakes isn’t going anywhere!

Thanks so much for your time and curiosity. It’s been life-changing for me. And, more importantly, it gives me hope for the future of the scientific ecosystem. You come here because you are curious about new ways forward and to learn more about what we did well in the past.

With genuine curiosity and the desire to constantly improve, so many things are possible. It’s one of those special things that makes humanity great. It’s what draws people to progress studies in the first place. This Substack tries to remind its readers of how far this approach can take us in every piece. But, every time a new reader tunes in, the reader reminds the author of just how many people are motivated to continue this tradition. (I still receive an email every single time one of you subscribes. I’m not sure I’ll ever turn that notification off. It’s an injection of motivation for me every time it pops up.)

Looking forward to continuing to serve your curiosity. Please enjoy:)

Share

And if you’re receiving this email because your friend just recommended FreakTakes to you now, the following are some of the “best hits” of the Substack.

• When do ideas get easier to find?

• A Progress Studies History of Early MIT — Part 1: Training the engineers who built the country

• Bombs, Brains, and Science

• Irving Langmuir, the General Electric Research Laboratory, and when applications lead to theory

Weaver’s wide-ranging accounts of all parts of America’s scientific ecosystem, from World War I up until around 1970, provide fruitful lessons for pretty much all sub-areas of progress studies. This short will focus on three separate subjects which Weaver briefly addressed that are remarkably pertinent today: the “burden of knowledge,” scientific bureaucracy in government departments, and why theorists should never stray far from applications.

Many of you may know Weaver — if you know him at all — as the co-author of The Mathematical Theory of Communication with Claude Shannon, the more public-facing book written to popularize Shannon’s then-not-so-famous paper in the Bell System Technical Journal which marked the birth of information theory.

But, as it turns out, helping popularize Shannon’s theory was Weaver’s third most significant contribution to science AT BEST. He was also instrumental in helping bring into existence early research pivotal to the field of molecular biology and in setting up the research institutes responsible for the findings that led to miracle rice — massively increasing the efficiency of rice growth in the then-developing world.

Those two efforts will be the subject of the following post. So make sure you subscribe if you don’t want to miss that post. Having been a physics researcher, an early member of the CalTech faculty when it was just getting started, in charge of the Rockefeller Foundation’s natural sciences grants, and a right-hand man to Alfred Sloan at the Sloan Foundation, Weaver witnessed and was around for a bit of everything in the mid-20th century boom in science and scientific philanthropy. And, as a kicker, he was freakishly smart/thoughtful.

The three sections of this post cover stories related to:

• How long it took for one of America’s most flexible and fast-moving scientific departments, the National Research Council, to grow so large and bureaucratic that it couldn’t (effectively) do anything new.

• The account of yet another top scientist who lived and worked through the golden era of physics blaming conference sizes, not the growing size of the literature, for it being hard to keep abreast of new findings in his field and related fields.

• And, as in the Langmuir piece, another theoretical researcher (in this case a mathematician) becoming most useful when working in close proximity with applied researchers.

  

Applied work: a theorist’s annoying best friend who makes good points

The following is an example Weaver recounts of some World War I work in which yet another theorist heavily benefited from proximity to applications. Unlike in the Langmuir piece, where we saw new theory result from proximity to the lab environment, in this case, the outcome was that was able to make his not-so-useful work useful.

The story took place at a then very young CalTech. Weaver writes:

There was considerable scientific activity at Throop [later CalTech], which, in the early months of our national participation in World War I was oriented toward defense, particularly in the aeronautical area. The submarine menace being an obvious and indeed terrifying one, there also were various studies related to high-frequency sound production, transmission, and detection. I read up on the piezoelectric effect, whereby a block of crystal can be electrically driven to oscillate at high frequencies. Harry Bateman, the English mathematician previously mentioned, was investigating, with his powerful but extremely theoretical methods, the possible advantage of electrically driving a hollow spherical shell of crystal, so that it could be used as a source of underwater sound waves. One day I asked him how serious would be the effect of the discontinuity where the two halves of the hollow spherical shell were cemented to form a complete hollow sphere, It turned out that Harry had not thought of that. He just calmly assumed that some practical chap could put a concentric spherical hole inside a solid crystal sphere without tampering in any way with its homogeneity! Bateman had been the First Wrangler at Trinity College, Cambridge, was a profound scholar with a vast store of knowledge about partial differential equations, and a sweet and gentle person. But he was not precisely practical.

While, technically, Weaver was studying a math-heavy form of electromagnetic field theory as a professor, he was a formally accredited civil engineer, having majored in civil engineering at Wisconsin-Madison (and simply did a lot of extra physics and math on the side). He even took time off of his research one summer to do some work on geophysical prospecting with an engineering firm. He was exactly the kind of application-minded researcher that could help a true theoretician achieve an elevated level of translational impact.

Just as in the GE Lab for the Langmuir piece, this theoretician was:

1. Allowed to use his own methods and lead his own research BUT had a specific application area of use chosen for him.

2. And this theorist made use of his proximity to applied men to come up with newer or more practicable lines of their research.

Many theoreticians just shouldn’t be allowed to work off to the side as high a percentage of the time as they currently do, existing in professional silos for the vast majority of their research time.

When I tell these stories, such as in the Langmuir piece, I take care to make sure that they are all more or less representative of this boom-era of applied and fundamental research — and not some edge case. The misapplication of a generation of theorists is something I feel very strongly about, so I hope to continue to share these anecdotes with you in the shorts both because they are interesting and because they will add data points to your mental dataset of how exactly the life of a theorist is benefited from exposure to applied work — even if it’s not the allocation of time they would choose for themselves.

And this is not me over-fitting my own opinions onto history. This opinion on the relationship between applied and fundamental work was echoed by many at the time as well, including Weaver himself. At the end of World War II, a war in which Weaver was put in charge of an entire division of applied mathematicians, Weaver reflected:

The experience of the war had demonstrated the practical national, as well as the personal intellectual, value of work in this field, and had shown what history had in fact proved over the centuries — that mathematics, including its most “pure” and abstract portions, is healthiest and strongest during invigorating contact with a wide range of problems in the real world of experience.

The researchers working in Weaver’s applied math department were not all applied mathematicians regularly. The department’s ranks included many technical researchers with mathematics chops who did not normally do this. There were physicists, actual applied mathematicians, more traditional mathematicians like Oswald Veblen, and even folks like a then-Ph.D. student named Milton Friedman.

They all put in their time. There they learned, firsthand, what kind of fundamental work would have a path to usefulness and what kind was likely only interesting to people in their academic circles.

How fast a government scientific bureaucracy can grow (wildly fast)

Most people will not argue that government scientific organizations like the NSF were productive, efficient, and generally good at achieving their knowledge-creation goals with minimal bureaucracy in their early years. But, as researchers have spent entire careers studying, these organizations can rapidly become (relatively) ineffective both in their sluggish pace and their inability to properly adjust goals/processes that work against the org’s mission to create useful knowledge.

Many times, in a large central government, if you want something moderately new done, you need to erect an entirely new department to do it. Feynman mentioned an example of this in his oral history. As he was trying to explain why he had no desire to join the National Academy of Sciences.

No. I’d never heard of the National Academy of Sciences. I received this thing, and I didn’t know what it was. Somebody told me Epstein would know, because he was the only guy around at the time. Bacher was out or something. I said, “I don’t want to join it, Professor, because as far as I know, they don’t do a damned thing.” He said, “But they have this National Academy of Sciences thing that they publish. They have meetings.” I said, “Yeah, but I don’t read the National Academy whatever it is.” I don’t even remember what the name of the publication is now. It’s a publication, but the articles in physics are not impressive there. I never had to refer to it. Never knew it was there, and I never knew anything that they did. “Well, it’s an honorary society.” I had already made, when I was a kid in high school, a principle, see. Like a nut, like children do, you make ideal principles, and then later you make yourself miserable in life by having to change the principle. See, I had become a member of what was called the Arista, which was an honorary thing for the students, and the only thing we did in the Arista was to select other students who might become members of this thing. So it was a mutual patting on the back society, and I looked at what we were doing, and I thought, “That’s not right. All we do, we get into this thing, and we give the honor to the next guy, the great honor.

I’ll tell you more that I don’t like — the history of it. It was invented by, I believe, Lincoln, in the Civil War, to help scientifically, to give advice, to assist the Union in winning the Civil War. It had therefore a real purpose, hm? Ok. By the time the next war came along — I don’t know which; say the First World War — it’s ineffective, so they can’t use it. Instead of that, they appoint another thing whose name I don’t — oh, the National Research Council, hm? — in order to give advice to help win the First World War. We have these two things now — hm? Ok? And they don’t buy, it doesn’t buy. It has its building in Washington. It’s paid for by the taxpayers. It doesn’t buy, because it can’t do its job, and they knew all this. Second World War comes around — neither of the two organizations works now. They need another thing — OSRD, or something like that, hm? And they put that on top of the thing. I don’t remember all how it goes, but this stuff just doesn’t make sense. If it doesn’t work, forget it. And if it works, use it. But not just pile one on top of the other, because it’ll just get like barnacles. So they have now four or five organizations. They all exist, from back in history.

Weaver confirmed this state of affairs in a slightly more polite manner saying that, by World War II:

History then repeated itself. By 1940 the National Research Council had become so deeply engaged in the programs it had developed over the preceding twenty-year period that clearly a new organization, flexible and uncommitted, must again be set up.

Even at that time, with a smaller and, I believe, relatively speedier federal government, the length between one World War and the next was enough for one war research department to become stale and another needed to be created that could materially contribute to the war effort.

But, that begs the question: does this mean the decline in the effectiveness of the department was actually that rapid? You might be thinking, “It is a federal department, after all. It might’ve been shockingly slow to begin with and it getting even 15% more bureaucratic could break it.”

But the answer to that is actually a little scary. The National Research Council, the World War I department mentioned above, was probably way more startup-y than you’d think.

Weaver painted a very clear picture of how surprisingly quickly and effectively projects in the National Research Council could be done. He writes:

Toward the end of the spring semester at Throop in 1918 I was inducted into service at the request of Charles E. Mendenhall, Chairman of the Physics Department at the University of Wisconsin. Mendenhall was then a major in an organization closely related to the National Research Council, set up in World War I (as was Vannevar Bush’s Office of Scientific Research and Development in World War II) to bring science to the service of the armed forces. I went in as a private, but after some months was made a second lieutenant.

I worked chiefly at the Bureau of Standards in Washington, in a group which — how ridiculous and ineffective this seems in retrospect — consisted of myself and an expert mechanic. There were few aircraft flight instruments available in those days, and I was primarily with the design and test of turn and bank indicators. It was very easy to design and construct a gyroscopic device that would tell the pilots when they were flying a straight line. But bank indicators, especially ones which would continue to operate with useful accuracy during the acrobatics of aerial dogfights, presented difficulties we never overcame. I did get to do some very exciting acrobatic flying at Langley Field, where returned fighter pilots would subject the instruments (and the testing scientist!) to the latest flying tricks. But this was not the sort of participation in the war I craved, and I was relieved when I was discharged.

Even in friendlier and comparatively less sclerotic-inducing conditions, a department as scrappy as this grew so inflexible that a new department needed to be created in twenty years to meet the needs of a new war. This should serve as a reminder of just how careful we need to be in the design of fledgling orgs like ARPA-H. A startup-y culture in the early years is great, but designing systems that allow an org the needed flexibility to adapt with the times and somehow stay useful (against the odds) is another thing entirely.

I don’t have any fleshed-out ideas I’m confident enough in (yet) to include in this piece — which is why I’ve put it in a short and not a longer-form piece. BUT I suspect that the answer will at least partially be a form of engineered decentralization and empowered individual decision-making that you see in something like ARPA-E and DARPA project selection and management. This kind of individual empowerment/agency is, in practice, MUCH more lacking in processes like NSF and NIH grant dispensation — in terms of how risk-averse and sheepish they are in their overall decision-making in comparison.

Michael Kearney and Anna Goldstein have a fantastic paper in which they looked at internal ARPA-H project selection data and found that, with the ARPA-H setup, projects that generated disagreement amongst experts were particularly likely to be selected. The following is an excerpt from the abstract of their paper:

Using internal project selection data from the Advanced Research Projects Agency-Energy (ARPA-E), we describe how a portfolio of projects selected by individual discretion differs from a portfolio of projects selected by traditional peer review. We show that ARPA-E program directors prefer to fund proposals with greater disagreement among experts, especially if at least one reviewer thinks highly of the proposal. This preference leads ARPA-E to fund more uncertain and creative research ideas, which supports the agency’s mission of pursuing novel ideas for transformational energy technology.

That is prime-time proof that leaving room for individual personalities and discretion is both possible in a government scientific organization and can lead to impressive results.

There is hope yet.

The burden of knowledge: a social problem or a knowledge problem?

Lastly, I’ll leave you with another interesting data point that is a follow-on to one of the Feynman shorts. In Weaver’s book, once again, we hear a 1970-ish account of a researcher who lived through and was active during the golden era of physics complaining about growing conference sizes, not the growing number of publications, making it more difficult to keep up with related areas of the scientific literature. Not only were conferences growing in size, but, in response, they also separated various disciplines into their own conferences to make conference logistics more manageable.

This is very interesting because, in much of the economics of innovation discourse on the burden of knowledge problem, researchers often assume that it is the sheer size of the literature causing the problem. But both Weaver’s and Feynman’s accounts seem to more directly blame the highly statistically correlated, but fundamentally different, change in conference dynamics and logistics that came with more and more researchers flooding into the community.

Weaver wrote of the early AAAS, having been a member since its inception:

This vast organization, with the largest membership of any general scientific society in the world, embraces all fields of pure and applied science. In the earlier days each branch of science conducted intensive sessions at which its own specialized research reports were given, and there was also some mild attempt to organize interdisciplinary sessions of general interest. Then the meetings got so large that they almost collapsed under their own weight. One group after another, first the chemists, then the physicists, the mathematicians, the biologists found it necessary to hold other and separate meetings. Attendance fell and the function of serving as a communication center between the various branches of science became less effective.

Now, I can’t go back in time and ask Weaver or Feynman, in their eyes, what percentage of the problem was a social problem and what percentage was the sheer size of the literature. But both of them mentioning one and not the other in their accounts should not be disregarded. And there is something poetic about the situation if this is a social problem. Weaver, and many other scientists of the day, seemed to acknowledge that science was often about people as much as ideas. And, when framed in this way, the burden of knowledge problem seems to become a lot more tractable as well.

Let’s think of putting on a good conference as like throwing a party. It seems obvious that growing the size of a party from 25 to 250 is not at all an obvious improvement in most cases. Sometimes, maybe yes. But usually, no. And, if the last party was too big, we should probably find a way to have smaller parties — even if more parties need to be thrown to account for everyone who wants to party. What groups like the AAAS, and many others, decided to do was, essentially, to throw separate parties of the 50 people who were the most similar to each other in terms of what they worked on. And, while you can do that, it’s kind of a ridiculous thing to do if you’re trying to foster cross-discipline idea sharing. Not to mention, it’s probably a much less good party than the wide-ranging one from before.

At a time when publications had roughly 10x’d, both Feynman and Weaver chose to talk about the masses of people flooding their conferences (and then conferences being held separately) rather than papers flooding their journals.

If the conferences cared about unique idea generation and facilitating new interdisciplinary work, they’d probably have scaled them in a very different way. If the AAAS wanted to double its membership, it could’ve held two separate interdisciplinary meetings where members were randomly assigned to one or another. Or, if it was afraid of its anomalously productive superstars getting separated from each other, it could’ve held interdisciplinary A-team and B-team meetings. Any number of things, really, as long as the goal was idea creation and the generation of new scientific branches.

But giving all of the fields their own meeting was a truly horrendous idea that didn’t take into account (at all) the production function of America’s most accomplished scientists. There were a non-negligible number of scientists, like Feynman, who kept abreast of related fields not by reading the literature, but simply by asking acquaintances at these conferences/attending talks as these (relatively) intimate conferences about the big ideas and developments of related fields. These conferences, at the time of Weaver’s writing and Feynman’s oral history, were really losing their souls.

More frank individuals, like James Watson, acknowledged that much of conferences as his field was growing was a handful of first-rate minds who consistently were generating an inordinate amount of the novel work talking to each other and learning with a lot of non-main characters taking up space who tended to contribute work that Feynman would call “addendums” to the important work in the literature.

If one wouldn’t want to take Watson’s word for it since most people think he’s an arrogant asshole, I’d get it. But even someone characteristically gracious like Feynman has hinted at this dynamic in his interviews. If you believe that kind of thing, an A-team and a B-team (etc.) AAAS conference is probably what you’d want. And if you don’t, something like two (or way more) randomly assigned conferences is probably up your alley.

In one of the coming long-form posts, I hope to further dive into the specifics of why certain fields, such as math and physics, began to diverge according to individuals that were around to watch it happen. Many researchers, such as Freeman Dyson, also, do not point to the size of the math and physics literature as the cause of those fields’ divergence. In general, being a researcher who spends a lot of time studying the history of this period, I am skeptical of arguments that make the size of the research literature out to be by far the most significant driver of the “burden of knowledge” in the mid-to-late 1900s.

While I am currently working on some pieces that dive into this hypothesis further, for now, I will leave you with Feynman and Weaver’s accounts and allow you to do with them what you will.

Until next time:)

Subscribe now

Share

If you liked this post you might also love:

• When do ideas get easier to find?

• Irving Langmuir, the General Electric Research Laboratory, and when applications lead to theory

• Bombs, Brains, and Science

Hoel is a great thinker who integrates fields seamlessly, and that’s part of the reason he chose to leave academia. Academia, embarrassingly, has little place for this kind of thing anymore. When Hoel was listing reasons why he was leaving in his post, Goodbye academia, hello Substack — which I encourage everyone to read — he wrote:

I work in interdisciplinary areas where there’s not much grant money—sometimes none at all. Second, being a successful professor nowadays means not just crafting your research so it will receive big governmental grants, but also being involved with the student body, doing extracurricular activities, volunteering, taking on a bunch of busywork like organizing special editions of journals, citation-maxing, paper-maxing, not to mention sitting on the right committees, advancing the right political causes, etc. And I just. . . don’t do any of that stuff. Not only that, but I have this pesky writing habit to contend with. A tenure committee will never say “Oh, you wrote a novel and a bunch of popular essays, wow, that’s a huge plus for our Biology department.” To them, this looks like the behavior of a maniac. Everything I write here is bad for my academic career. Every time a book manuscript of mine is delivered to a publisher, it’s bad for my academic career. I’ve never once had anyone in any administrative, hiring, or grant-giving capacity show anything but hesitation about these things.

Hoel feels that little of what he does, whether it’s writing his fascinating Substacks or his novel-ing career, registers in the increasingly formalized scales of academia. In fact, it makes him taken less seriously.

Do some of his Substacks more explore some new ideas and theories at a high (and entertaining) level using less rigorous methods and more cross-disciplinary thinking than is used in modern academia? Yeah. Do I think that should make people shudder in horror and go, “Oh the horror! Does this professor not know there are pipettes to be pipetted and assays to be screened? Does he not know a serious mind should be concerned with that and only that?! He must not be serious about true research.” No.

(Yes I know Hoel probably doesn’t actually do any of that kind of lab science. But it was funnier that way.)

Regardless, even if Hoel’s colleagues and superiors are not repulsed by his daring grab at a curiosity-driven career in the academy, they likely do not support it so much that they would open their minds beyond the current bureaucratic standards that constitute the “right metrics and track record” to grant him the tenure at a top university — that most in the progress community would likely say he deserves due to his wholistic contributions to several related areas of thinking.

Before, there were birds and frogs. Now there are also squirrels.

I have supported Hoel for a while. He has been INTERESTING. And that has been in spite of the incentives. He writes:

I’ve come to believe I can do more original and meaningful intellectual work outside of academia. For, to be honest, when I look back at my career and the things I’m most proud of, I did the majority in spite of the strictures of academia, not because of them. I did it in time squirreled away from bosses and administrators. And I’m tired of being a squirrel.

I believe this quote will ring true with many readers of this Substack. So many curious researchers that I’ve met have felt the need to hide what they were doing — work that they were interested in, was novel, and relevant to their general area of research — just because it wasn’t solely in their discipline or had an uncomfortably high (for grant administrators) chance of not working.

In many of the more expensive sciences with high material costs, such as the life sciences, many go as far as to commit (on paper) borderline fraud, repurposing money from a low-risk/approved research project to a high-risk, unapproved one just to do some real science. This type of behavior, from here on on this Substack, will be referred to as “squirrel behavior.”

It’s kind of poetic when you think about it. Freeman Dyson, many years ago, spoke about mathematicians as tending to be either birds or frogs. In describing what birds and frogs were, Dyson, a renowned frog, said:

Some mathematicians are birds, others are frogs. Birds fly high in the air and survey broad vistas of mathematics out to the far horizon. They delight in concepts that unify our thinking and bring together diverse problems from different parts of the landscape. Frogs live in the mud below and see only the flowers that grow nearby. They delight in the details of particular objects, and they solve problems one at a time. I happen to be a frog, but many of my best friends are birds. The main theme of my talk tonight is this. Mathematics needs both birds and frogs. Mathematics is rich and beautiful because birds give it broad visions and frogs give it intricate details. Mathematics is both great art and important science, because it combines generality of concepts with depth of structures. It is stupid to claim that birds are better than frogs because they see farther, or that frogs are better than birds because they see deeper. The world of mathematics is both broad and deep, and we need birds and frogs working together to explore it.

Over the years, many have said that they believe something like this birds and frogs distinction applies to their field of scientific research also. Well, the increasingly risk-averse and bureaucratic “strictures of academia” have maybe seen fit to grant us a third designation of high-level, curiosity-driven researcher: the squirrel. Squirrels are expert at bending rules, norms, and grant-funding dollars to pursue novel research that is not accepted by a broken system.

I’ve heard many admiring stories of these individuals talked about in private by professors and grad students. The reverence many researchers have for great squirrels’ ability to get science done working around the rules of the system often has a tone of admiration similar to elder physicists explaining Fermi’s ability to grasp a problem in a nutshell. They don’t include it as an aside about a researcher in many cases. It can be considered something of a special superpower, the thing that makes them special above all others.

Now I know what to call them. Squirrels.

In conclusion

Hoel’s departure is just one data point. But it says a lot that someone left one of the only jobs that could, even if only tenuously, support his three-headed career of researcher, essayist, and novelist. He gave up the salary and the chance at finding a place that might give him a salary and professorial comfort for life just to take a shot at experiencing true intellectual freedom and seeing where it takes him.

So, if you’re ever wondering how stifling modern academia is to a really really curious person — which most of our great scientists were — this is a data point I encourage you to remember.

If you’d like to support Hoel’s journey and become a paid subscriber to his top-tier Substack, it’s right here.

Until next time.

In the coming week or two, I’ll be posting some pieces on the birth of Molecular Biology, how quickly a young and fast-moving government scientific department can become sclerotic, and more!

Subscribe if you want to stay posted:)

Subscribe now

Share

If you liked this post, you’ll probably love John von Neumann: A Strange Kind of Bird and When do ideas get easier to find?

1. How different journals and journal reviews were in the early/mid-1900s

2. The level of open-mindedness and amicable debate that characterized the physics profession at the time

3. And why Feynman felt the once extremely productive conferences were growing less useful later in his career

How scientists in this period utilized journals and approached conferences was remarkably productive and should serve as a source of inspiration for any readers looking to revamp how publishing and conferences work today.

Enjoy:)

Subscribe now

Publishing in the early-20th century

Early on in this Substack, in When do ideas get easier to find?, I briefly alluded to the blistering pace of early/mid-20th century physics and how flexible researchers could be in citing something as scientific evidence. I wrote:

In 1962, Holton cited work by M.M. Kessler which found that 50% of the references cited in research papers published in the Physical Review, the top physics journal at the time, were less than three years old. And only 20% were more than seven years old. Decades on from the paradigm-changing discoveries of relativity and quantum mechanics, physics researchers had still been rapidly creating new branches of research and leapfrogging each other in the race for human progress, constantly citing younger and younger papers and letters. Science was moving so fast that a non-negligible number of citations in the journals were citing personal correspondence and conversations with other physicists as opposed to published papers.

See, at the time, the journals (and the scientists) were under no allusions that it was the journals’ job to be the arbiters of correct and incorrect, the gatekeepers of what was scientific and unscientific. The point of the journals was to be the bulletin board of new science from around the world; they were a way to re-create the culture and knowledge-sharing of a departmental colloquium with scientists from around the world. Getting your paper in the Physical Review was nice because a lot of people read it, but I’ve never seen an account of a top scientist bragging that they got a paper in the Physical Review as an accomplishment in and of itself — and, speaking frankly, they brag about a lot. And when they got told to submit in one journal instead of another, like when Feynman got told by a referee at the Physical Review to maybe submit in the Review of Modern Physics instead, they didn’t seem to think it was a big deal.

This, in my opinion, was academic publishing in its most healthy and useful form. A field like physics:

• Had researchers that more or less agreed on what compelling evidence looked like

• Had quick and easy access to journals to share and consume new ideas

• And its researchers understood that they had to read the papers in the journals and come to their own conclusions on how much they should trust the paper’s findings.

It seems that Feynman, whose memory is a little faulty, received less than a handful of revise and resubmits in his career. When asked him if he’d ever had any difficulties with referees, he replied:

Ever since the first paper, which is the Review of Modern Physics paper on path integrals, in which there was a small objection which I mentioned, there’s never been anything. I mean, I send it in and it gets published, just the way it is.

The reviewers seemed to quickly check the paper for egregious mistakes in math or logic as well as to give some brief commentary about whether they felt anybody had done this work before. And that was mostly it. From time to time, they did miss that a paper was repetitive or contained a mathematical error.

Feynman himself had a very strongly held personal reason that he didn’t review papers: how was he supposed to know if the other guy was wrong or he just didn’t get it? Feynman’s brilliance largely lay in his desire and ability to figure things out for himself. Oftentimes, he found it easier to use his own methods to work his way from an author’s premises to their conclusions than to follow the author’s logic step-by-step. Many times in Feynman’s own work, he knew for a fact that other researchers were not quite understanding what he was saying. And he was positive that he might be prone to this same error in understanding the work of others that seemed a bit off. He says:

Well, I started to try to look at the papers of other people but, you see, I have a funny thing. To me there’s an infinite amount of work involved. I would have to first understand how he’s thinking about it — not just understand the problem, but what he’s thinking about it. Then I’d have to go and see, is it Ok? Hm. Or what is it? I mean, it’s too much work, darn it. It’s like almost research: checking the ideas, seeing if it really works, and so on. It’s like research, and I can’t do somebody else’s research. I’m not built that way. I can’t think his way. I can’t follow and try to go through all these steps. If I want to worry about the problem, I read the paper to get the problem, and then maybe work it out some other way. But it’s too much work. Now, to read and just check steps — I can’t do it. And then, if a paper comes out that’s bad, that’s not very good, I’d feel very uncomfortable to say that there’s something the matter with it, or that it’s not OK, because maybe I’m not understanding. Maybe it is OK; maybe somebody else will see that it’s all right. I think it’s a lot of nonsense. Finally, I think most of the papers are a lot of nonsense and not worth publishing. And so, altogether it’s a miserable business, and I just say I won’t review any papers in order to simplify it because if I start reviewing some and not others, then it sounds like a criticism. There are a number of other things — I have resisted the outside world on this and a number of other things. For example, I never give commentary on whether a man is loyal or not loyal. You know this kind of investigation. And I got everybody off my back on that by just saying I won’t do it. And I never review papers. And one thing I would like not to have to do, but I can’t avoid, is writing recommendations for students. But after all, sometimes nobody else knows them, and they’re trying to get a job. So I have to do that. But I find it very distasteful. I don’t like to judge other people, or their work, at all. I don’t. I don’t want to judge somebody else’s work.

Feynman talked at length on many different occasions in the oral history about how, early in his career, the old guys like Bohr and Dirac just didn’t seem to get it in certain areas. They were guys he idolized since he was an undergrad and admired their contributions to the field, but he thought they weren’t really properly updating their thought processes as the field was developing — and it was developing rapidly. But, unlike today when issues like this come up in fields, he didn’t carry any malice towards them. They told him he was wrong; he told them they were wrong. It was friendly. This is how they debated. They had no power to affect his ability to share his ideas by, purposely or accidentally, influencing journal referees.

Feynman remembers a conference where one of these disagreements happened:

Feynman: Bohr was at this meeting and somewhere, after I’d tried and tried and I talked about trajectories, then I’d swing back — I was forced back all the time to explain. Finally I go back to the idea of an amplitude for each path; that quantum mechanics can be described by the amplitude for each path, and after that Bohr got up and he said, “Already in l9” — something, 1924, ‘25, or something –- “we know that the classical idea of the trajectory in a path is not a legitimate idea in quantum mechanics” and so on. In other words, he’s telling me about the uncertainty principle, you see, and so on. And when I hear this, this was the least discouraging of the criticisms, because it was patently clear that there was no communication, as you like to say. Because he’d tell me that I don’t know the uncertainty principle, and I’m not doing quantum mechanics right. Well, I know I’m doing quantum mechanics right, so there wasn’t any fear or anything. I mean, it was no trouble. It’s just that he didn’t understand at all. And I simply got a feeling of resignation. It’s very simple, I’ll have to publish this and so on, let them read it and study it, because it’s right. I wasn’t unhappy from that, you understand me? From Bohr’s criticism.

Weiner: Was there antagonism in this criticism?

Feynman: No. No, only the usual personalities. I mean, Teller, full of excitement, and Dirac mumbling “Is it unitary?” No, there was no trouble. It wasn’t antagonism. But to tell a guy that he doesn’t know quantum mechanics is to say, you know — It didn’t make me angry; it just made me realize he didn’t know what I was talking about. And it was hopeless to try to explain it further. And I said so. I gave up. I gave up completely, and I simply decided to publish it, because see, I knew it was OK.

The top physicists were (for the most part) friendly and respected each other a lot, but they could also argue with the tone of childhood friends. Here, they couldn’t agree, so the idea should get published for the whole community and they would go from there. That’s just how it worked.

Feynman on the open-mindedness of the physics profession at the time

Feynman generally found the conferences early in his career to be very useful. Just as in the scene described above, there was a lively give and take among the physicists and they felt comfortable and friendly enough with each other to be lighthearted and direct at the same time.

Above, Bohr was not being passive-aggressive or diminutive when he was saying that Feynman had forgotten basic laws of physics. Bohr, in fact, would make a point of setting private meetings with young Feynman at Los Alamos, going on long walks alone with Feynman because he valued his back and forth with the young man so much. This is just how they argued.

Also, in this era, it did not feel absurd to put forward theories that violated or replaced some new law or principle of quantum mechanics. After all, if you’re someone like Feynman or Schwinger and talk to the old heads of the field like Dirac and Bohr, and you know for a fact that you understand the recent developments better than they do, then it doesn’t feel that strange to think your additions to their theories might have some merit.

These conferences were a place where many people — even if not everybody — felt free to discuss new, hole-filled ideas out in the open. Here is an example from a 1956 Conference in Rochester:

I was rooming with a man at the time — Martin Block. So Martin Block said to me when we were going to bed, after the discussion of the experimental situation on this problem — he says, “All you guys worrying all the time about this Tau-Theta puzzle.” Tau-Theta, I guess it was called — there was a Theta meson and a Tau meson, now called the K meson. He said, “You know, from an experimental point of view, it’s very easy. It’s just the same particle. It’s only that your principle of conservation of parity is cockeyed.” He said to me, “What would be wrong with assuming that the conservation of parity is wrong?” So I said, “Well, let’s see. That would mean you could distinguish right and left, in a fundamental way, but there’s nothing the matter with that. I don’t see anything wrong with it. But I haven’t been involved in these things, and I’ll ask the experts tomorrow, hm.” So I said, “That’s a very good question, and you should ask the guys tomorrow.” So he said, “No, you ask them for me. They won’t even listen to me.” “All right,” I said, “Ok.” So I got up and I said, “I’m asking this question for Martin Block.” I’ve been teased a lot about that. People tease me on the grounds that I said that because I thought it was such a ridiculous idea. But it was quite the opposite. I said that because I wanted to establish the correct priority for the idea. I swear that. I mean, it was not because I thought it was silly, but because I thought it might be possible. And it was so good an idea, and might be possible at that time, that I wanted to be sure they knew where it came from. I said, “I ask this question for Martin Block. What goes wrong if you assume —” you know, that parity is not violated. And I think Lee answered it, or something. It was a long complicated answer that I didn’t understand. Then afterwards Block said, “What did he say?” And I said, “I don’t know, Martin, what he said. It seems to me still possible that parity is violated.”

To the people in the room who followed what was going on a little more closely than Feynman, apparently the idea felt a little far-fetched. Oppenheimer, once the group had discussed the idea for a bit and explored the possibility together, closed that bit of the discussion saying something like, “Well, I think it’s time to close our minds again.”

Block’s point, in the end, did turn out to be right. And the community deserves credit for their extreme openness to constantly examine the possibility that one of their new principles, which they were surely proud of, should possibly be replaced with something better or removed entirely.

There was a general sense of understanding in the physics community that the modern theories that physics rested upon, then about four decades old, had also felt absurd to the field when first proposed. So, a physicist should be careful when calling any idea silly before really thinking it over.

When Charles Weiner, the oral history interviewer, asked Feynman to expand on the rapport that he and Professor Wheeler had when Feynman was a graduate student — casually tossing frequently incorrect but always interesting ideas back and forth — Feynman said the following:

Yes. Well, what he did, you see, things like — I’d like to remark that the moment he mentioned advanced waves — that is, against causality and all this other stuff is against cause, the causes would precede the effect — no, the causes would follow the effects instead of preceding them, and so on — I didn’t ever say, “But that’s impossible!” or anything like that. I was not ever upset by any of the obvious troubles, as against some principle of causality or something. This was from the training we had in physics from Einstein and Bohr and so on. See, the history of physics was that a crazy idea like relativity, which is so evidently nutty — like when one man thinks two things are simultaneous, some other guy riding by doesn’t say so — or, that you can’t measure simultaneously position and momentum, or something — It had been discovering that you must always think carefully about the real experimental situation before you cavalierly say such a thing is impossible, you don’t like it. So I never objected to any of these crazy ideas, on those grounds. I never said, for instance, “How can it go backwards? How would it know when it’s going to meet an electron?” I knew that that was something we would have to study — that that wasn’t obviously cockeyed. The fact that there were protons and not positrons were an obvious trouble, but I let him get away with it, so someday we’ll discover how the protons go, wind up in this knot, too. But never mind. His brilliance, the wildness of his ideas, apparently impossible ideas, did fall on fertile soil, because I never objected to what other people would immediately have objected to, you know. All the books would say we can’t use advanced waves because this would mean effects would precede causes. But things like that never bothered me.

Feynman continued:

Feynman: Those didn’t bother me, until we would sit down and analyze and find out, this is necessarily against experiment. That was clear, that we always had to do that, because you see, it would be too easy to object — it was a lesson that you can’t object to Einstein’s ideas on Page 1 in spite of the fact that they look like they’re wrong. How can something shrink when it moves? Sit down and analyze if it’s not impossible. But it isn’t impossible, see? That we had learned. I’m telling you this because it shows something about the history of physics, the connection — that the lessons from these other men were just precisely that. Don’t take it too quick that it’s obviously wrong, just because it says something nutty, because you have to first make sure that the nuttiness is really nutty. In other words, take a real experiment; think very carefully that you will get an advanced effect that is directly opposed to what actually happened. When we tried to do that we didn’t get anywhere, see? We didn’t find such a thing. We got around all the paradox. So everything’s OK. So I know. These things never bothered me. And as soon as I tell people these ideas, they often come to me with all this, “Wait a minute, how’s it going to —” But I never had that trouble, in the beginning. The history…

Weiner: — was it a self-conscious reference to history, or an absorbed tradition?

Feynman: Probably an absorbed tradition. Just an absorbed tradition — that you know that nature can look very, very strange, in the fundamentals, and yet produce in the end the natural phenomena in a way, very different looking than you would think at first. It’s all right. You’ve got to think it out, you can’t just jump that it’s wrong.

Feynman on the declining usefulness of conferences

Towards the end of his career, Feynman felt that academic conferences were getting far too big and becoming less useful. To put a rough date on this transformation, around 1956 Feynman was still making positive remarks about the conferences happening such as the Martin Block Conference in the prior section. But by his final 1973 interview for the oral history, he seems to be making generally negative comments about the changes in conferences at the time.

The following excerpts, more or less, should give the reader a rough idea of Feynman’s thoughts on the changing state of conferences towards the back half of his career.

The series of conferences Feynman found most impressive were, by far, a series of small conferences in the immediate post-war period put together for a group of about 20 theoretical physicists. He recalls:

There were meetings of theoretical physicists, of relatively small groups, 20 theoretical physicists who would get together in different parts of the East. The first meeting was on Shelter Island, called the Shelter Island Conference, and there were some theoretical physicists, also Pauli. We were supposed to discuss the theoretical physics problems of the day. The Shelter Island Conference was my first conference with big men, you know, and I was invited there. Bethe was there, Oppenheimer was there, Pauli was there, Breit was there. Everybody of any importance in theoretical physics who was still alive and was around that part of the country was there. Now, where the money came from and who — Oppenheimer had a great deal to do with it, I think, in inviting the people and organizing the thing, somewhere, somehow. Money came from the National something. There was no National Science Foundation then? National something.

He continues:

Many of the problems of the day were discussed: puzzles about what the mu meson is, a suggestion by Marshak of an intermediate meson, a meson produced that disintegrates into mu, which was the pi-meson. A lot of exciting things were suggested and talked about. However, in spite of all this, they ran out of ideas. And they asked me if I wouldn’t explain my path integral way of doing quantum mechanics. So I did. Now, I must have been preparing my manuscript, or finishing writing the manuscript, because it was all organized, and if it had been before this time of working, I would have been so disintegrated (I hadn’t looked at anything) that I couldn’t have done it. So this gives us a certain amount of timing. All right. And I did OK. I explained it. Of course, it’s hard to pay attention to some new ideas, and they didn’t pay much attention I suppose. Then, at this conference or at a later one (and it’s an historical question, it’s easy to figure out) some questions about the Lamb shift business were suggested or measured or indicated, or somebody said they were going to measure it, or it had been indicated that there was some shift. And Schwinger claims to have said that he thought it was due to this electromagnetic energy shift that had been coming out infinity up to now. And I tried to estimate it by how much our damped oscillator shifted in its frequency, but I didn’t understand the real problem. Schwinger understood it better than I, and got too low a value by very many thousands. But Schwinger said that wasn’t the right way to figure it out. I remember this. See, this was up in the meeting that we were talking about it. I would estimate and say, “What’s the matter with this? It comes out too small.” He says, “No, no” — and so on, and so on. So there was some conference at which we discussed this question. And also there was a conference, but I think it was one later, in which it was indicated that the magnetic moment of the electron was not right, that the magnetic moment of the electron was not exactly the Dirac moment but was slightly corrected, and that this possibly was also an electromagnetic correction. Probably at the first conference, but possibly at the next conference, this went on. OK? So we were aware of the problems, and we were beginning to get, from Rabi’s group, some evidence that the magnetic moment was cockeyed, and from somewhere else some evidence that the Lamb shift existed. So that’s interesting and important to me, of course. These three conferences — as far as I can remember there were three conferences—were to me of very great interest and importance, and I was very unhappy when they ended. I asked Oppenheimer later why they ended when they ended, and it was, he said, because they were getting big. It was very difficult, after you invited somebody, not to invite him again. But people always had to be new ones because they were doing something experimental, they had reports, something — and it just began to grow. And there were too many insults, everybody was insulted, and everybody was writing, “Why didn’t you invite me? Why didn’t you invite me?” And he was sick and tired of it, and so he quit this thing. But this was a very important conference. There have been many conferences in the world since, but I’ve never felt any to be as important as this.

About seven years after the 1966 Feynman interviews where the above quotes were collected, Weiner followed up with Feynman for an additional interview. And Feynman had quite negative things to say about what he viewed as the decline in the usefulness of conferences in helping generate new ideas. His negative comments were, once again, related to the propensity of conferences to only grow in size:

Weiner: How are they [conferences] going from what you’ve seen over the years? Is this the same kind of continuing tradition of the same kinds of people coming together? Because in the early period — we talked about the meeting where you got up and said: “Mr. Block has an idea that I would like to tell you about,” and you talked at the following meeting on that, and these were very exciting things. Has it continued in that same tradition?

Feynman: No, they’ve gotten too big. For example, they have parallel sessions which they never had before. Parallel sessions means that more than one thing is going on at a time, in fact, usually three, sometimes four. And when I went to the meeting in Chicago, I was only there two days before I broke my kneecap, but I had a great deal of trouble making up my mind which of the parallel sessions I was going to miss. Sometimes I’d miss them both, but sometimes there were two things I would be interested in at the same time. These guys who organize this imagine that each guy is a specialist and only interested in one lousy corner of the field. It’s impossible really to go — so it’s just as if you went to half the meeting. Therefore half is not much better than nothing. You might as well stay home and read the reports.

Of course, Feynman’s is only one opinion and many things were changing in science at the time. However, he does seem to have accurately observed the trend of incentives/pressure towards research silos early on, which he thought was ridiculous, and pushed against it in his own work. This was not some over-the-hill physicist. In the early 1980s, about ten years after this interview, he would publish his famous quantum computing paper as well as the Feynman Lectures on Computation. If he was an old dog, he was one with many new tricks.

In an age when people feel it’s increasingly impossible to make novel discoveries in multiple areas, Feynman’s in-the-moment observations on how the research incentives were changing and his conscious decision to not give in to them should give us a lot to think about.

Until next time:)

Please leave a like, DM me on twitter, or drop a comment in the new Substack chat if you are enjoying these shorts and would like to see more of them.

Subscribe now

Share

This post is an experiment. It is a ‘Short’ and is made up of four stand alone tidbits that I think you’ll enjoy from Feynman’s oral history. The quotes detail:

1. Feynman’s first reaction to Princeton as a graduate student

2. A bit of what drove his work at the Manhattan project (and how it relates to the Langmuir post)

3. An anecdote on just how hands-on the experimental physicists at Los Alamos were

4. And the backstory behind Feynman’s proof related to Dirac’s work that led him to approach Dirac for the first time (which is a now famously awkward first meeting that you can read more about in this Privatdozent post)

Subscribe now

When Feynman first saw Princeton

Feynman was a bit trepidatious about what life would be like at Princeton before he first arrived. He had never seen the place and, really, had wanted to go to graduate school at MIT. However, MIT would not have him because, even though he was known as one of the Institute’s brightest students, his mentor, MIT Professor John Slater, felt that he should get his Ph.D. at a different university to experience how science was done elsewhere.

So, Feynman chose Princeton because he would have the opportunity to work for Wigner there as a research assistant — an opportunity that fell through due to some administrative mix-up.

Anyway, he was nervous about the social life at Princeton because he knew he was not a good interpersonal fit for stilted formalities — the University seemed to have a strong desire to maintain certain Oxford-like traditions — like afternoon tea and dinner gowns. This was very much the opposite of what MIT was like at the time as exemplified in Part I of my early MIT series.

Feynman remarked on what happened when he and his roommate were told to come to tea on their first day:

I was scared because of this same thing I was talking about — I’m not so good at this. “The Dean’s tea” — it sounded so silly, you know, and high class. He [the roommate whose father was a renowned chemist] took it in stride, because he was that kind of guy. And we went to the Dean’s tea the first day I was there — it was a Sunday, I guess — and the Dean, Dean Eisenhart, was in the line going in. I told him my name, and he said, “Oh yes, I know, you’re from MIT,” and so on. I was kind of pleased by that. Then, when I went in, I was looking around, where to sit and everything, I was concerned with all these matters, and there were some girls around. I felt rather stiff. Then I heard a voice behind me say, “Would you like cream or lemon in your tea, Sir?” It was Mrs. E., and I said, “Both, please, “-– absentmindedly — and then there was a nervous laugh that I could hear and she said, “Surely you’re joking, Mr. Feynman!” Then I had to turn around and figure, what was I joking? What was the question? It was really quite — so I started out on the wrong foot with the social things.

That could’ve gone better, but he knew that some of these more formal bits of Princeton life wouldn’t be a good fit for him. But that was to be expected. A big reason that he picked Princeton, on top of the opportunity to work with Wigner, was that, in reading the Physical Review, he had noted that there was a lot of very very good work coming out of Princeton. He had particularly noticed that great work that interested him was coming out of the University’s cyclotron.

Having seen MIT’s extremely high-tech, pretty cyclotron, he could only imagine what this beautiful piece of equipment would look like at this cyclotron research powerhouse. Feynman remembers:

I had read the journals, yes, and I knew things that were going on because I’d seen the articles, and I knew that from the Princeton cyclotron research lots of work came, good work. I also knew that at MIT they had a marvelous cyclotron. They were very proud of it. MIT was self-confident and proud, and everybody at MIT thinks it’s great, and I thought that it was great. It was essentially gold-plated, if you know what I mean — I don’t mean literally — and the control board was in another room, with special glass panels and knobs and everything. It was very nice. I’d seen the cyclotron. But I knew from the journals that not much was coming from the MIT cyclotron, relatively, and therefore the Princeton cyclotron must really be something — you know? Of course, the MIT one was big, in two rooms, and so on. So I got to Princeton, and the first thing, when I was there and I went to the physics building, I asked immediately, “I want to see the cyclotron” — because I was very excited. And they said to go down in the basement and the room down at the end of the basement — which seemed to me incredible, stuffed away…Anyway, I went down in the basement, and I walked into the room where the cyclotron was, at the end of the basement. And it wasn’t 15 seconds before I understood why the Princeton cyclotron had lots of results, why Slater had told me to go to another school — I understood the whole thing. The whole mirage, the whole idealism of MIT collapsed. Because I recognized something in that room, which was the same as in my laboratory at home. The cyclotron was in the middle of the room. There were wires all over the place, hanging in the air, just strung up by somebody. There were water things — there had to be automatic water coolers, and little switches, so if the water stopped it would automatically go on, and there were some kind of pipes and you could see, you know, water dripping. There was wax all over the place, hanging, where they were fixing leaks. The room was full of cans of film at crazy angles on tables. You see, completely different than at MIT. A place where somebody was working! Where the guy who was working was close to the machine, could fix it with his own hands. It was not in an insulated box with knobs. I understood it immediately, because I’d had this experience in laboratory. It looked like my kid laboratory, where I had everything all over the place and the tools were put down where I last had ‘em. And I realized that this was really research, and that I had been fooled — that good engineering design is what they had at MIT, in a kind of abstract way, but not the real work with the machine, that they were separated from it. I understood it very quickly, as soon as I saw the machine. I loved it. I knew I was in the right place. They were guys of the old — the way I had felt when I was a kid. Fiddling is the answer. Experimenting is fiddling around. It’s not an organized program, elegance — it’s impossible. I noticed it. I mean, I realized right away that Slater was right. I had thought that was the best school in the world, and here was a thing I’d imagined must be three times as great, ten times as large, and four times as elegant, in order to get that much more research. But as a matter of fact, it was smaller and completely inelegant, and that was the secret. So I loved Princeton right away.

Accounts like this, in an age of bigger science and fancier machines being the name of the game in so many research areas, should stick with us.

A Feynman quote that relates to the recent Irving Langmuir piece

Feynman spent hours accounting to Weiner, the historian conducting the interviews, what his daily life looked like at the Manhattan Project. To many, including Weiner, these stories paint the picture of an overly energetic theorist, with innovative problem-solving skills and an eye for good experiments, darting around Los Alamos and working on whatever exciting thing caught his eye. He has, in fact, become known for this type of energy. And it’s true, in part. But, when Weiner asks about this, Feynman makes a clarification that echoes the themes of the Irving Langmuir piece.

A great deal of it is play, you see. I mean, I look for problems and I do things. I know, but play that was contributing, you see, and not fiddling around. I never fiddled around there. I played a lot, but I always played in a way that was directed. I could always explain the play as not useless, you see. There was a tremendous amount of play. That’s really what it was — so many problems — I’d look for them, because I liked all these crazy things. Yes, very much like play. But always with a purpose in the end.

He was playing, but with the constraint of only working on problems related to work streams already underway at Los Alamos. Just like Langmuir, this model brought a lot of interesting and productive fundamental research questions directly to Feynman, whether it be thinking through how to construct a more useful fission counter or novel work on the compressibility of substances under extreme pressures.

Now, I do not believe that Feynman — in any way — required being bombarded with these problems to come up with great research questions because he seems to have had a freakish production function that worked well independent of this. But much of the work and ideas that came his way from this applied work, a lesser mind could have also made a dent in.

And that, after all, was the point of the Langmuir piece. Feynman, for a variety of reasons, could just play ball. You just needed to stay out of his way. But most people need a little more help for their inspiration. And the well-balanced, constrained freedom of a place like Los Alamos is yet another example of an incentive structure similar to the early GE lab — probably a little more constrained in this case given that they were on a time crunch.

A reminder of how hands-on many researchers were in Feynman’s era

The following is a famous excerpt that I always think back on when I try to conceptualize just how applied and hands-on the skillsets of so many of the experimentalists of early and mid 1900s were:

When I got there [to Los Alamos]…there was Paul Olum, my assistant, with a clipboard and paper, checking the trucks of dirt and boards that were coming in the gate, you know, checking them off — how many loads of lumber. Then I went into the building just finished, one of the few buildings that was finished, and somebody said, “This is John Williams — Dick Feynman” — you know. And I had heard of John Williams. He had his name on papers in nuclear physics. I’d heard of many people but this was when I first met them. He was somebody I respected a great deal, big scientist, you know. His job — he was in his shirt sleeves, sitting there with big blueprints all over the place in front of him, like he was a building inspector or contractor. He gets up, “Hi, glad to meet you,” and sits down — some workman comes over and he says, “Now, you put a line in here, you put this in there.” In other words, we went up there, and the experimenters who had nothing to do because there was no laboratory, finished the building. They helped the contractor. They just went in there and they checked the trucks, they carried the blueprint information over, you know, they did everything that the contractors would do, to make it faster, to help them. So it was a very exciting interesting thing. Dust from the trucks, half-finished buildings — you have to get a picture of it. We were there ahead of time. And we helped to make it, you see. I don’t know if this appears anywhere, but it was an exciting business.

If you’re looking to repair, fiddle with, and improve upon lab instruments, in an effort to constantly expand what can be measured and known, the researchers who could complete Physics Ph.D.s AND build the laboratory buildings from scratch are probably the ones you want on your team.

I can’t say for sure that our current experimentalists have lost their edge in this area, but I’d probably take 10:1 odds that the answer is “yes.”

Feynman’s quest for usefulness and why he approached Dirac for the first time

Dirac and Feynman first met in person at the Princeton Bicentennial Conference. Dirac was giving a paper and Feynman was to introduce him and the paper.

At one point in the conference, Feynman remembers noticing something strange as he glanced away from some of the great scientists of the day like Bohr and others arguing about something:

And they’re all sitting around worrying, and they’re talking, and I look out the window, and through all this Mr. Dirac, paying no attention to anybody, had walked out and was sitting on the grass, lying on the grass with his elbow against his head looking up at the sky. I thought, that’s interesting, and I went out too. I went out to him and I said, “I guess you don’t care what they’re saying — I don’t remember, something like that, because I didn’t believe what they were saying either. I felt a kindred spirit. I talked to him a minute. But the main thing I remember was, I’d wanted always to ask him a question, and I asked it.

Feynman said:

He was lying on the grass, and I said to him, “By the way, Professor, you know that paper in which you say those quantities are analogous,” and so on. He said, “Yeah.” I said, “Did you know they’re proportional?” He said, “Are they?” I said, “Yes.” “Oh. That’s interesting.” That’s all.

This entire scene is very typical of Dirac. In fact, that is comparatively a lot of words for him to have said to a stranger. Also, it is interesting that Feynman would see in Dirac, his interpersonal opposite, a fellow kindred spirit. Feynman is not the only person to notice this odd similarity between him and the near-silent Dirac — who was an idol of his. Wigner once noted on young Feynman, “‘He is a second Dirac, only this time human.”

Getting back to the more useful point, how exactly young, grad student Richard Feynman had proved this equivalence in Dirac’s work was very typical of Feynman and very relevant to this Substack’s recent coverage of usefulness and its importance in American basic research.

There was a quality in quantum mechanics, defined by Dirac, which was an integral kernel to carry the wave function from one time to the next instant of time. It’s defined in the equation just after equation 33, the function of 𝜲′. And Dirac pointed out that this function in quantum mechanics was analogous to the exponential of 𝜾 times 𝜺 times the Lagrangian, where the velocity for the Lagrangian you put 𝜲 minus 𝜲′ over 𝜺, and for the position just 𝜲. It said in the paper that the two things were analogous. Jehle [a German physicist] showed me this and I read it and I said, “What does it mean that it’s analogous? What is the significance of saying that something is analogous to something else? “It just means that it’s similar, it’s analogous in some way.” I said, “I don’t know. What’s the use of that? It can’t mean anything, it has no use.” Jehle said, “You Americans, always looking for a use for a thing!” So I said, “Well, Dirac must mean that they’re equal. It doesn’t mean anything otherwise.” He said, “No, Dirac doesn’t mean they’re equal.” I said, “Well, let’s try and see if they could be equal,” so I substituted one expression for the other, and calculated what the wave function would be at the next instant, and found that if I didn’t make them equal but rather proportional, by multiplying by a constant, that as a matter of fact it was equivalent to a statement of the Schrodinger equations. So I worked out the Schrodinger equation from that right on the blackboard, and turned around to Jehle and said, “See, Dirac meant they were proportional.” But Professor Jehle said, “No, no, Dirac didn’t know that, you have just made an important discovery,” and he was very excited and copied everything into his notebook. I didn’t realize — I was only trying to interpret Dirac — but he realized that I had discovered something that wasn’t known. He said, “You Americans, always trying to find a use something! That is a way to discover new things.” He was quite convinced of it after that. I never was sure, really sure, that Dirac didn’t think they were proportional until way later, in 1947, when I saw Dirac at Princeton.

I can’t conclude any better than that.

Share

Subscribe now

I’ll release another short on the Feynman oral histories in the next few days. Please dm me on Twitter to let me know if you liked this experiment in changing up the content or feel free to drop a like on the post! It’ll help me decide what types of content to work on moving forward. Until next time:)