Today’s piece was put together with the help of several conversations with Answer.AI co-founder Jeremy Howard. It is not a “traditional” FreakTakes piece; the research and advice are much more tailored to a specific group of individuals than usual. However, I was excited to share it on the Substack for those curious to know more about Answer.AI or what the applied side of my FreakTakes work looks like.
Jeremy Howard (former President and Chief Scientist of Kaggle) and Eric Ries (creator of The Lean Startup movement and Long Term Stock Exchange) have teamed up to found a new applied R&D lab: Answer.AI.
When speaking with Jeremy, he made it clear that many details of Answer.AI's structure are still being worked out. Only announced a month ago, the org is still in its early development stages. But the founders have conviction on certain principles. The most prominent of them is one extremely relevant to this Substack: The founders seem to be particularly inspired by Edison’s Menlo Park Lab and the early days of commercial electric research.
Previously, I’ve covered Edison’s Menlo Park Lab, the early GE Research Laboratory, and many other industrial research shops. Since FreakTakes is explicitly written for individuals founding new science orgs, has a particular focus on applied R&D, and has covered the lab that partially inspired Answer.AI, I thought it would be worthwhile to put together a piece specifically for Jeremy and Eric. The goal is to distill into a single piece the takeaways from prior FreakTakes pieces that are most useful to their nascent company. If they take even one nugget of information and use it to run their organization slightly differently, I’ll consider that a win.
In the piece, I’ll briefly examine the (working) plans for the lab and do some historical analysis, detailing:
What the earliest electrical R&D labs can teach Answer.AI
Useful rules-of-thumb from other historically great applied R&D labs
Potential pitfalls to keep in mind as they move forward
You can find more thorough historical evidence in my prior pieces for any of the lab details I mention, listed below:
Thomas Edison, tinkerer published in Works in Progress
Tales of Edison's Lab (podcast)
Irving Langmuir, the General Electric Research Laboratory, and when applications lead to theory
How did places like Bell Labs know how to ask the right questions?
“The Third University of Cambridge”: BBN and the Development of the ARPAnet
An Interview with Chuck Thorpe on CMU: Operating an autonomous vehicle research powerhouse
A Progress Studies History of Early MIT — Part 2: An Industrial Research Powerhouse
How Karl Compton believed a research department should be run
Each of the orgs listed has lessons to teach Answer.AI. But none are a perfect analog. So, as the piece progresses, I’ll explain which lessons I think most strongly apply to Answer.AI. With that, let’s get into it!
Answer.AI in a Nutshell
Jeremy’s blog post announcing Answer.AI makes it clear that the org is, to a large degree, inspired by the field of electricity’s path of progress in the 1800s. He believes the current state of the AI field is similar to the state of the electricity field between the work of Michael Faraday and Edison’s lighting projects. This was an era in which new electrical findings were being pieced together, but few had made any progress in turning the potential of electricity into great applications.
I don’t find this comparison crazy. So far, I don’t believe AI has come close to the level of breakthrough that electricity proved to be. Electricity brought the sunlight indoors for a negligible cost and powers so many of our modern conveniences— refrigeration, TVs, central heating, etc. That’s a high bar. However, given that human ingenuity created the breakthrough that was electricity and each of those applications, it is surely worth considering that AI could grow to be the most impactful field of them all. Whether AI does reach that level of promise, to me, is a question of human ingenuity. So, I have no issue with Jeremy comparing the AI field to the electrical field c. 1830 to 1910.
With that elephant out of the way, let’s briefly examine what sets Answer.AI apart from AI labs like OpenAI and Anthropic. From a funding perspective, Answer.AI seems much, much cheaper. The founders have initially raised $10 million. This stands in stark contrast to the gargantuan initial rounds of OpenAI and Anthropic. Also, Answer.AI's research agenda is more application-centric. The following excerpt from Jeremy’s blog post highlights what he thinks differentiates the lab’s approach:
At Answer.AI we are not working on building AGI. Instead, our interest is in effectively using the models that already exist. Figuring out what practically useful applications can be built on top of the foundation models that already exist is a huge undertaking, and I believe it is receiving insufficient attention.
My view is that the right way to build Answer.AI’s R&D capabilities is by bringing together a very small number of curious, enthusiastic, technically brilliant generalists. Having huge teams of specialists creates an enormous amount of organizational friction and complexity. But with the help of modern AI tools I’ve seen that it’s possible for a single generalist with a strong understanding of the foundations to create effective solutions to challenging problems, using unfamiliar languages, tools, and libraries (indeed I’ve done this myself many times!) I think people will be very surprised to discover what a small team of nimble, creative, open-minded people can accomplish.
At Answer.AI we will be doing genuinely original research into questions such as how to best fine-tune smaller models to make them as practical as possible, and how to reduce the constraints that currently hold back people from using AI more widely. We’re interested in solving things that may be too small for the big labs to care about — but our view is that it’s the collection of these small things matter a great deal in practice.
It would be unfair to say that an application-centric research agenda is necessarily less ambitious than AGI. Those biased toward basic research might say so, but I don’t think that opinion is very historically-informed. Edison himself was application-centric above all else. His deep belief in market signals is fascinating when juxtaposed with the market indifference of many great academic physicists. In the book From Know-How to Nowhere, a history of American learning-by-doing, Elting Morison described the interesting nature of Edison’s motivations:
If the means by which he [Edison] brought off his extraordinary efforts are not wholly clear, neither is the cause for his obsessive labors. No diver into nature's deepest mysteries carrying next to nothing for the advancement of knowledge and even less for the world's goods, he would become absorbed in making something work well enough to make money. The test in the marketplace was for him, apparently, the moment of truth for his experiments.
Edison built his god-like reputation by dreaming in specific applications. He kept market, resource, and manufacturing constraints in mind from the earliest stages of his projects. Edison dreamed practical, realizable dreams. And when the limitations of component technologies stood in the way of his dreams, he often had the talent to invent new components or improve existing materials. Edison’s biggest dream, the light bulb, mandated that Edison solve a much broader set of problems. The following excerpts from my Works in Progress piece on Edison paint a clear picture of his ambitious but practical dreams:
After Edison’s bulb patent was approved in January 1880, he immediately filed another for a ‘System of Electrical Distribution’. Filing for these so close together was no coincidence. To Edison, it was never just a bulb project. It was a technical business venture on a possibly unprecedented scale. Edison wanted to light up homes all over the world, starting with lower Manhattan.
Bringing the project from dream to mass-market reality would require solving over a hundred technical problems. His was a new bulb that needed to be powered by a generator that did not yet exist at the start of the project, strung up in houses that had no electricity, connected via underground street wiring that was only hypothetical, and hooked up to a power station that had never existed before.
Yet, at the end of two years’ time, Edison would do it. And, just as importantly, the entire venture was profitable by the end of the project’s sixth year.
Edison was clearly doing a different kind of dreaming than those who do basic research. His lighting work embodies what extreme ambition looks like in application-centric research. Answer.AI making this kind of ambitious, applied work their North Star is an extremely interesting goal.
This goal has the potential to give Answer.AI a comparative advantage in the growing space of for-profit AI labs. For example, the most ambitious aspects of OpenAI are considered to be in its research, not its work on applications. Answer.AI’s particular setup can also set it apart from AI startups and academic labs. New AI startups do some research on how to commercialize new AI models in new ways, but they generally have short runways. In this kind of environment, only specific types of research projects can be pursued. Academic labs — for many reasons covered elsewhere on this Substack (such as in the ARPA series) — don't have the right combination of incentives, experience, and staffing to build new technologies in most problem areas. The main incentive of the profession, in a simplified form, is producing many paper studies that get cited many times. Answer.AI has the chance to let its alternative focus lead it to areas under-explored by academics, companies with brief timelines to hit revenue benchmarks, and more AGI-focused R&D labs.
Legally, Answer.AI is a company. But in practice, it might hover somewhere between a lab and a normal “profit-maximizing firm” — as was the case with Edison's lab. The founders seem perfectly content to pursue high-risk projects that might lead to failures or lack of revenue for quite a while. In saying this, I do not mean to imply they are content to light money on fire doing research with no chance of a return. Rather, they hope to fund a body of research projects that ideally have positive ROI in the long term. They are just not overly concerned with short-term revenue creation.
(Making the pursuit of research agendas like this easier is actually one of the founding goals of Ries’ Long Term Stock Exchange — which I address later.)
There is apparently no pressure to produce a product that can hit software VC-style revenue goals within 12-24 months, or anything similar. This is good. Seeking to satisfy these types of metrics does not traditionally permit a company to act like a truly ambitious R&D lab. I’m not saying it can’t happen — DeepMind seems to have made it work in its early years — but it does require pushing against investor pressure quite strongly. The VC money raised for Answer.AI has left the founders with enough voting shares that investors can’t veto founders’ decisions. Additionally, Howard says the company’s investors understand what they are trying to build is, first and foremost, a lab. This is a great step towards building an organization focused on building very useful, very new things rather than the most profitable thing possible — which often comes with bounded technical novelty.
Interestingly, Answer.AI will also keep a small headcount. Jeremy built Fastmail up to one million accounts with only three full-time employees. He hopes to keep the Answer.AI team exceptionally talented and “ruthlessly small” in a similar way; he believes keeping teams small is important to building new, technically complex things.
Now that I've outlined some important pieces of Answer.AI’s vision, I'll dive into the historical analysis. In the first section, I detail lessons that Answer.AI can draw from both Edison’s Menlo Park laboratory and the Early GE Research Laboratory. In the following section, I'll share useful lessons from other historically great industrial R&D labs. Lastly, I’ll highlight the bureaucratic details that explain why the operational models of the great industrial R&D labs have not been replicated often.
Learning from the First Electrical R&D Labs
I find it exciting that Edison’s Menlo Park lab is a North Star for Answer.AI. I covered Edison’s work in several pieces because I think evergreen lessons can be drawn from his work. But I think a more complete way to incorporate lessons from the 1870-1920 electrical space is to draw on the work of both Edison’s Menlo Park Lab and the young GE Research Lab. The latter operated as a more traditional industrial R&D lab. GE Research’s history holds many lessons to help steer Answer.AI’s problem selection and work on its standard projects. However, exceptionally ambitious projects may draw more heavily on the lessons of Edison’s lab.
(As a note, while Edison General Electric was one of the two companies that merged to become GE — along with Thomson-Houston Electric — Edison had essentially nothing to do with the formation of the iconic GE Research Laboratory.)
Different types of projects characterized the work of the two electrical labs. When it came to electrical work, for years, Edison’s lab and mental efforts were focused on doing everything necessary to bring a single, revolutionary product to market. On the other hand, GE Research usually had many separate courses of research underway at once. These projects all sought to improve the science and production of existing lighting systems, but they were otherwise often unrelated to each other. Additionally, GE’s work could be categorized as more traditional “applied research.” The lab was not actively looking to create a field of technology from scratch as Edison did. GE Research's projects were often novel and ambitious, but in a different way than Edison's.
Later, I will explore the types of novelty the GE Research Lab pursued. First, I’ll give the reader a more fine-grained idea of how Edison’s lighting project actually operated.
Lessons from Edison’s Work on Electricity
Edison’s lighting work provides great management lessons for those looking to direct a large chunk of a lab’s efforts toward a single, big idea.
Edison’s major contribution to the field of electricity was not inventing each of the components in his lighting system, but in turning a mass of disparate gadgets, scientific principles, and academic misconceptions into a world-changing system. The burden of doing “night science” — as Francois Jacob refers to it — largely fell on Edison. In the late 1870s, nobody knew much about electricity yet. The existing academic literature had more holes than answers, and many of its so-called “answers” turned out to be wrong or misleading. From this shaky starting point, Edison proceeded. He combined his unique mix of attributes and experience to deliver a world-changing system. These included: knowledge of several adjacent scientific fields, deep knowledge in then-overlooked experimental areas, market knowledge, manufacturing knowledge, and the ability to adequately operate a small research team.
In large part, Edison created his lab as a way to scale himself. As a result, to understand how his lab operated, one needs to know how Edison himself carried out his explorations. Edison was one of the more stubborn experimentalists of all time. He spent most of his waking hours carrying out one experiment or another. While he did pore over scientific literature, for him, nothing was settled until he proved it for himself at the lab bench.
I write in my Works in Progress piece:
Edison respected scientific theory, but he respected experience far more. In Edison’s era of academia as well as today’s, many professors had a certain preference for theory or ‘the literature’ over hands-on improvement. Because of this Edison did not care much for professors. He was even known to go on long diatribes, during which he had assistants open up textbooks, locate scientific statements that he knew to be untrue from experience, and quickly rig up lab demonstrations to disprove them. ‘Professor This or That will controvert [dispute with reasoning] you out of the books, and prove out of the books that it can’t be so, though you have it right in the hollow of your hand and could break his spectacles with it.’
Contained in his head was a database of countless experiments and results that made it seem as if his “intuition” was far beyond his contemporaries. This left him with an unparalleled skillset and body of knowledge. If anyone could feel comfortable pursuing a project that others had previously failed at, it was Edison. Edison’s confidence in his skills was never more on display than when he chose to pursue his lighting work. Many in the scientific establishment knew electric bulb lighting was technically possible, but claimed they had proven that it could never be economical. Edison disagreed.
On top of Edison’s admirable approach to experimentation, he brought a high level of practicality to his process. He knew his inventions needed to make commercial sense in order to make it out of the lab. So, even in early courses of experimentation, he kept factors like manufacturability in mind. He wouldn’t commit much time to something that didn’t make commercial sense. With that being said, Edison wanted to change the world with his technologies more than he wanted to get rich. So, the practical factors he paid aggressive attention to were primarily treated as constraints. He did not optimize for profitability, but he knew his ideas needed to be profitable. Nobody who wanted to optimize for profit would have pursued lighting in the way Edison did. The technical risks were too great.1
Edison was able to imagine an ambitious system that required many technical advances. It was so futuristic that maybe only he was capable of coming up with it. But just as impressively, he was able to do it profitably and on schedule. His dogged commitment to experimentation seems to be largely responsible for this. Edison and “the boys” constantly experimented on every piece of the process to improve and learn more about all the sub-systems in Edison’s grand system. They wanted to know how every piece of every sub-system performed in all conditions. I’ll share just two excerpts from my Works in Progress piece as examples.
The first is from Edmund Morris' biography of Edison. It recounts how thoroughly Edison and his trusted aid, William Batchelor, were in carrying out round after round of filament experiments:
For week after week the two men cut, planed, and carbonized filaments from every fibrous substance they could get — hickory, holly, maple, and rosewood splints; sassafras pith; monkey bast; ginger root; pomegranate peel; fragrant strips of eucalyptus and cinnamon bark; milkweed; palm fronds; spruce; tarred cotton; baywood; cedar; flax; coconut coir; jute boiled in maple syrup; manila hemp twined and papered and soaked in olive oil. Edison rejected more than six thousand specimens of varying integrity, as they all warped or split…
In the dog days, as heat beat down on straw hats and rattan parasols, the idea of bamboo suggested itself to him. Nothing in nature grew straighter and stronger than this pipelike grass, so easy to slice from the culm and to bend, with its silicous epidermis taking the strain of internal compression. It had the additional virtue, ideal for his purpose, of being highly resistant to the voltaic force. When he carbonized a few loops sliced off the outside edge of a fan, they registered 188 ohms cold, and one glowed as bright as 44 candles in vacuo.
This approach went far beyond bulb filaments. The following excerpt describes the work of one of Edison’s lead mechanics in turning the Menlo Park yard into a 1/3 scale model of what they would later install in Lower Manhattan. I write:
[Kruesi, Edison’s mechanic] along with a group of engineers and a team of six diggers, turned the excess land of the lab in Menlo Park, New Jersey…into a one-third-scale model of Edison’s first lighting district in lower Manhattan. This team tested and re-tested the electricity delivery system, digging up Menlo Park’s red clay to lay and re-lay an experimental conduit system. The team carried out countless tests to ensure that they found materials to efficiently carry the electric current while also keeping the delicate materials safe from water and ever-present New York City rats.
The entire process was marked by the classic trial-and-error of the Edisonian process. The first subterranean conducting lines and electrical boxes the group laid were completely ruined by two weeks of rain — despite being coated with coal tar and protected with extra wood. While the diggers dug up the failed attempt so the damage could be examined, Kruesi and a young researcher…studied and tirelessly tested unbelievable numbers of chemical combinations — making full use of the laboratory library and chemical room — until, finally, a blend of ‘refined Trinidad asphaltum boiled in oxidized linseed oil with paraffin and a little beeswax’ was found that protected the electrical current from rain and rats.
Edison built his own style of dogged experimentation into the culture of his lab. Since the lab was meant to scale Edison, this makes perfect sense; he was a man with far more ideas than hands. So, he hired more hands. Edison did not search far and wide to hire the world’s best research minds, and many of those he employed did not even have scientific backgrounds. This didn’t matter much to Edison because most of them were employed to undertake courses of research that he had directed them to pursue. A couple of his Menlo Park employees had advanced scientific degrees, but far more did not. For the most part, the lab and its activities were steered by Edison and his ideas. As a result, the productivity of his lab followed wherever his attention went. After some time working on a project area, Edison would often grow antsy and wish to move on to the next thing — he craved novelty. The lab’s resources and extra hands would move with him. As we’ll see in the next section, this stands in stark contrast to how the GE Research Lab recruited and chose problems.
Menlo Park's electrical activities provide a great management playbook for what it looks like to direct a lab’s efforts toward a single, major system. If Answer.AI does not want to go all-in on one thing, it can still find a way to apply this playbook to a certain focused team of employees while leaving the others to tinker around with exploration-stage ideas. In Edison’s less-focused experimentation periods, his lab served as more of an “invention factory,” doing this sort of fiddling. Additionally, Edison's preference for application and commitment to experimentation over theory in a young area of science can surely provide Answer.AI some inspiration.
Of course, Edison did some things better than others. Edison’s most easily-spottable “deficiency” is that his lab was largely dependent on him. Without him and his big ideas, the lab would have probably ground to a halt. While Edison’s technical vision, practicality, and experimental approach are absolutely worthy of emulation, the lessons of GE Research should probably be added into the mix as well. GE operated as more of a prototypical industrial R&D lab with an approach quite suited to the fact that the science of electricity was beginning to mature in the early 1900s.
Lessons from the Young GE Research Laboratory
The young GE Research lab took a different approach to electricity research than Edison. The lab worked on many unrelated projects at once, recruited more talented researchers, and allowed these talented researchers more freedom to exert the scientific method on commercializable projects. The lab did not undertake projects that were as purposely futuristic as Edison did. Nobody from the lab earned nicknames like “the Wizard of Menlo” or “the Wizard of Recorded Sound.” But early GE Research was still responsible for a Nobel Prize and making the light bulb a much-improved, more cost-effective technology.
Elting Morison wrote the following on the lasting impact of GE Research’s early decades:
There seems little doubt that…much that was done in Schenectady in electrical engineering and some parts of physics was both better done and more interesting than what was being done in those fields in any American university.
In its heyday, even great researchers like Karl Compton hoped to shift their academic departments to operate more like GE Research.
While GE did simultaneously pursue diverse projects, there was a unifying thread holding all of the projects at GE Research together. Each project aimed to improve the quality and profitability of GE’s products and manufacturing. Under that unifying theme, all kinds of projects were encouraged. Much of the research was very applied, particularly in the early years when the lab was still proving itself.
William Coolidge was one of the lab’s most talented applied researchers in its early years. Coolidge joined the lab in 1905, part-time while teaching courses at MIT. Coolidge had the kind of toolkit typical of many MIT professors in that era. He had a far greater grasp of the science of physics and metallurgy than somebody like a blacksmith; he was simultaneously far closer to a blacksmith than one would ever expect a university researcher to be. With this differentiated toolkit, he did science in a way that was not typical of academics. In describing the process that led to his successes at GE, he claimed that he was, “guided in the main by experiment itself rather than by metallurgical knowledge.”
Coolidge expertly applied practical skills in concert with scientific knowledge to pursue the problem. Elting Morison described a small sample of Coolidge’s workflow:
He suspended tungsten powder in an amalgam of bismuth, cadmium, and mercury. He then passed the resulting substance through tiny dies — drawing it — and obtained a silvery pliable wire. At that time, he thought he had reached ductility and the search was over. But when a current was passed through this wire the mercury, cadmium, and bismuth distilled out, leaving, unfortunately, a nonductile tungsten. But it also proved to be tungsten in the purest state he had yet produced.
I continue in my FreakTakes piece, writing:
He eventually iterated his way to a workable process where…the more pure tungsten was put through a specific combination of metal-working processes at a temperature that worked that produced rods of tungsten about 1 mm in diameter. These 1mm rods could then be drawn and re-drawn through rods of decreasing size until you were left with wires of tungsten .01 mm in diameter. When put in the vacuum-sealed bulb, electricity ran through the tungsten filaments and demonstrated an efficiency of 1 watt per candle — extending the life of a bulb up to 27x.
Within 5 years, 85% of all lamps would be made from tungsten. As the project went on, more and more research chemists and technical assistants grew to be involved in the wide-ranging steps and combinations involved in Coolidge’s experiments. But it worked. GE had the factories re-fit and deployed the new bulb. Coolidge moved on to other research.
The success of Coolidge’s hybrid work style, not dissimilar to Edison’s, is surely a useful data point to Answer.AI. But GE Research also did work that went far beyond Coolidge’s technically adept, applied science. The lab was fantastic at making use of talented individuals who were very academic. Irving Langmuir was a prime example. I described his interests in my original piece:
It should be noted…Langmuir did not even care about lightbulbs. Well, I guess that is not technically true. The bulb interested him because 1) he thought a metal like tungsten was cool because it could accept really high temperatures which opened up options to the scientist working with it and 2) these vacuum-sealed bulbs provided a pristine environment for controlled scientific investigations.
To Langmuir, light bulbs were primarily a playground in which to do his science. But Willis Whitney knew how to take an individual like that and direct his energy towards productive ends. The lab deployed a principle that I call extending a “long leash within a narrow fence” to basic researchers like Langmuir.
The way the lab facilitated this was rather simple. On his first day, Langmuir was told to walk around the applied end of the lab and ask people about their projects. Whitney permitted him to undertake any course of investigation of any phenomenon he wanted, but it had to be directly related to an existing problem/limitation/constraint that the applied folks were working through. These applied folks were working on projects that rather directly plugged into GE’s operations, so there was minimal risk of Langmuir’s work not amounting to anything useful if he succeeded and found answers. With that assurance of applicability, Langmuir was given extensive timelines to find answers to open questions.
Langmuir’s first course of research focused on the constant bulb-blackening problem common to bulbs at the time. The problem was generally attributed to a bulb’s imperfect vacuum. Langmuir found this problem to be a great excuse to carry out a course of experimentation he found interesting. Morison described Langmuir’s thought process as follows:
If residual gases — imperfect vacua — produced a bad effect — blackening — here was a fine opportunity to study the effects produced by different gases introduced one by one into the bulb. What he wanted to do, he told Whitney, was simply to plot the interactions of various gases exposed at low pressures to very high temperatures in the filament. Nobody knew very much about this phenomena and he wanted to look into it simply “to satisfy [his] own curiosity.”
Langmuir carried out this course of research over three years. There were many gases and temperatures to test, which took time. But unforeseen results constantly took Langmuir off in different directions. Exploring these unforeseen results often entailed new courses of experiment altogether. With his long leash, Langmuir was able to figure out that imperfect vacua were not what caused bulb blackening at all. Rather, it was that tungsten vapor particles were finding their way onto the wall of the bulb. Temperature was the issue.
He also discovered that different gases markedly changed the rate of evaporation. One extreme example was nitrogen, which reduced the evaporation rate by 100-fold. However, adding nitrogen to the bulbs caused the electrical efficiency of the system to decrease drastically. So, the existing bulb design with nitrogen added was less cost-efficient than the normal bulbs. But Langmuir was undeterred. This was progress.
Existing fundamental research in this area led him to believe that this efficiency issue could be alleviated by increasing the diameter of the filament. Further experimentation proved this to work. He also found that coiling the filament in a certain way could mitigate the heat loss issue. The final result was a novel bulb that used an inert gas instead of a vacuum to reduce bulb blackening. Along with the coiled tungsten filament, this new bulb only required .5 watts per candle and lasted three times longer than any other bulb.
Once he passed the bulb project onto the engineering team at GE Research, Langmuir set his sights on an anomaly he had come across talking with the lab’s more applied staff. The bulbs in the lab had a design that depended on only a few milliamperes of current flowing across the space between one end of the filament and the other. Langmuir noted this anomaly in a letter to Scientific Monthly, writing:
This fact seemed very peculiar to me, for the work of Richardson and others had indicated that at temperatures as high as those used in the tungsten-filament lamp, currents of many amperes should flow across the space. In other words, according to the then-accepted theory of the electron emission from hot filaments, a serious difficulty should have been encountered in the construction of tungsten-filament lamps. The fact that we did not meet any such difficulty therefore seemed to me a peculiar fact that should be investigated.
In the brief course of exploration that followed from Langmuir, he discovered what is now known as the space-charge effect. This work combined with follow-on work from Coolidge to produce an entirely new kind of GE X-ray tube.
Under this “long leash within a narrow fence” guideline, Irving Langmuir would go on to be partially responsible for a handful of new and improved product lines at GE. Additionally, the knowledge he created with his tungsten filament work went far beyond padding GE’s balance sheet. Over the course of his project, he noted that the way tungsten vapor condensed did not gel with existing academic theory. His subsequent exploration of this phenomenon led Langmuir to be credited with founding the field of surface chemistry. Langmuir earned himself a Nobel Prize for his efforts.
There was a symbiosis in the GE lab between Langmuir types and the Coolidge types — the latter skillset being more standard in the lab. I imagine Answer.AI will have no shortage of Coolidge-like individuals: bright, Kaggle Grandmaster-type individuals who understand academic theory but whose specialty is in expertly applying their craft in dirty, practical situations. Someone like Jeremy Howard will likely have great intuition about how to utilize these individuals. The GE playbook — with its “long leash within a narrow fence” principles — can help Answer.AI think through how to deploy basic researchers in its operations
Langmuir’s career at the GE Research Lab provides a clear roadmap for how to optimally leverage a basic researcher’s energies in an applied context. Langmuir getting paid to investigate any anomalies would likely have satisfied his curiosity. However, it was his investigation of the right anomalies that made this a beneficial arrangement for GE Research.
In general, there is a time and place to apply insights from either Edison’s playbook or GE’s. The maturity of a given research field or technology area has a strong hand in dictating which set of principles is more applicable. Edison came first and had to shoulder the burden of developing an extensive technical system to power the “killer app” that was his bulb. GE Research had the benefit of working on an existing technology area with moderately developed science and existing user technology (thanks to Edison), but the technology still needed a lot of work to become reliable and economical.
A lab can simultaneously employ both playbooks. Even most of Edison’s projects were modest in relation to his lighting work. When inventing for existing fields, such as telephony, Edison contained his inventive streak to working within existing technical systems. He knew nobody would rebuild entirely new telephone infrastructure just because the young inventor had rigged up a moderately improved but completely different version. When adding to Bell’s telephone, he simply invented a carbon transmitter that could plug directly into the system. This device made voices come through much clearer. That was it: one gadget that cleanly plugged into the existing system. Technologies like these may not be as earth-shattering as Edison’s lighting system, but they were still enough to make him a world-famous inventor in his own time.
It was about impact. In optimizing impact, I thoroughly suspect Answer.AI to make great use of the playbooks of both of these small industrial research giants.
Learning From Other Historically Great Industrial R&D Labs
I’d now like to highlight applicable lessons from other research operations covered on this Substack. I’ll cover the orgs in no particular order.
Striking the Balance of BBN and CMU’s Autonomous Vehicle Group
FreakTakes recently covered two historically great DARPA contractors who expertly balanced the competing pulls of project novelty and deployable technology. The first was Bolt, Beranek, and Newman (BBN), the contractor primarily responsible for the ARPAnet. The second was Carnegie Mellon’s autonomous vehicle groups.
BBN embodied what it meant to be a “middle ground between academia and the commercial world.” The firm was initially set up by MIT acoustics professors to pursue their contracting work more ambitiously. In its early decades, the firm gradually expanded its contracting efforts into the computing space, initially under the leadership of BBN VP J.C.R. Licklider. BBN soon became a common home for the best researchers in Cambridge, abandoning their academic positions to work for BBN. The firm's growing reputation even earned the monicker the “third university of Cambridge.”
The firm’s revenue was primarily sourced from research contracts given out by orgs like DARPA, research grantmakers, and aerospace firms. BBN’s positioning was somewhat unique; when compared to industry, the firm emphasized novelty and cutting-edge technology work. This insistence on novelty helped the firm recruit individuals who felt a bit too talented to waste away working on derivative projects at Westinghouse. When compared to academia, BBN emphasized working on real technology that people would use in the near term. J.C.R. Licklider is just one prominent example of an individual who left a tenureship at MIT to work on more useful technology down the road at BBN. Leveraging this positioning, the firm was able to recruit the best talent.
BBN also provided its most talented individuals latitude to ply their minds broadly. Many projects at BBN showcased the extreme potential of small teams of talented individuals with broad technical knowledge. Only eight BBNers were primarily responsible for pushing the early ARPAnet into existence. The size of the team was no accident; Frank Heart, the engineering lead of the project, described why he preferred a team of this size in his oral history:
I tend to believe important things get done by small groups of people who all know all about the whole project. That is, in those days all the software people knew something about hardware, and all the hardware people programmed. It wasn't a group of unconnected people. It was a set of people who all knew a lot about the whole project. I consider that pretty important in anything very big. So I suppose if you call it a management style, that would be something I'd state. I think also that they were a very, very unusually talented group. I think things tend to get done best by small groups of very, very good people — if you can possibly manage that. You can't always manage it. So if you again want to call it a management style, it is to get the very, very best people and in small numbers, so they can all know what they're all doing.
Lockheed Skunk Works legend “Kelly” Johnson also held quite similar beliefs when putting together teams to build experimental aircraft. Particularly in the early stages of novel projects, there is a strong case for keeping things small, with specialists who understand the fields that touch theirs. To me, Jeremy’s belief in small teams seems well-validated by technical history.
BBN demonstrates the ideal case of a research firm that wholly embraces technical novelty. CMU can be thought of as the flipside of that coin: a university that wholly embraced systems-building and used firm-like management practices to do so. The highlight of CMU’s later-1900s systems work was its autonomous vehicle projects. The academic group staffed itself with researchers responsible for technical integration and management-style work to effectively carry out novel technological systems building.
Similar to BBN, CMU’s positioning was differentiated from both industry and academia. This fact became very clear as DARPA’s mid-1980s autonomous vehicle work progressed. CMU was seemingly the only contractor excited about technical novelty and systems integration. Martin Marietta — the defense prime in charge of DARPA’s Autonomous Land Vehicle — obsessed over ways to hit DARPA’s demo benchmarks while using unambitious, dated technologies. Simultaneously, the academic vision research groups cared more about using the camera data to write papers than helping directly contribute to building a functional driving system. CMU was the only contractor involved in the project who truly cared about building a novel, functional system. DARPA eventually recognized this and gave them ample funds to build successive generations of autonomous vehicles. The rest was history.
CMU carried out this work with a management structure that was more firm-like than most academic labs. For example, Chuck Thorpe did project management-style work for the team with firm-like incentives — he was a researcher promoted based on vehicle performance, not his h-index. While this was a firm-like position, the group also had academic-style positions. Its use of graduate students on the projects is one prominent example. Each student on the project had to own a piece of the project that was all their own and could be written up as a thesis.
(I explore how the team mitigated the risks of theses not panning out in my interview with Chuck Thorpe.)
These academic incentives partially enabled the CMU team to continually innovate. Oftentimes these students’ theses perfectly plugged into existing systems, such as a thesis on reducing the processing time of an existing sensor’s data from 15 minutes to 90 seconds. But on the most extreme occasion, in 1988, this incentive structure led a grad student named Dean Pomerleau to successfully train a neural net to steer the vehicle. In that particular case, the requirement to allow each grad student to try something new changed the world.
Answer.AI similarly cares about deep technical novelty and building deployable technology. As such, Answer.AI could benefit from emulating BBN and CMU’s strategies to balance the two. The success of BBN and CMU should hopefully embolden Answer.AI’s founders to trust in the priorities they have set. This balance of goals is uncommon today, but orgs from history have expertly balanced the two to world-changing effect.
With that said, time elapsing without world-changing results might be unnerving. This ambiguity is partially what pushed academia to rely on near-term outcome variables that incentivize the incremental. Fear of wasting time and money is real. The Answer.AI founders would surely like some way to ensure that they are spending theirs on good problems. To deal with that, I think the Bell Labs’ approach to problem selection has a lot to add to the approaches I’ve already covered.
The Bell Labs Approach to Problem Selection
Bell Labs’ management of researchers in its golden era is famous — and it should be. However, Bell researchers were not left to their own devices to pursue whatever they wished, despite what many think. Bell managed their researchers with an approach similar to the “long leash within a narrow fence” approach of GE — which one long-time Bell chemist called “circumscribed freedom.” The most effective tool it used to do this was its corp of excellent systems engineers.
Bell had an expansive product line with massive scale — even more so than GE. Even modest improvements from the research team could have outsized returns. This, of course, is not the case with Answer.AI. However, Bell’s use of systems engineers can still be extremely instructive to Answer.AI, even if Answer.AI may deploy them in different ways.
As I covered extensively on FreakTakes, Bell’s systems engineers often combined several knowledge bases to expose the right researchers to the problems that most needed solving. Within one mind systems engineers often combined STEM backgrounds, knowledge of the nitty-gritty details of Bell’s manufacturing, an understanding of Bell’s implementation problems, detailed knowledge of Ma Bell’s expenses, and familiarity with the researchers at Bell Labs.
Bell knew these systems engineers were a massive part of their secret sauce, ensuring Labs deployed its limited resources on the right kinds of problems with sufficient upside. I wrote in the conclusion of my Bell Labs piece:
Finding a problem in these systems is not so hard for those familiar with the systems. That’s why many researchers and engineers do not feel the need to bring in help. But finding a set of good problems is not finding the best problems. Finding the best problems is a profession in and of itself. A systems engineer is worth it when, under the right scrutiny, it might turn out that the best problem is 10X as financially valuable, does 50X the social good, or is 2X as likely to work as just some run-of-the-mill good problem.
My prior piece delves deeper into the specific problems towards which Bell’s systems engineers led Bell’s researchers. For now, suffice it to say that I think that any new applied science org that can dedicate an (ideally full-time) individual to doing the work of a systems engineer should strongly consider it.
Of course, these systems engineers would need somewhat clear marching orders on what sorts of technologies they should be exploring. Nowadays, many existing roles in industry and academia train individuals to equate revenue or potential citation count with impact. Answer.AI will not be satisfied with these metrics as proxies of impact, and they shouldn’t be. What to direct these systems engineers toward instead should surely be up to Jeremy and Eric.
As a firm that pursued novelty and was not attached to a large industrial operation, BBN might be an interesting source of inspiration. Several of its hallmark projects started with systems-engineer-style contributions. Three examples are DARPA PM Larry Roberts putting out the ARPAnet contract, J.C.R. Licklider’s visionary Libraries of the Future Project, and BBNer Jordan Baruch’s early-1960s pitch to the NIH on a system to build a computer system to facilitate modern hospital operations. All three project initiators had Bell systems-engineer-like exposure to the people and problems of their field — technical backgrounds, regularly spoke with the best academic researchers, knew modern industry’s issues, were able to project the costs and complications of potential projects, etc.
Few have used systems engineers as effectively as Bell Labs, with Bell-style goals, since the great lab was broken up. It would be amazing to see a lab like Answer.AI commit significant staff time to this purpose.
The Cautionary Tale of Thinking Machines Corporation’s Funding
The case of the Thinking Machines Corporation (TMC) is not as directly instructive as the examples above, but TMC made one key mistake that makes it worth mentioning. TMC put itself in the unfortunate position of raising some of its funds from investors whose incentives were not aligned with theirs.
For those who don’t know, TMC was a complete failure as a commercial firm. As a result of its bankruptcy, many write the firm off as a holistic failure. But the firm did accomplish many of the technical goals it set out to achieve. Since the company was conceived with technological goals in mind, rather than a specific market, this was no small feat. The company was founded by Danny Hillis, a PhD from Marvin Minsky’s lab at MIT. Through TMC, he sought to build the machine he conceptualized in his graduate thesis: a truly parallel computer to improve the capacity of all scientists. The young company recruited the best researchers — including scoring Richard Feynman as its “intern” for several summers — and achieved many technical goals that helped pave the way for the modern field of parallel computing. Jeremy, who knows far more about the technical aspects of parallel computing than I do, sang TMC’s technical praises in our first conversation. He emphasized how shocking it was that TMC seemed to be the first to employ so many methods the field still uses today.
However, TMC’s high-powered team and great technical work were not enough to overcome their management follies. The firm spent money as if its financial standing was in line with its technical reputation, which it was not. In retrospect, some of these management decisions — such as a comically expensive long-term lease — could have been avoided without modifying the company’s general approach. However, on a deeper level, there was a dissonance between the company’s technical goals and the funding it raised.
TMC’s two major funders were DARPA’s computing office and private investors. The goals of Hillis and his top-flight technical staff were only aligned with DARPA’s goals. In looking to build the technically most ambitious parallel computer possible, DARPA funding was ideal. The DARPA computing office also felt that TMC’s work was progressing exceptionally well for most of its early years. However, the level of enthusiasm Hillis and the technical staff had for building a machine for science did not bleed into the most profitable areas — like deploying the machine on banking databases or managing logistics for Walmart. As time wore on, it seems that pressures were beginning to mount for TMC to pursue work more in line with those areas. It seems highly likely that TMC would have had to disappoint one of its funders sooner or later, even if it spent funds more wisely in its early years.
Had TMC just raised funds from DARPA and spent them much more modestly, the company might still exist today; it may have even earned a reputation beyond that of NVIDIA.
Copying mid-20th C. Industrial R&D Models is Hard for Incumbents
The middle 20th Century saw both the rise and fall of ambitious American industrial R&D labs. It is not just nostalgia that makes modern researchers look back on these labs with fondness. Not long after these labs were formed, it was becoming clear to many top researchers that the model was special.
In 1927, Karl Compton wrote a prescient letter to Science that praised these labs. At the time of writing the letter, Compton was the head of Princeton’s Physics Department and a part-time GE contractor. His letter asserts that these organizations were doing some of the best science in the country, even though the top universities often had the best men. A portion of the letter, in which Compton praises the labs’ management of scientific projects, reads:
This has been strikingly demonstrated in some of the big industrial research laboratories, from which the output has greatly exceeded the individual capacities of the research workers.
Compton believed there were many lessons that university departments should steal from these exceptional industrial labs. The first was the need to specialize when building a portfolio of researchers and projects. Why should every department attempt to loosely approximate the makeup of researchers and research questions in the field as a whole? No company would ever do such a thing. He believed that, “Such a policy dissipates effort, and if every institution followed it we should have the spectacle of a great many universities all very much alike and all with struggling, mediocre departments.”
On an organizational level, Compton believed the equilibrium of N autonomous professors with N separate budgets and a few grad students in their lab under their control was just not efficient for most projects. It would be silly for all departments to function that way. In the letter, Compton proposes what I’ve taken to calling a “Compton Model” research department. This model is far more structured than a department of mostly autonomous professors doing ad-hoc research with their own separate funds. Compton describes it as follows:
There is another direction in which more effective organization is possible within the universities themselves! Departments of a somewhat more flexible nature than those to which we are accustomed and which could, more than now, be built around one or two outstanding men in the department, could give these men an opportunity for organization and concentration of effort which is now rarely possible.
Another way to describe this model is as a “fund department heads, not projects” model. The model allows one or two individuals to largely shape the research vision, hiring, project selection, capital purchasing, etc. of an entire department at once. In addition, it allows these individuals the latitude to replace salaries spent on additional professors or grad students with full-time engineers or discretionary capital expenses as needed.
The model makes perfect sense. However, it’s remarkably difficult to make happen at a real university. Compton couldn’t succeed in doing so when he took over as MIT President. The existing stakeholders and structures are just too hard to shift in this direction. Absent some special circumstance, most university administrators wouldn’t find the idea even worth considering. However, CMU’s President Cyert was able to build a department that loosely resembled a Compton Model department. And its results were exceptional! But this was the exception. Pivoting an existing department to run like a Compton Model department has proven infeasible in almost all cases.2
The great, old models of managing labs didn’t disappear from industry because they lacked scientific merit. These labs largely began to disappear in the 1970s and 1980s. The 1970s saw a deep recession, which usually hurt R&D budgets. In the 1980s, new corporate management trends surfaced that led to companies being managed more myopically than ever before or since. These, along with other non-scientific trends, were largely responsible for the labs going away when they did. With them, the operational know-how that had slowly been built up throughout the century dissipated. Now, to learn how these orgs operated, one must read oral histories or talk to now-retired engineers.
The first thing these retired engineers will often tell you is that we should bring the old models back! Bell’s long-time researchers had confidence in the Bell model even as a late-1900s court case ended Bell Labs as we know it. In the 1920s, Compton dreamt of copying much of the GE model to use within a university. One should not be afraid to put their time and resources behind bringing these models back. Great engineers largely maintained confidence in these models from their inception until they faded into the background.
Many in academia know it should operate more like BBN, CMU, or GE research. They just can’t change the structures to make it happen. Many R&D leaders know that their company should think more in 10-20-year time horizons when planning research expenditures, but shareholders and shareholder-wary CEOs often do not find this view actionable. Answer.AI can be free from all this. The org can apply the great old models in a new era on new technology.
(Ries being a cofounder of Answer.AI should be comforting. Ries has railed against the types of myopic management trends that make activities like maintaining an expansive industrial R&D lab difficult for firms. He founded the young Long Term Stock Exchange, in part, to help mitigate issues like this.)
Conclusion: Running Your Own Races
It’s up to Answer.AI’s two-headed management team to stick to the organization’s comparative advantages. There is no need to race academic CS researchers or corporate R&D departments in races those two groups feel incentivized to run. Corporate R&D, NSF-funded CS researchers, and AGI-focused labs all have areas in which they clearly will and won’t operate. And those areas don’t come close to covering 100% of the good ideas somebody should obviously be working on. Answer.AI is free to run its own races, uncontested.
Many will feel the org is using an untested model. However, Answer.AI’s founders — like me — believe that this model is proven, but has simply gone away for a while. So, shouldering this “organizational risk” that others are seemingly not willing to do, they have the chance to work on problems without much competition. If their $10 million experiment works, it has the chance to spark a rush of emboldened researchers and engineers to found small research firms, leveraging the models of the once-great dragons of American industrial R&D.
I wish Jeremy and Eric luck in the early stages of their mission. To any researchers and engineers — across all areas — reading this and wishing a BBN, CMU, TMC, or Answer.AI existed in your area, please reach out to me on Twitter. I’d love to see if there’s anything I can do to help.
Thanks for reading:)
Of course, Edison’s lab also worked on more modest projects and often helped make improvements in related manufacturing operations. But the labs lighting work is the reason the lab earned its name in the history books to the extreme degree it did.
Even in the case of the ILLIAC IV — where the firm-like ILLIAC project was somewhat separate from the other professors at UIUC and received its own resources — the non-ILLIAC professors felt the project should be moved outside of their department.
Wow this looks awesome, working at another non profit lab and being interested in the idea of leading my own at one point.
I wish there was an audio from you and Jeremy’s voice 😃