Fortunately, there is a narrower (albeit more delicate) path, as long-time FreakTakes readers know: BBNs.

Originally named after an ARPAnet contractor named Bolt, Beranek & Newman, a BBN is just a funding mechanism for scientists to work on ambitious ideas. Unlike academia, BBNs focus on applied problems. Unlike VCs, BBNs are not concerned with making billions of dollars in the next decade. And unlike FROs, BBNs can launch quickly — often with a few hundred thousand dollars — and extend their runway by winning grants and contracts. BBNs and FROs can share essentially identical technical ambitions, so the distinction is mainly funding. FROs begin with a large upfront endowment ($10–100M) that lets them pursue a big goal from day one, whereas BBNs pursue the same kinds of goals but more steadily.

A few weeks ago, when Eric Gilliam and I sat down to lunch, I riffed on some BBN-shaped ideas for biotechnology. He asked me to write them down, and I agreed. There were two reasons for my doing so: First, I think more people ought to know what a BBN-shaped idea is; and second, ideas are a limiting ingredient in launching more BBNs.

In this short article, I riff on three biotechnology ideas which might be suited to a BBN: “Flowers by Design,” “A Biosensor for Everything,” and “Proteins for Pennies.” Readers may reject or disagree with elements of these ideas, but my hope is that, by writing them out explicitly, people will at least be inspired to start BBNs.

Share

Subscribe now

Eric is in Boston through March 27th, and he’s eager to talk to anyone with interesting BBN ideas. If you’re around, reach out at gilliam@renphil.org.

Three Quick Ideas

Flowers by Design

The first idea, “Flowers by Design,” is not my own; there is already a small yet serious group of people using genetic engineering, pathogens, and other molecular tools to create entirely new types of flowers. (I wrote about one such flower designer for Asimov Press.)

These designers, all of whom are also plant biologists, make engineered flowers with unique patterns, colors, and shapes. They tend to be brilliant not only because they are adding beauty to the universe (a goal one should not scoff at), but also because, by mixing these methods together, they are making important discoveries about how plants actually work. The “big” goal of a flower design BBN might be to develop a general theory of plant programming, such that one can control the traits and growth patterns of a wide array of plant species.

Any tools created to engineer flowers would also unlock capabilities in food crops. Flower designers work across dozens of species by necessity; far more than the handful of model organisms that academic labs use. Also, the DNA delivery methods used to engineer a dahlia or a morning glory are the same methods needed for sorghum or cassava, two crops where transformation efficiency is currently so poor that most genetic engineering efforts fail.

The same transcription factor proteins that control flower structure — petal number, organ boundaries, and symmetry — are also closely related to the genes which determine grain size, seed coat thickness, and fruit morphology in food crops. Engineering flowers is thus a way to create beauty in the universe while discovering the developmental and genetic principles that matter for food yield.

Unfortunately, flower design does not easily fit into existing career paths. It is not an obvious VC play because it can take several years to circumvent regulatory barriers alone, and returns often take more than a decade. (The Juliet Rose took 15 years to create, at a cost of $3 million, but is profitable today.) If any genetic engineering methods are used to make a flower, scientists must abide by USDA regulations overseeing how that flower is shipped across state lines.

More important than regulations, though, is the simple fact that everyone I’ve met who calls themselves a “flower designer” is extremely idealistic; almost allergic to business. They see themselves as artists and scientists first and foremost, and they don’t want to spend several years making one plant just to sell it repeatedly. Instead, they want to make hundreds of different flowers, with thousands of bespoke designs, and this takes a lot of time and money.

This isn’t an obvious fit for academia, either. The goal of flower design is to make beautiful objects, rather than publish papers. It also isn’t an obvious “pure” philanthropy play, though philanthropy would be better suited to this idea than academia or business. It just isn’t clear, to me, that a wealthy person would give $10 million to a person making flowers when that money could instead go toward curing cancer or something with more “social capital.”

Fortunately, there are many contracts and grants available for flower design, which could be used to bootstrap a BBN. Japan funded an effort to make a genetically-engineered morning glory for the 2020 Olympic Games, though the project later fell through. (Japan, in fact, has one of the largest flower industries in the world, commercializing about 40,000 new varieties each year.) A single USDA SBIR Phase II grant (~$2M over two years), paired with one or two annual contracts from plant biotech companies interested in new ways to engineer traits in flowers (typically $100–300K each), could plausibly sustain a small team of three to five people indefinitely.

Biosensor for Anything

My second idea is called a “Biosensor for Anything.”

Biology has evolved, over billions of years, to sense millions of unique molecules. Our noses can differentiate two molecules differing by a single atom, or even mirror images of the same molecule (S-carvone smells like spearmint; R-carvone smells like caraway seeds.) And we don’t only sense molecules, either, but also other ‘forces.’ Some birds migrate thousands of miles by sensing magnetic fields, for example, and proteins inside cells can sense individual metal atoms, like zinc or iron.

I suspect there is a BBN-shaped idea around using biotechnology — especially engineered cells or AI-designed proteins — to sense a wider array of molecules and forces. (There is a related ARIA program focused on building so-called “Hypersensory Intelligence.”) Whole-cell biosensors can be created using genetic circuits, where one protein binds to a molecule and a second protein emits an output, such as fluorescence or a color change, in response. A platform technology BBN, focused on building tools to make virtually any biosensor, would help us respond to outbreaks more quickly, or create cheaper diagnostic tests. (About 500,000 children in the U.S. have lead poisoning, for example, and the main way to test for it is with a $5,000 device.)

Specifically, I’m envisioning a BBN that systematically collects large datasets on protein sensors — the biological molecules that detect a signal — and the output modules that report what they’ve found, perhaps by emitting light or changing color. These datasets would be used to validate which sensors work reliably across many different cell types and experimental conditions, and to build predictive models that help scientists design better sensors without having to test every possible combination by hand. The most promising sensors would then be put through field trials in the messy, real-world environments where they’d actually be deployed: sand, soil, water, blood, and so on.

A few years ago, there was lots of excitement about paper-based diagnostics. The basic idea was that scientists could engineer a biosensor in living cells, and then kill those cells and extract their “juices,” including the synthetic DNA. This “juice” could then be coated onto a piece of paper and freeze-dried. When scientists added liquid back to the paper, the genetic circuit would reactivate and change color if the molecule being sensed was present. These paper biosensors cost pennies to make, yet I’m not aware of any companies that have successfully commercialized them, either because the paper strips were not accurate or reliable enough for clinical diagnostics or because — in places where accuracy matters less — the amount they could be sold for was too small to generate large returns.

This idea, much like flower design, isn’t obviously suited for pure academia or philanthropy. The work requires deep expertise across multiple disciplines, including computational protein design, genetic circuit engineering, and materials science. One lab can’t do all of this. And even though philanthropists might fund the creation of a specific diagnostic test for a narrow disease, they’re less likely to fund a much broader effort to make “biosensors for anything.”

That’s why this feels like a BBN-shaped idea to me. A small team could win contracts and grants to build biosensors for underrated diseases, using that money to engineer proteins and prototype devices. Domain-specific contracts — from funders like Arnold Foundation on lead, or from the EPA on environmental monitoring — could cover much of the operating cost of a BBN in this space, but the group would also need about $300–500K in unrestricted funding each year to keep doing R&D and make sure they don’t drift down a narrow diagnostic path. Over time, the BBN would use what they learn to build better reporters, more stable freeze-dried systems, and also better computational design tools to speed up prototyping.

Proteins for Pennies

A third idea, and my personal favorite, is “Proteins for Pennies.” The overarching aim here is to make a “protein fabricator” such that scientists can create any protein, of any amino acid sequence, for a few pennies; ideally within a few hours. We are nowhere near achieving this, but doing so would not require breaking any laws of physics. Cells “eat” nutrients, like sugars and nitrogen, and turn them into useful biomolecules. Indeed, a single cell synthesizes tens of thousands of proteins simultaneously, using only the atoms repurposed from its environment. Why can’t we do the same, but synthetically?

There are currently two main ways to make proteins in the laboratory. The first option is to order a DNA sequence encoding a protein of interest. Each nucleotide costs about $0.07 to order from a DNA supplier. Given that an average protein has about 300 amino acids, this means scientists must order a gene with at least 900 nucleotides (three “letters” of DNA encode one amino acid) to make such a protein. In this scenario, the DNA sequence alone costs at least $63. And even after the DNA sequence is made, scientists must then clone it into a microbe, grow those microbes in large batches, and purify the protein. This takes time and money.

The second option is chemical synthesis, where chemists click amino acids together, piece by piece, until the full protein is completed. The problem with this approach is that each step has a particular efficiency — say 99% — and a single error is enough to ruin the entire molecule. (For a theoretical protein with 300 amino acids, a per-step accuracy of 99% would yield just 5% of proteins with a perfect sequence.)

A BBN could work towards lowering the costs of protein synthesis by 100x. Protein-based therapeutics, some of which are currently too expensive to manufacture for rare diseases, could be made on-demand. The cost of protein design would also fall sharply, since wet-lab testing of computer-generated designs is the main cost — and most time-intensive part — of the whole process.

A BBN focused on protein costs could begin by miniaturizing and automating existing protein synthesis technologies. With enough scale and multiplexing, perhaps costs could fall by 3-5x. A recent paper from the Baker lab, for example, describes a semi-automated workflow for purifying and characterizing hundreds of designed proteins per day using standard laboratory equipment. This reduced the cost of testing each protein design to just five dollars per construct. Still, my suspicion is that radical price reductions will require a fundamentally new technology; something with near-100% accuracy (like living cells have) without requiring a DNA template to build the proteins.

Previously, I wrote about one “Protein Printer” idea together with Julian Englert, CEO of AdaptyvBio, a protein testing company. Englert’s idea was to build a circular loop of RNA containing a codon for each amino acid, attach a ribosome to it, and then move that ribosome back-and-forth to each codon to make a particular protein. Perhaps this could be done using flashes of light, where a “blue” wavelength triggers the ribosome to move one codon forward, and a different wavelength triggers it to move back one codon; akin to a rotary telephone. In this way, scientists would not need to synthesize an original DNA template for each protein; they could just use these engineered ribosomes and circular loops of RNA to make any protein using light. But this is not the only idea to make a “protein printer.”

Grants could easily be used to bootstrap this BBN. Many funders are excited about the intersection of AI and biology and, again, protein synthesis costs are a major bottleneck for testing computationally-designed proteins. Therefore, the cheaper we can make proteins, the larger the datasets we can collect for a given amount of funding, and the faster we can improve AI models.

ARIA recently announced a £50M Universal Fabricators program, specifically to develop scalable protein-based manufacturing. If a team of three to four people could win some grants to build a fence around R&D efforts, they could then go chase after contracts for more narrow efforts (like bespoke protein manufacturing for therapeutics companies.)

Everything in this essay is me shooting spitballs against the proverbial glass window. But I hope this article inspires others to think of BBN-shaped ideas in their own fields. Which ideas would move your field forward in a meaningful way, yet are not well-suited to academia, VC, or large-scale philanthropy? Write them down!

Niko McCarty is a fellow at Astera Institute and a founding editor of Asimov Press.

Ivan’s program has some very BBN-shaped pieces of work, and he believes some of his most ambitious applicants might see his call for proposals as their chance to launch a BBN of their own. If that sounds like you, and you can solve the problems Ivan outlines below, reach out to him here. (To learn more about “BBNs,” see this prior FreakTakes post.)

As a bit of context — for those that did not see my interview with ARIA’s founding CEO — ARIA PDs are empowered to fund brand new R&D orgs into existence if it suits their program’s needs. The org covered in the most recent FreakTakes post, Syntato, is a case in point. ARIA’s Programmable Plants work needed a group to build tools to transgenically gene-edit crops. Using its ARIA contract to do just that, Syntato was formed.

Ivan’s program may offer similar opportunities. In our discussions, Ivan was particularly excited about the idea that, at their best, BBNs can be technology-maximizing firms. I hope that a few of your pitches live up to that bar.

Share

Subscribe now

Two quick notes.

1. I initially intended this series of BBN-related guest posts to be a weekly series. That was overly ambitious. I get over-eager sometimes, sue me. The new cadence will be two or three pieces a month.

2. In the UK, many have begun to call BBNs “Frontier Research Contractors” (FRCs). Ivan uses that terminology below.

Universal Fabricators

By: Ivan Jayapurna

Ages of human history are defined by new materials: Stone, Bronze, Iron. However, civilisation-defining materials, from steel to optical fibres, changed the world not at discovery, but only when processes were developed to make them cheap and abundant. Our physical world is still dominated by ancient materials (wood, concrete) and antiquated ones (steel, plastic) built using brute-force heat and pressure. Continued reliance on manufacturing paradigms from the Iron Age and 1900s chemical engineering has trapped us in brittle global supply chains, resource conflicts, and a tiny corner of the total possible materials design space. Despite advances in material synthesis, many desired electromagnetic, thermal, optical, and mechanical properties remain inaccessible.

Mass manufacturing with molecular precision is crucial for state-of-the-art material performance, and proteins represent a uniquely powerful toolkit to achieve this. Yet despite recent breakthroughs in protein engineering (e.g., Alphafold, de novo design, directed evolution, non-canonicals, cell-free synthesis), subsequent investment and applications have primarily been in pharmaceuticals and biocatalysis, leaving the potential of proteins in materials assembly severely underexplored. Today, most protein engineers only design drugs and enzymes. If this programme is successful, they will design next-gen materials across electronics, energy, infrastructure, and more — and proteins will become “universal fabricators.”

As an ARIA programme director, I don’t have a lab of my own. My role is to design the programme, select contractors to fund, and orchestrate them towards the programme’s North Star technical vision. As such, success is almost entirely dependent on the programme’s contractors, who will develop proteins into universal fabricators. I need people and labs that are excited to become universal fabricators. It’s possible that FRCs — or BBNs as they’re also known — could be ideally-suited to do substantial portions of work for this ambitious, engineering-heavy, and radically interdisciplinary programme.

Why FRCs

As Eric has written in a prior FreakTakes piece:

[FRCs] will tend to have ambitious goals that are too engineering-heavy, multidisciplinary, or applied for academia. And they will tend to focus on markets ill-suited to VC — either (a) because these are not billion-dollar markets, or (b) because the financial impact of the work could be massive, but will not be realized within the ~10-year life of a VC fund.

Of course, there are some select academic research groups able to overcome these barriers. Often, these labs are well-funded, highly interdisciplinary, and have worked with ARPA-style agencies before (DARPA, ARPA-H, etc.). These labs have a culture of spinning out companies, allowing for R&D to be distributed between basic scientific research in the main lab and more “engineering-heavy” or “user/product-focused” work in spinouts. However, these labs are few and far between; and where they do exist, their spinouts are often subject to the constraints of venture capital funding.

For example, it is all too common a tale that an exciting materials startup with dreams of building revolutionary platform technologies is pushed by market and investor incentives towards a niche in the pharmaceutical industry, or incremental advancements on existing systems. These are all useful, but tend to fall short of the founder’s original ambitions. For profit-maximising firms this is understandable, but once you narrow in the technical scope to only those markets that fit VC constraints, many lines of societally exciting technical inquiry are lost.

I view FRCs as technology-maximising organizations that seek contract revenue from customers not as the objective to maximize, but as fuel to develop coupled science-engineering breakthroughs. These groups should seek to do work at the edge of the impossible. For that reason, FRC-like groups are a great fit for a programme like ours.

Universal Fabricator FRCs

The majority of our £50m programme will fund a small number (<10) of teams developing proteins into universal fabricators. There are multiple different renditions of what an FRC that could successfully build a universal fabricator could look like. One variant might look something like a team made up of the following archetypes:

• Protein Engineer — Designs the building blocks and interactions on the molecular scale.

• Complex Soft Matter Expert — Manipulates short- and long-range interactions between the building blocks, such that they assemble at the right length scales and time scales.

• Inorganic Materials Mineralization Expert — Programs nucleation and growth of crystalline or amorphous inorganic matter into highly valuable functional materials.

• Process Engineer — Designs reaction environments to create “error-correcting” assembly systems that impose selective pressures for assembly through fields and flows.

No matter how they are structured and composed, these teams should be able to rapidly iterate design-test cycles — quickly going from a sequence, to the design of single molecules, to ensemble design, to field/flow-aligned assembly (liquid → slurry → solid).

To further emphasise, this all-star team is just one vision (mine) of how to design a universal fabricator FRC. We’re looking for creatives with their own takes on team composition, and imaginations that exceed our expectations to fill the white space that sits between:

• Biology and solid-state physics,

• Protein assembly + templated mineralisation (sequence → structure → function) and scalable materials manufacturing (processing → structure → performance)

If successful, I hope these universal fabricators can do the work that defines the next era of human history. I want to read proposals from any team with a vision of how to make that happen.

Protein Production FRCs and Beyond

The programme will likely benefit from other FRC-shaped groups, beyond teams building protein-programmed fabrication platforms. One big challenge that will need to be solved is scaling up protein sequence-to-function design-test cycles. Today, in silico screening and automated assaying is done efficiently at the molecular scale. Scaling this up by many orders of magnitude, to the materials scale, today, would optimistically take months. Ideally, we want to shrink this timeline to under a week, to match the current state-of-art in drug design pipelines.

An FRC focusing on advancing rapid turnaround — large quantity, bespoke, engineered protein production — would likely be one of the most important suppliers for our programme. Specifically, I’m looking for a group to develop a system where, if I give you an arbitrary protein sequence, within a week you can give me enough material to e.g. template the assembly of a magnet large enough to measure (BH)max. Today, this would likely be done by living systems, but it’s conceivable that it could be achieved by drastically improved non-living production systems (e.g., cell free, protein printer). If a group has the capacity and ambition to substantially improve the state-of-the-art, it would be a valuable organisation for us to potentially nucleate into existence.

We expect this service to be exceptionally useful to a variety of customers, beyond the needs and length of our material science programme. In the near-term, we expect that a sustainable inflow of contracts could also be sourced from markets ranging from pharmaceuticals and biocatalysts to food, fibres, and packaging. Contracts like these, in addition to the sizable needs of our own program, would allow for the steady funding required to knock down technical barriers in pursuit of ambitious long-term goals in this space.

These are just two examples of possible FRCs aligned with my programme’s needs. I’d love to read your pitches for more of them. For example, FRC proposals in an area like metrology would be very welcome. Is there an FRC-shaped group that could quickly measure defect-free incorporation of proteins into macroscopic structures?

Call To Adventure

Prior to joining ARIA, I was a researcher trapped at the interface of materials and biotechnology — the only non-healthcare company in a biotech incubator, the only biotechnology in a climate accelerator — as well as between academia and profit-maximising startups. ARIA offered me an alternative, more direct path towards achieving my North Star of ‘Manufacturing Abundance’. Through the Universal Fabricators programme, I hope to open alternative paths for others. If this piece resonated with you, let’s go on this journey together.

Share

Subscribe now

Karen is the perfect founder to present the first example of a modern BBN in this series. Syntato has both an extremely ambitious technical goal and a great customer — one that has enabled the org to be a break-even operation from day one. Beyond those attributes, Karen is an exceptionally effective writer.

While Karen’s name may be new to readers, his work might be more familiar; Karen previously co-founded Light Bio. Light Bio inserts genes from fungi into petunias to make them glow (see below). Syntato’s work builds on methods developed at Light Bio and ideas from Karen’s research group at London’s MRC Laboratory of Medical Sciences.

In today’s piece, Karen will paint a picture of where Syntato hopes to go. In reading it, I hope readers develop confidence that BBNs can swing big in the modern era while covering their own costs.

Subscribe now

Share

This piece is a part of a FreakTakes series on BBNs and the BBN Fund. To read more about the BBN Fund itself — and how great BBNs from ARPA history contributed to breakthroughs like the ARPAnet and early autonomous vehicles — see the opening piece in this series. TLDR: the BBN Fund will be a time-bound, thesis-driven fund at Renaissance Philanthropy, headed by Janelle Tam and me.

Our goal is to be for the BBN ecosystem what the best ARPA PMs are to speculative areas of R&D. As an alternative framing — to steal a new term from Nan Ransohoff’s excellent piece from a few weeks ago — we will be “General Managers” for the modern BBN experiment, directly responsible individuals dedicated to finding ways to lower the barriers to founding ambitious BBNs in the modern era. For us, success will mean paving the way for dozens of BBNs like Syntato to be founded per year. And we are actively seeking our first funders to make this happen. So if you are a funder interested in partnering with us, please reach out at eric.gilliam@renphil.org and janelle.tam@renphil.org!

With that, I’ll be out of your hair! Enjoy:)

From primitive engineering to the design of new plants.

By: Karen Sarkisyan

— How can we design a novel crop?

— Or program the behaviour of a house plant?

— Or turn a field into a super-organism that breaks the growth-defense trade-off through division of labour?

— Can we domesticate marine plants to produce food without deforestation?

Today, our ability to engineer plant traits is primitive. We are able to write one, two, and sometimes three lines of the lowest-level DNA code, and these programs are typically species-specific. None of the engineered plants on the market goes beyond the line:

      always express genes A-G

In the future, we may be able to design plant traits we need. But with the current rate of progress, this will not happen before our existing agriculture practices and need for energy and materials destroy most of the planet. (This image of Borneo deforestation, below, is one of the saddest things one can find on the internet — and there seems to be no realistic solution, except possibly an economic shift driven by biotechnology.)

— Is it possible to have a truly novel crop on the market in 10 years, by 2036?

— Probably not. Not with the current tools. And which team is even in a position to work on that?

Surprisingly, across the world, there are almost no competent teams working on ambitious plant engineering from first principles. Even the startups supposedly doing “cutting-edge” plant biotechnology almost always create yet another version of “overexpress genes A-G”. And most effort — shaped by regulations — goes into modifying existing plant genomes, one small edit at a time.

Several factors limit accessibility of deep plant re-engineering to biologists, most importantly:

1. difficulty of transforming and regenerating most plants, especially non-model cultivars,

2. lack of predictable building blocks to program plant traits,

3. immature “hardware” technology: molecular tools to build, host and update large DNA programs in plants.

Fortunately, plant transformation and regeneration (=1) has seen a lot of progress in recent years: a variety of novel tissue-culture-free (as well as tissue-culture-based) approaches have been developed [1, 2], with many studies already expanding way beyond model plants [3, 4, 5, 6]. We are certain that in the next 5-10 years, hundreds of academic groups will be trying out diverse approaches, largely removing this bottleneck without the extra help.

In contrast, predictable building blocks for programming plant traits (=2) remain scarce. Over the last decade, great work has been done standardising molecular infrastructure and creating collections of lowest-level DNA parts (promoters, genes, terminators) for different species of dicots [7, 8, 9, 10, 11] and monocots [12, 13]. However, very little work has been done at the higher level of genetic circuits and metabolic pathways.

Configuring pathways into openly accessible and easy-to-use building blocks lowers the technical “activation energy” barrier — with a non-linear, catalytic effect on their adoption. An illustration of this is the reporter RUBY which encodes biosynthesis of a red pigment: after being configured into the easy-to-use form of a single transcription unit in 2020, it is on the way to become one of the most popular reporters in plant biology [14, 15].

Let’s review a hypothetical program (on the left) and the building blocks needed to execute it (on the right). This program enables a farmer to deliver a critical weather forecast using a drone-sprayed chemical signal, helping plants survive environmental stress and maintain crop yield. To reduce costs for the farmer, the plants themselves propagate the information from the spray site throughout the field

Some of the building blocks required for such programmable traits have not been developed, despite the availability of components; others have only been shown to work as a proof of concept, but not as robust plug-and-play tools. An assessment of block compatibility and joint performance within higher-level programs has not even been attempted. There is no validated library of low-level genetic abstractions one can use to build a DNA program in plants.

Plant “hardware manipulation” technologies (=3) are at a similar place: we do not have solutions to routinely build and manipulate plant synthetic chromosomes, let alone do so at a cost compatible with iterative prototyping of large DNA programs. (This problem is the focus of Syntato’s ongoing ARIA-funded work — to create a technology to inexpensively build plant synthetic chromosomes.)

Due to GM regulations and the cost of obtaining regulatory approvals [16], there is little economic drive for enabling more complex plant programming. In the absence of such a drive, the current lack of tools and low-level genetic abstractions is unlikely to change soon: we are stuck without having the tools, despite the technology components already in place. A well-funded academic lab may attempt to work on this, but as the effort does not necessarily yield scientific novelty, but instead requires large-scale multi-year iterative refinement of genetic designs, it will be misaligned with the academic incentive structure. A startup cannot afford to do this general-purpose work, as it is too broad, expensive, and far from the market. A large seed company is unlikely to approve such spending as it’s not directly aligned with the goals of their crop development efforts. Without a focused effort for large synthesis of available technology to make good tools (“molecular hardware” & “DNA software libraries”), the activation barrier will stay high enough to prevent crop design from becoming accessible to a broad range of bioengineers.

By focusing on building tools, we aim to help move the field from the current primitive plant engineering towards the design of novel complex, adaptive traits — and eventually, just like we design computer chips today, we will design and manufacture genomes of new plants.

Contracts and grants

Syntato is the lead contractor on an ARIA-funded project to create plant synthetic chromosomes technology. It is a 3-year £4.1M+VAT contract, the budget of which is shared with collaborators and subcontractors. The contract with ARIA allowed us to buy equipment, build a small team, launch lab operations, and create a frontier screening platform for plant engineering work – all in under 9 months. This initial investment into the robotic infrastructure and technology created a basis to amplify the value of future grants and contracts: an opportunity to follow a BBN-style organisational path.

The platform we developed enables high-throughput experimentation in fully intact plant cells, with the highest reproducibility we’ve seen among plant biology transgenesis platforms. We offer two types of contracts:

• Standard screening of customer’s genetic designs on our pipeline (cheaper, simple paperwork, no custom assay development, no IP ownership).

• Tailored co-development of custom plant-based assays (more expensive, IP may be shared).

Syntato’s platform is universal and can be adapted to any plant species. It can be used for large-scale herbicide screenings, testing of AI-designed protein binders, directed evolution of proteins in plant cells, genetic circuit prototyping, and optimization of natural product biosynthesis. These services are particularly valuable for organisations without a strong plant engineering team or high-throughput plant testing capabilities. A use-case example would be a player in the horticulture industry interested in engineering a biosynthetic pathway to produce a pigment in an ornamental plant. Or a newly established startup that chooses to derisk their proof-of-concept on our platform instead of launching a fully functional lab. Or a seed company that outsources their protein engineering work instead of building that expertise in-house.

Essentially, if an organisation has a trait design project, we are excited to work with them. What makes the contract worthwhile for us is the customer's willingness to fund the upstream R&D necessary to make the desired outcome possible — allowing us to build tools and methods that get us closer to our technical vision. In that context, contracts that help move the whole field forward, such as those from philanthropic or government funders, are especially interesting.

What’s next

In Eric’s BBN Fund post, he wrote how the long-run organisational goals of BBNs will vary. One of the goals he mentioned was the possibility of using a BBNs base of contracts and grants to build up the equivalent of an academic department. This resonates with Syntato’s vision: we aim to use our contracts and grants to eventually grow into a larger research organisation, an engineering bureau that designs and creates plants. Doing this well would not just mean building up the technical equivalent of an academic department, but an entire research institute.

What would it take to reach that stage? We estimate a required volume of contracts and grants in the range of $3-7 million per year for about 10 years. We would love to work with a philanthropic funder willing to accelerate this work — please reach out to karen@syntato.uk if you have a lead. At the same time, we also feel that the customer-focused BBN path — the one which keeps us closely tied to industry’s real challenges — is the best path for Synato.

Thanks for reading:)

Karen’s LinkedIn, Twitter, and email — karen@syntato.uk .

Subscribe now

Share

Please let me know if you found this structure of today’s BBN write-up useful. The structure I personally tend to use is a bit more detailed, but in the same spirit. It goes something like: technical vision, contracts and grants, initial steps, distinct advantages, recruitment strategy, growth strategy, key winners, midterm assessment, and asks. We presented a simplified structure today, but I’m happy to adjust/expand based on reader feedback!

Since then, some version of the following paragraph has graced the top of an astonishing number of operational documents I’ve written.

A close reading of ARPA’s early history yields a key lesson: exceptional projects were usually the result of exceptional contractors. Now-famous ARPA success stories like the ARPAnet and early autonomous vehicles depended on a common shape of R&D organization — one structured, incentivized, and staffed differently than typical academic labs or VC-funded startups. I call this shape of organization a BBN-model org — named after ARPAnet contractor Bolt, Beranek & Newman (BBN). If R&D funders plan to chase more early ARPA-style outcomes, we should build more BBNs to do it.

Like FROs, BBNs pursue ambitious technical goals, ill-suited to the incentives of VC, that are too engineering-heavy or multidisciplinary for academia. Unlike FROs, BBNs primarily fund this work with a mix of contracts and applied grants, rather than a $10 to $100 million upfront fundraise.

This past year, I’ve embraced the role of a ‘field strategist’ for the BBN ecosystem.1 In this period — Stage 1 of the modern BBN experiment — I sought to verify that there was both demand for BBNs from ARPA-like funders and a supply of top researchers eager to found BBNs. Thanks to the UK’s Advanced Research + Invention Agency (ARIA), both have now been resoundingly verified. To provide just one data point in support: according to ARIA’s most recent fiscal year data, a small set of scrappy BBNs won more in ARIA funding during the year than every lab at the University of Cambridge combined. Stage 1 of the modern BBN experiment is now complete.

It is now time for Stage 2 of the experiment: building a “Convergent Research for BBNs.” The BBN Fund’s objective will be simple: seed a modern ecosystem of BBNs and work to maximize their overall technical ambition.2 If successful, we will forge a new pathway for today’s best applied, ambitious researchers to pursue ambitious R&D agendas — as Convergent Research has done with FROs. The team’s functions will fall into two basic buckets.

• Capital Deployment. Using the capital we raise for the BBN Fund, we will deploy funds to drive the creation or growth of the most promising BBNs — using a mix of financial instruments including revenue-sharing agreements, revolving loans, and undirected R&D grants.

• Field Building. Using the time of BBN Fund staff and affiliates, we will cultivate BBN founders, source new BBN customers and funders, and develop the scaffolding needed to grow the BBN ecosystem.

I will be founding the BBN Fund at Renaissance Philanthropy with Janelle Tam, formerly Convergent Research’s Head of Programs. She also built and ran Y Combinator’s Series A program, where she advised hundreds of early-stage startups on fundraising and growth. We are aiming to raise an initial philanthropic fund of $10 million to fuel this mission, and are looking for our first funders. Beyond the baseline costs of staff and field building work, for each additional $1 million, we believe we can help found or scale another ~5 BBNs. If we succeed, the J.C.R. Lickliders of our time will be able to build their life’s work within a BBN, rather than shoehorning their best ideas into academic-paper- or VC-shaped boxes.

In today’s post, I will paint a picture of our grand ambitions for the BBN ecosystem, and how the BBN Fund can get us there. The first section summarizes the BBN model for those who have not read prior FreakTakes pieces on the topic. The second section provides an overview of Stage 1 of the BBN experiment — ARIA’s embrace of BBNs and its partnership with Renaissance Philanthropy to provide grants and support to fledgling BBNs.3 I will end by presenting our vision for the BBN Fund.4

Subscribe now

Share

Today’s piece marks the start of a series. Each Friday for the next two months, I will turn the Substack over to someone in the growing BBN community, usually a founder, to write a piece on a new BBN-related topic. This will include:

• Janelle Tam, my cofounder, on the potential to unlock large pools of capital for BBNs by demonstrating their potential as an investable asset class.

• Karen Sarkisyan, a BBN founder in the UK, on his new applied crop genetics BBN.

• Henry Lee, CEO of Cultivarium — the non-model organism FRO — on the prospect of transitioning his FRO into a BBN.

• Tom Milton, CEO of Amodo Design — a BBN with the desire to fold toolmakers back into the early stage science process — on the rate at which BBNs can grow, and what drove his org’s ability to increase its largest contract by an order of magnitude four quarters in a row.5

• And more! Stay tuned.

If you are (or know people who are) interested in becoming a BBN funder, founder, or customer, reach out to us at eric.gilliam@renphil.org and janelle.tam@renphil.org.

The BBN Model

Simply put, BBNs pursue ambitious North Star technical visions with a mix of contracts and grants. The shape of technical ambition will vary. It can range from an FRO-like technical goal, such as developing methods to cheaply map mammalian brains, to building a center of excellence in an area of R&D in which academia struggles, such as bespoke biotech instrumentation or chip building.6 Some BBNs may even pursue the speculative work necessary to help establish new subfields of research, e.g. computational law. What unites BBN founders will not be the shape of their technical visions, but their capacity to fund them by solving real problems for paying customers en route to these visions.

At the original BBN, J.C.R. Licklider leveraged the model masterfully to pursue his technical vision: building a future of interactive computing. It was at Bolt, Beranek & Newman that Licklider took his first step away from his MIT professorship and towards building this future. Similarly, CMU’s 1980s autonomous vehicle teams leveraged the BBN model to tackle their own FRO-like problem, succeeding in building autonomous vehicles where DARPA’s prime contractors could not and academic computer science would not.

In pursuit of Licklider’s vision, a series of technical problems related to both UX and real-time computing needed to be knocked down. The BBN computing group did just that in the 1960s by utilizing a set of cleverly assembled contracts and grants from groups like the NIH, NASA, ARPA, consulting projects with industry, and other contract partners. The group did this while building a research culture at the firm that earned the admiration of MIT professors and affiliates, many of whom described the original BBN using phrases like “the third great university of Cambridge,” “the Cognac of the research business,” and the true “middle ground between academia and the commercial world.” Licklider and others left positions like MIT professorships to join the firm. They felt it was a far more resonant research environment for their applied, ambitious research than MIT.

BBN’s computing team was able to strike this balance between ambition and contracting because they only worked with customers whose needs were exceptionally aligned with their technical vision. One example of an aligned contract was the firm’s contract to build an NIH-funded hospital time-sharing system. This contract, which was key in funding BBN’s real-time computing progress, was ostensibly a contract to build an administrative computing system for Massachusetts General Hospital. While BBN did not necessarily care that much about building computers for doctors, the team embraced the work because it funded years of real-time computing R&D en route to its technical ambition — R&D that was then too speculative for industry and ill-suited to MIT.

Like FROs, BBNs will tend to have ambitious goals that are too engineering-heavy, multidisciplinary, or applied for academia. And they will tend to focus on markets ill-suited to VC — either (a) because these are not billion-dollar markets, or (b) because the financial impact of the work could be massive, but will not be realized within the ~10-year life of a VC fund. The latter scenario was likely the case for both the ARPAnet and autonomous vehicles.

Like FRO founders, common BBN founder archetypes will include applied-minded postdocs, research-minded engineers, defected professors, and former deep tech founders who crave the longer R&D timelines and research flexibility that the BBN model allows. BBNs won’t build and hire as fast as FROs on day one, as they will usually have much less capital. In choosing the BBN pathway, founders trade the practicality of an upfront ~$10 million FRO fundraise for a different kind of practicality: finding a set of customers and applied funders whose needs are aligned with their technical vision. Like startups, if they are able to tap into a sustainable base of funding, they can steadily expand in pursuit of their technical vision, learning as they grow. Many inspired technical visions will not have a corresponding customer base, of course. But wherever you have an ambitious researcher with vision and an aligned set of customers/funders, you might have a BBN.

Early ARPA’s best BBNs shared three distinct traits, rarely found together in R&D labs. Each:

1. Was novelty-seeking or technically ambitious, with a strong preference for projects that pushed the technological frontier forward substantially.

2. Built useful technology for paying customers. This entailed professional contract management and a willingness to focus on difficult systems engineering tasks. Many BBNs were relatively indifferent to market size, so long as they could find adequate contracts and grants to pursue their North Star technical vision

3. Used more flexible team structures than academia. When compared to academia, they more effectively hired, organized, and incentivized researchers, engineers, and other experts to collaborate on applied projects in a common-sense fashion.

With a set of aligned contracts and grants, BBN founders can often begin working towards their technical goals with only a few hundred thousand dollars — some BBNs can even be bootstrapped. This path comes with a level of ongoing financial risk not present in FROs — at least not FROs that have successfully fundraised. But fundraising for FROs is a bottleneck that few presently pass. The BBN model empowers talented, practical individuals to get in the game using a scrappier model. We believe that by lowering the barrier to entry, in five years, dozens of BBN teams could be founded each year. That means dozens of founders or technical leads setting out on the path towards their ambitious technical vision in a research environment that is both resonant with their goals and cost-offsetting.

Stated plainly: the goal of The BBN Fund is to enable the work of the next J.C.R. Licklider. And based on the exceptional results of BBNs with ARIA so far, we believe this is a realistic goal.

(For a thorough piece synthesizing the BBN model, see A Scrappy Complement to FROs: Building More BBNs.)

Stage 1: BBNs’ Success with ARIA

Stage 1 of the BBN experiment sought to verify the demand for BBNs from modern customers and the supply of talented potential founders. The UK’s Advanced Research + Invention Agency (ARIA), led by fellow metascience nerd Ilan Gur, was singularly important in running this stage of the BBN experiment. But Ilan and ARIA were not specifically trying to run this experiment. They were just trying to find some way, as Ilan might frame the goal, “to find the right people, in the right institutional environments, with the right incentives” to drive ARIA’s technical agenda forward.7

ARIA’s flexible procurement rules made this possible. Its Programme Directors (PDs) are empowered to make large bets — even multi-million dollar bets — on orgs that are young or brand-new, run by individuals ARIA feels are the best on Earth to get a job done. How flexible were these policies, exactly? As Ilan proudly shared when I interviewed him for Asimov Press, when individuals like postdocs fill out ARIA grants, ARIA has taken the unusual step of enabling them to check a box that allows them to accept the grant as new, independent research orgs. They can staff themselves with multidisciplinary researchers and engineers, negotiate contracts without involving university admins, tackle projects academia does not incentivize, pay people market rates, etc. In essence, they can spin up BBNs.

Less than two years into funding R&D projects, ARIA-funded BBNs are performing exceptionally. To contextualize how exceptionally, I’ve put together the following table using the agency’s recent fiscal year data. The table seeks to answer the question, “If the few BBNs in the ARIA ecosystem were a university, how would the ARIA funding they win compare to other UK universities?” The answer: BBNs won more ARIA contracts than every university in the UK except one, outperforming even the University of Cambridge.

Amodo Design — whose CEO will write a guest post in this series — was, on its own, neck-and-neck with the University of Oxford. And excluding Amodo, the rest of the combined set of BBNs still won more contracts than top universities like the University of Edinburgh. And more BBNs are coming. Syntato, an ARIA-funded BBN with a technical ambition to build an ARM chip design for plants — whose proposal you will read in Week 3 of this series — was not even included in this data, for example. While any specific ranking in that table should be taken with a grain of salt — contract award data is lumpier than reality — the fact that BBNs immediately find themselves in the mix with the UK’s historical research giants is a major early data point indicating that BBNs might have as big a role to play in this century as they did in the last one.

The question is no longer, “Is there any demand for BBNs from modern funders?” Stage 1 of the modern BBN Experiment has been completed. Ilan Gur and the team at ARIA — in their search to have their problems solved by aligned R&D groups — have seen to that.

Momentum is building. ARIA has even set aside a yearly pot of £400k for a program with Renaissance Philanthropy to help found and scale BBNs aligned with ARIA’s goals. Several individuals from ARPA-like agencies in other governments have also begun reaching out to discuss similar programs to serve their own needs. Most importantly, the BBN founders themselves — a boundless source of ideas — are constantly finding ways to reach new types of customers — from hedge funds to academic labs — that can preserve their technical ambition and pay the bills. The time is ripe to undertake Stage 2 of the BBN experiment: the BBN Fund.

Stage 2: The BBN Fund

The BBN Fund will act as the vehicle to orchestrate this experiment in “applied metascience.”8 Our focus will be to help build more and more ambitious BBNs. In operating this experimental, nonprofit fund, we plan to deploy funds in ways that answer questions relevant to BBN-curious funders — both nonprofit and for-profit. How technically ambitious can BBNs become with a modest amount of undirected R&D grants? Under what circumstances can investments in BBNs not only push the frontier of scientific progress, but also beat index fund returns on a risk-adjusted basis?

We plan to answer questions like this with economy and speed. The capital we raise will be used to seed BBNs, and we will learn, by doing, how to found and manage BBNs in the modern era.9 The two levers we will rely on to do this work will be the time of BBN Fund staff and capital from the BBN Fund itself. The rest of this section will summarize our approach to using both.

Capital Deployment

An exciting characteristic of BBN founders is their ability to balance practicality and grand ambition. From day one, BBN founders design their org with practical realities, such as customer sales cycles, in mind. But even with practical planning from the founder and a set of aligned customers, several financial problems hamper the creation and growth of BBNs. The BBN Fund will leverage three financial instruments to overcome these problems. Each instrument, in its own way, will enable us to leverage the practicality of the BBN model to stretch R&D funding dollars abnormally far.10

But before discussing these instruments, let’s talk about the problems. The first is temporary cash flow problems. Even if a new BBN wins some contract, it may have cash flow problems for several months (at least), as contract expenses are often front-loaded. The second problem is startup capital needs. Many BBNs will require a modest injection of funds — the equivalent of a seed round — to do the technical and operational work necessary to become self-sustaining organizations.11 The third problem is ongoing research funding needs. For many BBNs, the flexibility provided by winning something like ~20% of their budget as undirected, NSF-style R&D funds — for investigations, core technology development, etc. — can make the org ~twice as technically ambitious.

The three financial instruments the BBN Fund will experimentally deploy to help founders overcome these problems are low-or-no-interest loans, revenue-sharing investments, and undirected R&D grants. While loans and revenue-sharing agreements are peculiar instruments in the world of philanthropy, we believe they might enable us to stretch a fixed pot of capital exceptionally far. Below, I describe the promise of each instrument in more detail.

A Revolving Loan Fund. Low-or-no-interest loans with no collateral requirements will enable far more founders to spin up BBNs. Several existing BBNs had to bootstrap, including Amodo Design. That is admirable, but presents a barrier to entry for many prospective BBN founders, limiting the scalability of the model. Judiciously applied, low-interest loans can provide instant access to capital to cover the salaries, equipment, leases, and other expenses BBN founders have to pay upfront to win and execute contracts. These instruments will have the same limited downside risk for founders as VC funding. Given out as loans instead of grants, the revolving loan fund offers the opportunity to spend dollars multiple times. Any repayment above 0% will drive a higher ROI than the status quo. That being said, with prudent decision-making, most of the capital loaned out can be paid back into the fund — many BBNs asking for loans will have a high payback rate, as they will already have warm contract leads. In the case where ~80% of funds loaned out are revolved back into the fund within 18 months, we can create ~five times as many BBNs using a fixed pot of capital, compared to deploying the funds as undirected R&D grants.12

Revenue-Sharing Agreements. The BBN Fund will experiment with BBNs as an investable asset class. We will identify scenarios and strategies in which investing in BBNs can be attractive to certain private investors and philanthropic funders. In my discussions with BBN founders, revenue-sharing agreements have been the most common category of instrument proposed to facilitate these investments. And I have yet to meet a BBN founder who, for a fair price, is categorically against partnering with a for-profit investor, sharing the upside with them. To lower the barrier to engaging in these revenue-sharing agreements, the BBN Fund will design a fit-for-purpose, YC-SAFE-style instrument to facilitate these agreements. Ideally, this instrument would allow the following deal terms to be customized to match any given situation:

• Percent of Revenue Shared

• Revenue Base (i.e., is revenue calculated from overall revenue, a certain line of business, etc.)

• Sunset Clauses (e.g., no payments after 8 years, once 5x the original investment is paid back, no sunset, etc.)

• Grace Period (e.g., no repayment expected for first 18 months)

Even in cases where the returns on these agreements do not outperform index funds on a risk-adjusted basis, they can still prove to be an exceptionally cost-effective tool for philanthropists and impact investors.

Undirected R&D Grants. Many BBNs require modest levels of undirected R&D funds to effectively pursue their technical visions. These funds do not need to be the majority of their budget, but can often be in the range of ~20%. This minority of funds, and the flexibility they provide, can be vital in differentiating the work of certain BBNs from a typical, unambitious contract research organization. While it will only be a minority of the fund, when it is cost-effective, the BBN Fund should deploy undirected R&D grants to BBNs. One exciting characteristic of even this ‘free money’ is its relative cost-effectiveness when compared to undirected R&D grants to labs that don’t take on contracts. E.g. If a BBN can fund ~80% a course of R&D via contracts that other groups would have asked philanthropists or the NSF to wholly fund via undirected grants, the extra ~20% in undirected grants needed to top off their budget should come as an obvious bargain.

Through our efforts, we hope to make BBNs a legible vehicle for investment for both philanthropic funders and for-profit investors. While we are a nonprofit fund that will reinvest any revenue-sharing profits into BBNs, we hope as many types of BBNs as possible prove to be profitable. That is because one of our goals is to stimulate a BBN investment market into existence. Profitable investments from our portfolio will serve as a proof-of-concept to investors that certain types of BBNs are a profitable asset class. And in those areas in which funding BBNs carries a negative profit margin, and seek philanthropic backing, we will obsess over cost-effectiveness all the same. We aspire to become one of the most cost-effective forms of scientific philanthropy on Earth — the group that, while ruthlessly selecting for ambition, still found a way to spend each dollar multiple times.

BBN Central Office

If the goal of the BBN Fund’s capital deployment is to create and amplify the ambition of BBNs via financial investments, BBN “Central Office” will do so through the direct effort of a small group. This group — Janelle and I for now — will be the hub of BBN field building, dedicated to addressing the non-financial bottlenecks that currently limit the growth of the BBN ecosystem.

In many ways, Central Office will be looking to scale and expand on my own field strategy efforts over the past year. At Renaissance Philanthropy, wearing my BBN field strategist hat, I spent significant time on the following tasks:

• Sourcing new BBN funders and customers, helping them understand which of their problems are BBN-shaped, and how I might source a BBN founder to solve those problems.

• Cultivating new BBN founders, finding exciting founders with aligned ambitions, and helping them understand that a BBN might be an exciting vehicle in which to pursue their life’s work.

• Field Building. In addition to my ARIA work, I’ve been able to use FreakTakes as a vehicle to excite readers — some of whom inhabit important positions in the R&D ecosystem — about the possibilities of the BBN model. This has driven an increasingly impressive flow of founders and funders my way, as well as helped bring existing BBNs together under one banner, with one common vocabulary.

Through my and Renaissance Philanthropy’s field strategy efforts, we’ve already had a hand in unlocking over $10 million in funds for BBNs.13 Our goal is for every dollar spent on BBN Central Office to unlock 10X as many funds from other customers and R&D funders through similar efforts. So far, we’re on track.

Scaling the above tasks will include accomplishing key goals like ensuring that, for every BBN founder, every aligned ARPA-style PM or philanthropic grant officer on Earth is one warm introduction away, and ideally is briefed on what BBNs can do for them before calls for funding go out.14 We’re excited to find ways to tackle these problems. In addition, we will concern ourselves with a variety of other bottlenecks that repeatedly come up when talking to founders. Each could unlock an order-of-magnitude more capital for BBNs than they cost to address. A sample of these tasks includes:

• Sourcing aligned, practical mentors. While BBNs are meant to be far more ambitious than typical contract research organizations (CROs), some have CRO-like revenue streams. That’s great, if it can fund their technical vision! But there’s a problem: many PhDs from places like MIT have never met anyone who runs a CRO. As top researchers now look to build BBNs in this area, we need to find mentors to show them the ropes.

• Sourcing aligned contractors. Similarly, Central Office will establish relationships with individuals needed to make commercial sales. Some BBNs have floated the idea of selling data from instruments they build to commodity traders, to offset costs. If this proves to be a common need, we will find data brokers who can explore and facilitate these contracts.

• Unlocking NSF-style grant funding. NSF and NIH grants can prove to be reliable sources of R&D funding for BBNs. But accessing NSF-style funds is not yet as straightforward as it should be. Certain BBNs claim they cannot win NSF grants at the rate they did at universities. One advisor believes this might be a problem of know-how, and is eager to work with a few BBNs to try to increase their win rate in applying for these grants.15

One way or another, it is our job to solve these problems for BBNs.

Small Teams, Large Shadows

The biggest upside of funding BBNs is not even their cost-effectiveness, but their potential to unlock categorically different research than existing institutions can provide. The original BBN delivered the first four nodes of the ARPAnet on time and under budget, for approximately $8 million today. This was a contract that companies like IBM ‘no-bid’ because they thought it was impossible. Early CMU’s autonomous vehicle teams accomplished technical breakthroughs that DARPA’s prior prime contractor failed at with 10x their budget.

The 20th Century’s great BBNs demonstrated that in R&D, with a differentiated model, small teams can cast a large shadow. Charting their own research path with a mix of contracts and grants, BBNs have a measure of freedom to direct their research agendas towards differentiated outcomes — with no worry for what’s unpopular in academia or ill-suited to venture incentives.

For years, I’ve obsessively studied historically-great R&D operations of many shapes and sizes. Based on my experiences with BBN founders and customers this year, I believe the BBN model can be a vehicle for the right founders, in the right situations, to fund a range of these orgs into existence. This includes:

• Bootstrapping new academic departments. The BBN model can enable founders to bootstrap (what are essentially) new academic departments into existence, outside the university.16 17 18

• Funding industrial R&D labs into existence. BBNs are, in many ways, self-sustaining frontier labs. Under the right circumstances, the BBN model may prove to be a way to make a dent in the problem of funding more groups like early DeepMind into existence — without the pressure to appease investors and exit to a high bidder.19

• An alternative approach to deep tech VC. With a bit of taste, we might demonstrate that investors can reliably pick modestly profitable firms when making revenue-sharing investments into BBNs. If savvy investors can reliably identify 3x outcomes, with 20x outcomes from time-to-time, a world of possibilities opens up in which BBNs can not just beat index fund returns from a financial perspective, but fund wildly ambitious R&D projects along the way.20 (Janelle will dedicate next week’s post to painting a picture of this possibility.)

The BBN Fund is being created to run this experiment and find out what’s possible. If the model proves capable of creating any of these types of orgs, in bulk, that would be a big deal. As of now, we believe the model might be capable of doing all of the above, at least some of the time.

While the Midwesterner in me blushes to speak of grand ambitions, in philanthropy, I believe ambition is a moral duty. We will not aim small. Our ambitions, over the long run, should be measured in metrics like Turing Awards per dollar, number of BBNs doing the most interesting work in their area, or the number of entirely new industries created. If the NSF and NIH annually spend $50 billion to return ~4 Nobel prizes, we will strive to be 100x as efficient.21

These are grand goals, but they are not crazy goals. The $8 million spent on the ARPAnet seems to be less than 1% of that year’s NSF budget. CMU’s initial NavLab vehicle breakthroughs were done for ~1/1000th of that year’s NSF budget. These were exceptionally cost-effective uses of R&D dollars — maybe some of the best in the history of US public R&D — and they were done by BBNs.22 Let’s build more of them.

If you are an interested funder, customer, or aspiring BBN founder, please reach out — eric.gilliam@renphil.org and janelle.tam@renphil.org, or dm me on Twitter.

Share

Subscribe now

As always, thank you to my faithful editors, Toren Fronsdal and Tristan Wagner. They have edited this and every other FreakTakes piece. Every piece would be longer and less intelligent without them.

And thanks to Adam Marblestone,23 Tom Kalil, Janelle Tam, and Tom Milton for each —in different ways — changing how I conceptualize what might be possible with the BBN model.

If you’d like to read more about R&D groups from history that worked in BBN-like ways, or with lessons to teach modern BBNs, check out the following FreakTakes pieces. Each paints a picture of how orgs fueled by contracts can raise the ambition of the R&D ecosystem.

• “The Third University of Cambridge”: BBN and the Development of the ARPAnet

• An Interview with Chuck Thorpe on CMU: Operating an autonomous vehicle research powerhouse

• Edwin Cohn and the Harvard Blood Factory, for Asimov Press

• A Progress Studies History of Early MIT— Part 2: An Industrial Research Powerhouse

• ILLIAC IV and the Connection Machine24

• MOSIS: The 1980s DARPA ‘Silicon Broker’

• Managing Lockheed’s Skunk Works

• How did places like Bell Labs know how to ask the right questions?25

Today’s piece is an interview with Jano Costard, Head of Challenges at SPRIN-D — which some call “a German ARPA.” The interview will give readers (or listeners) a peek inside the SPRIN-D experiment, from the perspective of one of its leaders. It should be a particularly interesting listen for those who aspire to be PMs of ARPA-style organizations, procedural nerds in the R&D community, and those who simply want to understand what is going on at ARPA-like organizations around the world.

I’d love to highlight two things before diving into the interview. First, one piece of the interview that fascinated me was Jano’s obsessions with SPRIN-D’s administrative processes, such as their speed. “Fourteen days time-to-money,” the amount of time from a research group submitting a bid to receiving a decision and the funds hitting their account, is exceptionally impressive on its own. But it’s also the sort of improvement which can have fascinating knock-on effects when applied to large chunks of an R&D ecosystem. Speed and operational excellence on that level not only save existing researchers substantial amounts of time, but can also bring entirely new sets of people and organizations to the table, ones that might have little willingness to tolerate typical government R&D funders.

Second, regarding today’s interview, I had to put it together in a bit of a rush. I had a death in the family followed by a work trip. Things happen, of course. I’d love to have taken the time to read more and interview additional SPRIN-D employees to better understand SPRIN-D’s processes, scientific perspectives, contractor ecosystem, etc. But the folks at SPRIN-D and I were very excited to put this interview out in public quickly. The reason why is simple: SPRIN-D is looking for aspiring program managers, and there’s an application deadline coming up in two weeks. (More details below)

So, while I’d love to do my usual and spend 100 more hours understanding SPRIN-D before publishing a piece on them, this is meant to be a practical Substack. And, for today, what’s practical is sharing this (relatively in-depth) interview on the unfolding SPRIN-D experiment with the ambitious researchers and engineers who read FreakTakes — many of whom might want to become program managers, at some point.

I really hope that any of you who are interested will throw your hat in the ring not just for this program, but other opportunities like it. Having the curiosity to both spend a large portion of your life becoming world-class in a technical area and read something like FreakTakes in your spare time, simply out of curiosity, makes many of you precisely the kind of people places like SPRIN-D want. If you’re interested, please know there’s a good chance they’ll be interested in you.

Enjoy:)

The cleaned up transcript — courtesy of my wonderful colleague Ales and my lovely girlfriend Katherine, which I did not have any time to work on myself for the reasons listed above — is below.

Subscribe now

Share

Big if True Science Accelerator. Applications for a program which trains and mentors bright researchers and engineers on how to design and pitch effective ARPA/SPRIN-D-style programs close August 1. To learn more about the program, called BiTS, check out this link. My colleagues at RenPhil are helping run it. I will not be involved in running it, but I had a fun time writing some of the reading materials for the participants.

The Interview

Eric: Welcome to another edition of The FreakTakes Podcast. Thanks for being here. Today we have Jano Costard, who is the head of challenges at SPRIN-D, which some people call “a German DARPA.” But, of course, as with ARIA from the last FreakTakes podcast, it's a metascience experiment all its own. And [I’m] excited to talk to Jano about it today.

So, Jano, do you want to give us a bit about you first, before I jump into the questions?

Jano: Yeah, absolutely. As you mentioned, I'm head of challenges at SPRIN-D. That means I came to SPRIN-D, which is the German version of an ARPA, to build a challenge program. I'm an economist by training, and I have been dealing with competition and innovation. I now have the chance to actually run and design innovation competitions, which is really exciting for me.

Eric: And how did you fall into this, exactly? Because some people would think of an economist as a natural pick for this, and some people might find it odd. So how did it come to be?

Jano: So in my previous post, I was with a consulting body of the German government, with the Commission of Experts for Research and Innovation.
And our job was to consult the German government with regard to policies of science and technology. And that body was one of the leading actors that demanded that something like SPRIN-D should be created. And then SPRIN-D was created! And I thought it would be really interesting to move from just consulting the government to actually go to implementation, and try to do something differently.

Eric: And when there was this push internally in Germany for something like SPRIN-D, what was the exact reason? Oftentimes, there's some founding story or quick tagline that informs…the very complicated…it's the simplified form of a very complicated goal of the org. So, like, you'll hear DARPA people repeat and repeat, "to create and prevent strategic surprise." Like ARIA people talk about standards of living and growth in the UK. Is there something analogous for SPRIN-D?

Jano: So I think, in general, different cultures are motivated in different ways. One might say that the German way of being motivated is being pessimistic. [laughter] And so one of the perspectives was that Germany used to be a place where new technologies came into being that ultimately created whole industries. If you think of the automotive industries, chemistry, pharmaceuticals, and things like that. But that was like 140 years ago.

I think Germany is still living pretty well off of that. But I think we're moving in the direction where we need new technologies, where we see that those industries might not sustain our wealth or sense of living for the decades to come.

We need to find new ways of creating new industries, to create the surroundings for a wealthy society, for a healthy and happy society — as well as create solutions that tackle the grand challenges of our time. I think these two things really belong together.

But looking at how we did in that respect over the last decades, I think our record is not that stellar. If you think about the breakthroughs that Germany created over the last decades, I think not many come to mind.

Eric: And I guess for the listeners, me being history-brained and 140 years not being very long to me, I still look at Germany with a lot of awe. For example, when you read a ton of the — we'll call them the MIT founding fathers, like the early MIT professors and presidents — they all would continuously take tours of German universities, and just copiously take notes. Because in the very beginning of that institution, all they could think is, “If we could create something as good as a German technical institute, what a success that would be!”

And you see similar things with a lot of the early industrial R&D labs, like them really admiring the German chemical industry and things like that. And so it's interesting that you talk about German pessimism, because you have an optimistic mission that they've laid out for you.

Has the pessimism found its way into the structures of how this [SPRIN-D] has been laid out in some way? Because different cultures go at these things differently, and there are strengths, there are weaknesses, et cetera. But I'm just curious, in that, [with] a German ARPA, even if you wanted to create something identical, culture will seep in.
So I'm very curious about how the culture has seeped in as you've been there, from planning to where you are today.

Jano: So the motto of Rafael Laguna, our Director, is "Pessimism is a waste of time." And that really is the SPRIN-D spirit.

In that sense, it somewhat breaks with that German pessimism. He is a software entrepreneur, not by training but by practice. He’s been an investor for many years, spent loads of time in the US, and therefore went back and forth between Germany and the US quite a bit. He has been educated and shaped in his experiences quite a bit by both continents.

And I think that really shows with SPRIN-D, with how SPRIN-D operates, what our values are. We really are techno-optimists in the sense of believing and truly believing that technology can be a great source for finding solutions for the grand challenges of our time, and that that can be created from within Germany and Europe. It's not that we have a pessimism that says, "Okay, we didn't perform as well as we wanted to. And unfortunately, that's how it goes, downhill from here." I think the foundation is there to create outstanding things out of Europe.

Eric: And what is SPRIN-D in a nutshell? And also talk a little bit about how people should think about SPRIN-D programs and why you've gravitated towards the structures that you have.

Jano: Yeah, SPRIN-D's mission is to create what we in Germany call Sprunginnovation, or breakthrough innovation. And that is something that we define as innovation that really changes the lives of people for the better, in contrast to incremental innovation that really takes what we know and can do, and makes it a little bit better. And with that, we share many things with other ARPA-style institutions in the sense of, you know, we are willing and able to take a lot of technological risk in trying to achieve these great breakthroughs, even if that means that most of our projects will fail.

That is something that does not come naturally to public institutions, at least in a setting that has not been exposed that much to ARPA or DARPA-style institutions such as the US. But I think it was really, really important when SPRIN-D was set up that we gained that freedom from typical ways of doing science and innovation funding.

Eric: And can you talk about one or two failed projects that you're very happy with? SPRIN-D has done 10 challenges, something like that so far?

Jano: Yeah, I think we are at 12 challenges right now. So one of the challenges that failed in the sense of… so every challenge, typically, runs in multiple stages. It might very much differ how long those stages are, how many there are. It's completely different between different fields. One that we ran was on tissue engineering. What we wanted to see was complex tissues, artificially generated, that not only reach a certain size, but also a certain functionality at a level that has not been demonstrated before.

And we stopped that challenge after stage one, because we didn't see the progress that made us believe that in that setting we would reach the goal of that challenge. In that sense, it was not successful in reaching those milestones. But what we still saw in all those teams is that they really made good use of that money and our support in the sense of… They built stronger teams. They hired new people that eventually made them progress further, acquired other funding, and raised funding rounds. And so even though we didn't meet the goal of that challenge, I think it was really beneficial for the teams themselves and would give us another shot as a founder at SPRIN-D or other funders to really take that further again.

Eric: And could you talk a little bit about the types of teams that were participants in that challenge?

Jano: So it's typically, as so often in our challenges, it's a mixture between small startups and academic institutions. So universities, non-university research centers, and the team I was referring to, that was a small startup. They raised a seed round before. And with our money — that was really an amount of money that made us one of their primary funders in that startup. That's also something where we therefore typically have a very close relationship with the teams. Not only because we are really focusing on observing the progress that they have, but also because it is a close relationship once you are really the primary funder, and ultimately are essential for that startup either living or not living.

Eric: And I'll ask you more about financial instruments in a few rounds of questions, but for now, I'd love to stick on structure. In one of the pieces of writing you sent over to me, you talked about being agnostic — the challenge model being agnostic towards which approach is used to pursue some challenge, some goal. But some might say prizes are even more agnostic. Can we chat about that a bit? Because, practically, I think I know why you all chose the challenge model, but I'd love to give you first crack at addressing that argument.

Jano: Yeah. Just to differentiate between challenges and prizes — or sometimes it's more confusingly called “challenge prizes,” and it's completely unclear what the difference might be.

So in our case, a SPRIN-D challenge is a setup where we formulate a highly ambitious goal where we think that if it could be achieved, it would be highly impactful — a breakthrough where most people would agree that would really be a very beneficial breakthrough. But equally, many people would agree that it's currently probably impossible to actually achieve that.

And then we fund a range of different teams, with different approaches, in parallel to try to achieve that goal. Do the R&D. Do everything that's around that to achieve that goal. Whereas with a prize competition, you equally formulate a very ambitious goal, but then set up a prize, in the sense that whoever proves first that he or she reached that milestone, they will receive that cash prize, or whatever that prize might be.

The difference being that, for one, we fund teams along the way for the R&D work, and everything else that they need to have to sustain that work towards that goal. This is typically not the case in prize competitions.

Another difference being that prize competitions are typically more open to a wider set of contestants, in the sense of potentially not limiting entry at all. So, anyone could submit any kind of solution and proof of that solution to claim that prize. Whereas we start with a call, and we might select an initial set of teams. And then we continue to work with that set of teams.

And I think there is certainly something that speaks to both approaches. Looking at our settings, the reason why we fund R&D along the way primarily is because there is a lack of funders that would support that R&D until they reach that milestone for the prize. So I think we need to…

Eric: I remember one time I was talking to somebody who did — it was an economist who did market shaping. It was in the life sciences, where a lot of your contestants would be the Mercks, the Pfizers of the world, with a big war chest.

And when I raised the issue of — I think a lot about small R&D groups that have difficulty getting funded but are very resonant, like BBNs, which I've written a lot about a lot on the Substack. And the economist kind of just looked at me weird and said, “Why would you want to condition on or incentivize something that's not the outcome of note?” Which, in a purely theoretical way, I guess is true, if you have a big enough war chest or you make the outcome prize big enough where other people will come. But practically, there are a lot of times where I think of valleys of death as just areas where there are not a lot of incentives or funding. So, to me, it makes a lot more practical sense if you're dealing with small, resource-constrained R&D groups to take this [challenge] approach.

Jano: And I'm very sympathetic to that prize idea in general. But if you talk about the Mercks and Pfizers of this world, you would need to have a substantial prize to make them move. And even then, I have been working on market shaping in drug development, on infectious disease antiviral therapeutics, in particular. And that is an area where I think market shaping, for example, in the form of advanced market commitments, is really something that is necessary to make sure that new technologies actually reach the market.

But in the conversations I had with pharma, for example, it became very clear that even if there is a cash prize, however large it might be, that would not be sufficient. Because what they are looking for are natural markets, or natural markets that might occur due to those instruments. Because any kind of cash prize would need to compete with the natural markets they are in: in oncology, in other blockbusters that give them way more return on investment. And competing against that with a one-time payment will be very hard.

Eric: Yeah. And can you talk about how you think about follow-on funders when you're coming up with challenges? Like SPRIN-D can't fund these things forever. Usually, there's some end state, whether it's a VC-funded company or whatever. So how do you give those instructions, or what line do you toe internally when it comes to follow-on funders?

Jano: So I think the most important thing is that we think about that, right from the start. And within SPRIN-D, you know, we don't have a transfer department. We don't do our R&D funding, and then the project, and then push it to the next building, or something, to other folks who work on transfer.

It goes so far that our challenges, and the goals of our challenges, are shaped by how we might think about commercialization. One example of that is we have been active in a space that is called carbon dioxide removal. That is about removing CO2 from the atmosphere.

That is something that — from all these climate institutions — we know we need to do if we want to reach our climate goals. From an economist's perspective, it's a difficult thing to do, because removing CO2 has large societal value, but pretty much no private value.

So if you are a company like Climeworks, working in Iceland and pumping CO2 underground that mineralizes there in the basalt, there is no private value for a company to do that. They don't have a product that they could sell. So, currently, that is very much sustained by voluntary contributions by industry, but that is not a condition that is ultimately scalable at a global level.

So what we tried to do, and funded in that challenge, were approaches that remove CO2 from the atmosphere, and do that in close relation to existing product markets. For example — including that carbon in long-lasting and valuable products. Think about carbon-negative concrete, for example. Concrete has an established product market. It is one of the largest flows of man-made material on earth. And if you were able to develop carbon-negative concrete, it would not only avoid emissions, but due to that negative effect, it would essentially remove carbon from the atmosphere, in principle, the more you build.

And that is where we start. And then it is something where we very closely look at how we select teams — whether they might have the capability to eventually go to market. And that's also something that we focus a lot on when we mentor and support teams.

Eric: And can you talk a bit about SPRIN-D — is it ‘challenge leads’ in your department, and do they go by' program managers or something elsewhere? Or how do you all name them? The equivalent of DARPA program managers.

Jano: So, at SPRIN-D, we call them innovation managers.

Eric: Innovation managers, yes.

Jano: Yeah. And we have a group of innovation managers in all different kinds of fields. And so they have a high degree of freedom on which problems to tackle, and how to tackle them. So it's not that we have specific innovation managers that only do challenges, or only do whatever kind of instrument. Because I think what's really worthwhile, with a set of financial instruments that is as varied as ours, is that we think good and hard about how we best combine them, to get the most out of the potential that we have there for developing new technologies.

Eric: And where have you historically found your innovation managers? And can you talk a bit about how you're looking for innovation managers currently? Can you tell us about that?

Jano: It's a rare type of people, let's say that! Because what we are looking for is a very interesting combination of deep technological and scientific knowledge and skill, but that combined with what I sometimes call multi-sector experience. So we wouldn't want to have someone that only spent his time in his academic lab all his life. We would want to look for someone who has additional experience from industry, from founding their own companies, from being an investor, because these are all aspects that matter when we eventually want to commercialize those technologies.

And as I mentioned, it's not something — that commercialization aspect — something that we do at the very end of our funding process. But we want to bake it in right from the start, do it in a very integrated manner. We need to have people who not only have that experience, but also have the perspective of what it means to cross those different sectors, which is not easy.

Eric: And are there fields where you've had an easier — they're hard to find — are there fields where you've had an easier time finding innovation managers? Just because some circumstances of the field have made it that more of these are likely? For example, when I talk with some venture capitalists who do deep tech VC, some tend to have a pretty good time with electrical engineers. There's something about the field and the circumstances and the history where they go, "Oh yeah, we actually find quite a lot of founders in this area." And they have more difficulties in some other areas. So I'd love to hear your experiences on that.

Jano: I think there was no field where it was easy. Obviously, one thing that clearly plays a role is competition out there, for certain talents. In the field of AI, getting world-class people is not easy. And I think that's also something that's very important that we think about as a public institution. How do we attract the best talent, both for our employees, but also out there in the teams that we fund?

For some reason — I actually cannot say for sure why it happens — a large share of our innovation managers have a background in physics. Even though they might have done something else in the meantime. That just happens to be the case. I don't know whether it is random or whether there's a larger trend behind that.

Eric: And how should potential innovation managers conceptualize the opportunity? Oftentimes, they might have a vague sense of what's “ARPA-shaped,” what's “ARIA-shaped,” what's “SPRIN-D-shaped,” etc. But you've probably done a lot of thinking about this, and can make it quite clear for them! If you wanted to tee it up for a thousand potential innovation managers, how they should think about the opportunity, and why this is a chance even more exciting than founding a lot of companies to gain a foothold and do scalable innovation management.

Jano: Yeah, I think the kind of theoretical aspect of that is that, in pretty much all that we do, just having one shot might be so risky that if you really want to change the world, you better have multiple shots. And you probably have a hard time having multiple shots if you just found a startup. Then you do your one shot.

If you come to SPRIN-D, or any other ARPA, and run programs, you have multiple shots on goal, and therefore also increase your probability of actually having success.

Plus, I think specifically for SPRIN-D, if I look at concrete examples, one being one of our innovation managers, Patrick Rose. He used to be Chief Scientist at the Office of Naval Research Global, based in London. And he came to SPRIN-D because we have a toolset and an environment in Europe that was unique for what he wanted to achieve. For example, he came to run one challenge, that is still running, which is called Circular Biomanufacturing. And that started at a point where we observed that industrial biotech is currently good at producing niche products with large margins.
And it does that built on feedstocks such as highly refined sugars. The societal impact of that is somewhat slim. What these processes cannot do is do bulk products — so, mass markets that typically have very low margins — and do that based on other feedstocks.

What he came up with was that challenge, where the goal is to produce these kinds of low-margin, high-volume products based on real waste streams. And I'm talking about the real messy things, like supermarket waste, food waste, whatever it is, whatever goes in the garbage bin, take it out and produce new materials from that.

And so that is something where — the way we work at SPRIN-D — was something where he thought it would give really the best chance to actually achieve those goals. As well as having the know-how on bioprocess design, as well as fermentation infrastructure in Europe, was a combination that he had been searching for and found here.

And that is something where I think also, for future innovation managers or challenge leads, that is their chance to use the framework that we have at SPRIN-D and leverage the know-how, the technical know-how that we have across Europe. And I think that is something where we can really push the boundary of what's possible.
And for me, it's really about matching talent and opportunities in the best way possible.

Eric: And if I can give you a slate of three or four questions, and you could probably answer them all at once, but I'll put them all on your plate. So what, are there any particular types of innovation managers you're currently looking for? Are any technical areas in versus out? Is anything too technically risky? And also, how many are you looking to hire or onboard? What are the timelines in which you do that? And I'll stop there.

Jano: Yeah. So, very good questions, because I think we might be operating a bit differently from other ARPA's in that regard, that it’s relevant for those questions.
The one thing is, in terms of topics and fields that we cover, I would say we are somewhat opportunistic in the sense of… we don't have siloed budgets. We don't have, I don't know, a rule where we spend a third on biotech, a third on AI, and a third on computers or whatever it is. We try to run the best programs we can, even if that would mean that for a year or two we spend a very large fraction on life sciences, and then you know, that weight shifts over time. It really comes down to finding the right people for the right program at the right time. We're willing to back and implement whatever we think has the most impact.

Also in that regard, it's hard to put a number on how many people we would want to hire. We just want to see the best people and then we'll onboard them, because it's also the case that the size of our programs varies quite a bit. So, just for challenges, the smallest challenge might be just a million euro in total. Whereas other challenges are larger, tens of millions.

And you could think of challenges, and we're actually having some in the pipeline, where it will be more about the hundreds of millions for a challenge. Obviously, that has an impact on how many of those you could run and how many people you onboard for that.

That is the pragmatism that we have. We are trying to find the best people, and then we'll find a way to work with them.

Eric: And let's say five really interesting innovation managers saw this podcast and reached out and they had programs in the 10 to 30 million range. Is that the kind of thing where if they were all interesting, you have the capacity to bring on that many, or is that kind of reaching the breaking point where you'd go, we'd hire some and we'd maintain relationships with others?

Jano: That's the fun part about being in a public institution currently. Especially, we are not in the same situation in the US and Germany, but we are certainly in a situation where we don't have an approved federal budget in Germany as of now. That means we actually don't know what our budget is this year and we don't know what it's going to be next year. Economists will know the debate about the debt brake in Germany, that it's limited how much debt Germany can make. That is a huge discussion currently.

But what we are currently creating is an exception from that debt brake for investments. So there is something where there's going to be investments in the hundreds of billions. And part of that will go into research and development and innovation. But we are currently at a point where we don't know what our budget exactly will be next year. So therefore, I think it is a great time and a great opportunity to think about the question of whether SPRIN-D might be a good place to really push those programs because we might really be able to implement them and run them.

Eric: And in terms of growing the supply of interesting innovation managers with actionable ideas, do you want to talk about the partnership you have spun up with my colleagues at Renaissance Philanthropy? Because of course, the program they've spun up comes from this line of thinking — that of the number of really interesting technical minds who could do the technical side of being an ARPA-style PM, a lot of them don't learn to think appropriately in terms of servicing a follow-on funder, thinking in demand from day one of an ambitious program. Because programs at SPRIN-D, or ARIA, or ARPA are applied and ambitious and it has been hard to marry the two. So can you chat a little bit about that and what you're doing with BITS?

Jano: Absolutely. BITS, standing for Big If True Science — it's a program by Renaissance Philanthropy, a US-based philanthropy.

And the purpose of that BITS program, that accelerator program, is to take on people who have a clear vision for future breakthroughs, about capabilities that really might change the lives of the people, and how technology could support us getting there. And the mentoring program supports them in designing R&D programs that enable firms, any kind of institution, to really take those steps to achieve that goal.

As we mentioned, it's a rare type of person who has the skillset that is necessary to lead such a program. Therefore, it's absolutely necessary to look for those people globally. And that's what we do. We look for the best people globally. It is really also a special skill and knowledge set, designing those programs, because that is not something you would typically do at a startup or in an academic institution or as a policymaker or in any other kind of position. Plus, the number of such programs that are run are — I would say — very limited globally. Therefore, there are not a ton of people who have very good experience and insights into what works, what does not work. So therefore, combining our experience at SPRIN-D with the experience and expertise of Renaissance Philanthropy, which can really draw strongly on experience from experienced DARPA program managers, for example, or other people who used to work at different ARPAs — that is something that I think is super critical in designing the best programs. Because in my perspective, designing the best programs matters strongly for our actual success probability in the end.

Eric: Yeah, and we're going to talk a lot about know-how as we get into the nerdy metascience bits of this conversation, including in a couple of minutes.
But essentially, if somebody technically thinks they have what it takes to be something like a SPRIN-D innovation manager, it's a kind of program meant to help you get up to speed on the know-how. And if you're interested, you can email Joshua, JP, or Ales at bits@renphil.org.

But now to get into some very nerdy bits that you know a lot about and SPRIN-D has been very impressive on. Do you want to talk about the range of financial instruments you've come up with, and why they're a big deal in your eyes, in unlocking a lot of the upside that an organization like SPRIN-D is meant to go after?

And I have it printed out here too, in case we miss anything.

Jano: Absolutely. In contrast to many other ARPA-style institutions, we have a very broad mandate in the sense of we might be working on life sciences, drug development, biotech, computing — in principle, everything that has civilian or dual-use cases.

That means we could potentially tap into every kind of field of technology or application. And I think there are fundamental differences between different fields of technology that we need to take into account when we want to design effective programs. Plus, we might start funding different kinds of institutions at different TRL levels, at different levels of maturity of that technology.

Taking all that into account, for me, means there is not a one-size-fits-all, "we just hand you over cash" kind of way of doing things. And therefore, I think we are pretty happy that we got a whole range of different financial instruments. So, different kinds of grants, different kinds of R&D contracts, equity investments, convertible loans, and I think that is something where we can really tailor our support to the needs of that particular team. An academic institution will have completely different needs than a startup raising a Series A or a Series B, and if we think they need our support, then we must be able to provide them the best support possible.

Eric: Could I run a couple of hypotheticals by you and you walk me through if they're possible at SPRIN-D and what the R&D piping to make it happen would look like? The first one is somewhat niche — but I think important at an organization like SPRIN-D.

Let's say there's a university with a postdoc and they’re in some niche area that's exceptionally important to an area you'd like to run a challenge in. And this postdoc, or them and a group of their colleagues, are just the best people. And what would make the most sense is for them and a team of two or three to leave and set up shop as a brand new company or nonprofit, and it would be inordinately important to the challenge being viable.

But what it might take is a half-million euro grant to this org that, while these people are proven commodities as postdocs — this is a day-zero org. It has no track record, it's never filed taxes. Can you do that? How would you do that?

Jano: So we can absolutely do that. That's not an issue. It's something that we see very often — that when teams apply with us, they might actually still be a postdoc, at whatever kind of institution, at an academic institution.

But they tell us, if you fund us, we are going to create a spin-out, and then that funding should go to the spin-out. So we can fund those kinds of no-track-record startups, institutions, people that have very little track record or no track record in that kind of sense. So technically, that's no issue at all.

What we would really focus on is a question of whether these people are actually willing to do it. And really, the part of being willing is way more important than the question of whether they're currently capable — in the sense of, when you have people who have spent their whole career so far in academia, they will not have a clue on how to build a company. Which is okay. That is to be expected.

What we really would need to see are signs that they're coachable, that they have potential and can develop. And that is certainly also something that is very crucial for us when we think about follow-on funding.

Eric: And so to turn two knobs a bit and see where this becomes difficult.
Is there an amount of seed funds where you'd say, “Oh, that's going to be difficult. We can do that, but doing it with 500,000 euros versus X are just qualitatively different things.” Like, where does it stop?

Jano: That's a very difficult question. I think it, again, it's also very case-dependent. So if we talk about nuclear fusion, there's probably nothing very cheap to be gained there. You would probably also be looking at very technical people that you would need to support there.

And so it would be a combination of the technical profile of the people and the amount of money that you would probably not be willing to spend on — for example, a software project where you have a purely academic person and hand them millions as funding — that's probably something that we'd never do.

And so I think, as an economist, as an academic, it depends.

Eric: Yeah. And is there, do they have to be in Germany? Do they have to be in the EU? Is the UK okay? Is the US okay?

Jano: We have a broad understanding of Europe and we fund all across Europe. So the UK, from my perspective, and from a personal perspective, I consider the UK still a part of Europe.

We fund in non-EU member states as well, including the UK, Switzerland, Norway, countries like that. Also including Israel. So for a technical definition of that geography — it's EU member states, states of the European Free Trade Area, plus the UK and Israel. That's our typical geography in which we fund.

Occasionally, we go beyond that, especially also when we run challenges where we have what we call a non-funding track. So sometimes, we would open up the challenge for teams that we don't fund but might still want to integrate.

So for example, we had a challenge on, let's say, digital infrastructure, digital proofs of identity. And we had Samsung and Google Android as participants, but we didn't fund them. So that was one of the cases where we extended our geography through that non-funding track.

Eric: And when you're selecting, let's say, postdocs who want to start a new organization, you brought up "do they want to leave?" as one big question.
And practically, of course, that's one of the big questions. But when VCs are funding firms, they also ask, “Are you a founder? Do you have the stuff?” And at VC firms, they're remarkably unstructured a lot of the time. Like even at the biggest firms, they make a judgment call on that. A lot of people maybe think a judgment call is the best way to do that, but they also don't think a random person making a judgment call is synonymous with how government typically does things.

So can you give us a sense of how this goes internally at SPRIN-D?

Jano: Yeah. So if I'm focusing in on challenges — what I really think is super important about challenges is that stage-based approach, that gives us the opportunity to take on a range of different teams where we might be uncertain about some of their qualities.

Obviously, there needs to be something in them that makes us really want to give them a chance. But we try to be very explicit on what it is that we give them a chance on. And that gives us a chance to — either we see someone who has an amazing track record in actually executing stuff, multiple companies founded, brought things to market, but might be somewhat lacking in the tech aspect. Or the other way around — maybe technically super strong, but no track record whatsoever in terms of actually implementing things that can go to market.

If we see the potential in those blind spots, we give them the chance, take them on, fund them for a bit, and give them the chance to demonstrate that they can do what we're unsure about their qualities. And then, at the end of the stage, during that stage, we might be able to pull the plug and say, okay, either they didn't perform as well as we hoped for, and then that journey is over, or they really outperformed.

And there might really be very positive surprises, because every once in a while we get those people who come out of academia, who have never been in any kind of other role than being a postdoc, PI, whatever. And suddenly, it's an awakening and they really have a very strong development up to the point where they say, “If I would have known earlier what I know today, I would've founded five companies before.”

Giving those chances through the challenge-based approach where we can just say, okay, we admit 10 and we see how it goes. It's something that I very much like — and I think it's very important if we want to be able to solve problems that apparently are so hard that no one else has solved them.

Eric: And in terms of the makeup of the performer pool, there's one — like a piece of what some people would think of as operational or contracting minutiae that to me is at the core of the SPRIN-D model. Can you talk a bit about output monitoring, not input monitoring? This is particularly fascinating to me, because in reading a bunch of the early DARPA oral histories, right around the 1980s, you start to get gripes from a lot of the long-time contractors, places like BBN, where essentially they have long complaints about the nature of procurement they’re changing and “there being more accountability for the pennies than accountability for the results.”

So this is precisely what you're going after with output monitoring. Can you talk a bit about how it works, why you thought it was essential, and how it changes the makeup of your contractor ecosystem?

Jano: Yeah. So I think if you look at how science and innovation funding in Germany would classically look like, especially from public institutions — you’d have some kind of call, you write a proposal. In that proposal, you write what you're going to do and what you're going to spend your money on, that’s typically in a very detailed manner. You would typically be required to follow that plan, and it might be a three- or five-year plan, a very detailed one.

And — I'm not kidding — you have to tell the funder how much your train tickets will cost three, four years from now. [laughter] That’s just weird. But that also means that, for one, you're very much focused on what your plan initially has been.

In theory, you might be able to change it, but then you have to reapply and ask for permission. It also means that you have to prove all your expenses were in accordance with that plan you originally submitted. And that is just super cumbersome. Everyone who has filed travel expenses or had that kind of grant funding knows how cumbersome that is.

And typically it involves people who would be better off working on innovation. I think it's really mind-boggling that we keep the best people of our generation — and I think that is the people that we want to address — busy with sorting receipts and doing those expense claims.

It’s highly inefficient for a project that is run. And I think it has a very strong selection outcome in the sense of, you know, certain types of people might not expose themselves to those mechanisms.

For example, I was very curious to see how our challenges would run in the field of AI, because people there just have very strong outside options. They really don't need funding from any kind of public institutions, unless it's really in the hundreds of millions and might be something different. But at our typical ticket size, it's nothing that they would require. I was talking to one of the teams that we fund in one of our challenges called Composite Learning. They are Y Combinator alumni. They are venture-funded. They don't need us.

And what they told me is that they specifically made a change in their policy to not apply for public funds because they saw that we act differently. That we don't waste the time of their most precious resource, which is the people. But we really support them in a way that makes sense. I think what's interesting is that we have to square that with our responsibility towards taxpayers. We cannot just get rid of input monitoring and then say, “Here's the money, have fun,” because that would be neither legal nor responsible.

So what we do is focus on what you mentioned, which is output monitoring. We are very close to the teams, so we have regular update calls, meetings, we visit them. And what we care about is new data being generated, demonstrators being built — something that is tangible, where we can see actual progress. Most of that will mean learning things that don't work, which is a great way to actually know what progress can eventually look like.

And by replacing that input monitoring with output monitoring, we actually make that monitoring mechanism worthwhile for the teams as well. Because if we are in a stage where we have people at SPRIN-D that are so good that the teams expect us to actually be helpful in overcoming their hurdles, they will voluntarily share with us what went wrong, because we might help them to actually solve those things.

That gives us a transparency that I think is way, way higher than if we would just look at how much you spent on a computer or on copy paper or whatever. I think it's way more productive.

Eric: And if sometimes groups will enter a challenge competition and they pick one technical path to pursue, and the path itself proves to be unfruitful — often for no fault of the team's own, other than a not-perfect initial hypothesis. But that's why you get in this game, is to test the hypothesis, not to be right off the bat.

If they proved to be an exceptional R&D team who just got unlucky with the hypothesis they explored, do you all have a way to keep them around and throw them at further stages of the challenge, if you thought that they proved themselves to be the right team to do that, and they just need to build off another team's learnings or something?

Jano: Obviously, it somewhat depends on the extent of that. But to give you a concrete example, in one of our challenges in drug development for antiviral therapeutics, one of the teams was expecting to use a specific type of delivery mechanism. It's one of the essential building blocks of the final product that will have huge consequences on whether it might be effective, what side effects might look like, things like that.
It's a really crucial part. And they changed — completely changed — what their delivery mechanism will look like mid-stage. I don't remember whether it was stage one or stage two of that challenge. And the process for that, if you can call it a process, is they call me or the innovation manager running that challenge and say, "Look, that's the data we generated. We think we have a better solution. We need a switch." And then we say, "Of course, you need to. Let's do it."

And it's the other way around [as well]. If, at the end of a stage, they would write, as they do, a written report and they pitch, and if they come to us and say, "You know what, months ago we figured XY is not working. Can we change it?" And then we would ask them, "What did you waste your time on, the last couple of months?" Because once you figure it out, that's the time when you need to change it.

Eric: And the government should care, 'cause that's time you paid for, too.

Jano: Absolutely. Yep. But it's not the typical logic. The typical logic is, we need to be able to make sure that money is spent the way we allowed it to be spent. And that keeps you on your path even if it might be completely bonkers. And that's the rule, not the exception, in our business, because I'm 100% sure that none of the plans of people that apply to us and get into a challenge can follow that path 100%. 'Cause technological uncertainty is just so high that it's impossible that everything will work out the way they planned. If that would be the case, they either lied to me or they weren't ambitious enough.

Eric: Yeah. And this is an approach that sounds common sense, reasonable, to most engineers and researchers. But also the economists love measuring things, and you're at least somewhat sympathetic to this.

You can imagine one variable, like an outcome metric of success here being, "Do you all get an inordinate percentage of people who are talented and are able to win a grant who have never won an EU grant before?" Because there's a certain bit of know-how involved in winning and maintaining an EU grant in the same way that, similar to SBIR grants in the US, where a lot of the researchers will roll their eyes and they'll say winning that is its own skill, and being good at research is just one thing they're meant to do, but it's probably not the primary skill.

So is this something you all track or would be interested in tracking or could care less about because you're busy with other things?

Jano: You mean in terms of it being a positive KPI if we attract people who never got an EU grant, because then they're the right type of people?

Eric: Yeah. Or something of that sort! You might think of a better one right now. I'm sure you will.

Jano: The difficulty there is, it's not a skill issue — for the pure fact that there are consultancies out there who write that application for EU funding for you. In fact, there has just been a study that was published on evaluating EU innovation funding.
One thing they found is that the effect of that innovation funding was very small, if at all positive, at those institutions that they funded. The largest effect that they saw was in consultancies that helped those companies and institutions get those grants in the first place.

And that's pretty much the only significant effect that they found on firm success — those consultancies.

Eric: So is something related to a consultancy popping up or being the major coefficient in your paper, I guess, a metric you would be interested in, then?

[laughter]

Jano: No, it's really bad in the sense of — a large share of funds that were supposed to be for innovation actually end up in consultancies, because it's so complicated to apply to most EU grants that you've got to have a consultancy that does it for you.

And I think that is a very bad state to be in.

Eric: Yeah. And can you talk a little bit about 14-day time-to-money, which is something you're very proud of and should be?

Jano: So 14-day time-to-money refers to how long it takes us in SPRIN-D challenges to go from the deadline of application to having teams selected and having contracting done.

If I zoom out a little bit, going from the ultimate goal we have, which is fostering breakthrough innovation — we care about what are success factors. Some of those success factors are attracting the best teams out there. And then, as we talked about, that might involve working in a way that is attractive to the best performers out there. I think therefore, time-to-money or speed of processes, lean processes, no unnecessary bureaucracy, is really crucial to address the best performers.

I think it is something that far too little attention goes into. It's also something that is not straightforward, in the sense that certain aspects of what is required to go at that speed are formalized. So for example — what are the legal requirements in procurement law or state law or whatever, what do our contracts look like? That is something that we worked out, that we can share, and that in principle every other institution could do.

What I think is the way more tricky bit, is doing it hands-on and focusing on the processes that are necessary to arrive at that goal. And that goes very deep down to what is the operating system of the organization that you are.

To give one example, how do we arrive at those 14 days? For a two-stage selection process that we typically have, it means that within a week we need to be able to analyze all applications that come in. That means I need to have a team, and typically it's a SPRIN-D internal team, part of our technical team at SPRIN-D, who have to be able to analyze all applications within a week. That means, for example, I need to be able to block their capacity so that four or five people, however many I need, are really there full-time, no nine-to-five, and work on those applications.

It also means I need to have in place a jury, because final selection at challenges is done with juries. I need to have that in place beforehand. I need to have those jury members, they need to be informed about what's coming, what the setting is, what we are looking for. You know, it's up until event organization. 'Cause at the end of that second week, we have pitch events where if we have 20 teams on the shortlist, we have an event where 20 teams are pitching over the course of two days.

Then we'll need to have a way of arriving at a funding decision at the end of that second day, at the end of those two weeks. Plus, we are looking at contracting as well. I need to be able to do contracting, in principle, overnight. That means at the end of the selection process, my team in the SPRIN-D office is preparing the contracts, sends them over to our directors, and they really sign those — I don't know, 15 contracts or however many teams we take on — overnight. That means the teams then the next morning have those signed contracts in their inbox.

That is something where, for example, as a counterfactual, I would have a very hierarchical organization with very strict limits. I would not be able to tell my team of analysts, "That's a week that you have to block full-time to do analysis." I would not be able to tell my directors, "You need to be there to sign 15 documents overnight." That very much goes to what I would call the operating system of that institution. It's first the ability and then also the shared perspective that this is something that is important.

Eric: And can you talk about your partnership with NSF and how you imagine transferring know-how on these matters back and forth?

Jano: I think that is something I would call tacit knowledge. Those formalized aspects, what are the legal underpinnings of what we do? What are the contracts? That is something that we frequently share. And anyone who wants to see our contracts can do so. In fact, we publish them all the time. But I think that for pretty much most institutions, it would be impossible to copy the results. Just because you have the legal basis and the contract, you will not be able to run this process in 14 days, or a week, or three weeks.

I think that requires very much hands-on process knowledge, experience, and knowing the limits of your organization. And that is also something where I think very little knowledge is shared, just because it is tacit knowledge. Also because such programs are not run that often. I've been running 12 challenges. I was talking to a DARPA program manager who said he was running a challenge and once he did that, he was trying to find someone at DARPA who used to run a challenge before.

But if running programs is so rare that you have to struggle to find someone who did it before, I think it's a very bad starting point for learning at a very professional level. And that's why, for example, in our partnership with NSF, what that encompasses is that we jointly run a challenge. So it will be run in that way that NSF funds teams in the US, SPRIN-D funds the European teams, but we will very strongly integrate all our operations down to the process level. Because I think that is one of the ways in which we can really share that tacit knowledge, the process knowledge, the know-how, and really understand how the other side ticks, what works for them, what might be applicable in each other's settings, and how we can actually improve on that.

That is also an aspect that I really love about the BITS program, because that is another way of really deeply integrating experiences, sharing experiences down to that level of operations that we otherwise very rarely have.

Eric: And are there any genres of, or sorts of innovation managers who you haven't had at SPRIN-D yet? For whatever reason, you've just failed to get a certain archetype that you would really like to come through that BITS program or hire as innovation managers at some point? I'm imagining, let's say, a potential innovation manager reading your site and saying, "Oh, they've never had anybody who looks like me before. This is not meant for me." I'm trying to give you a chance to flag that you would love X sort of person or Y sort of person.

Jano: You know, I wouldn't want to focus on one particular profile because I'd rather want to put it the other way around. I think there are potentially many people out there who might not know that they might be a great fit.

And they might come in various different shapes and forms. What is, I think, really crucial is that they have a strong commitment to change a field, that they have a strong conviction that what most people think would be impossible or might not even have thought about approaching or trying to deliver, that that is something we would really need to have.

In that sense, I really don't care whether it's in AI, computing, life sciences, or any other field. It is a strong conviction. And what I want to see as a track record is the ability to execute.

Pure theory will bring us, at least in this business, nowhere.

Eric: Yeah. And in selecting people like this, you do not have quite as finite an amount of money as maybe something like DARPA's BTO, which probably gets handed a specific budget, but the funds are somewhat finite.

While the specific number is hard to pin down and you have to make really difficult judgment calls between fields — like computer science and electrical engineering versus quantum biology, or how to generate more life sciences data. How do you go about making those judgments? What does that look like internally at SPRIN-D?

Jano: It's interesting in the way that I've been talking about the pragmatism that we try to have, and that has been supported, in very practical terms, by SPRIN-D's budget increasing from year to year. We are a young institution. And I think so far, for most programs we really wanted to run, we eventually were able to find the funds.

You know, we started out with a budget that came solely from our Federal Ministry for Science and Technology. What happened in the meantime was that, for one, more ministries from the federal German government saw what we're able to do, and that they're not able to do the same R&D funding, and gave us extra amounts of money to be active in certain fields. As well as other governments starting to approach us to run programs, just like the NSF collaboration was born out of the fact that NSF observed that we are able to run programs in a manner that's way different from how NSF would typically run programs.

That is something that we see increasingly — that we have other governments, mostly from across Europe, but also the US, trying to work with us. That also gives us the chance to have a much broader portfolio of programs that we can support.

Therefore, I really have yet to see the case where we say, "Okay, that's a great program, a great innovation manager, but unfortunately we cannot fund it." So far, in pretty much all the cases, we've found the funding. Because hopefully, if you can really talk about the impact that a successful program, a successful challenge in that field would have, and can show some of the track record that we have in the sense of — we really manage to bring people into the game that typically would not be addressed by public funding, and that really giving us an increased probability of success.

That is something that's very convincing to policymakers or funders — to actually go after that combination of topic and method.

Eric: So is there any really germane constraint that you would want innovation managers to know about before reaching out?

Like whether it's — is there a TRL you work in less, or is the answer, "Oh, your program needs to be a bridge to something interesting and it could be lower or higher TRL as long as it's a good bridge." Is there any quick framework that you think should shape their thinking?

Jano: Yeah. A very short answer would be, we do TRL three to seven, and that's it. But that's not doing justice to the complexity that we are in. When we talked about the difference between challenges and prizes — in prizes, you have that big honeypot at the end that might be very tempting, and we don't have that.

But our perspective is that we try to design challenges such that if a team is successful over the course of that challenge, then that should unlock follow-on funding. That might mean different things in different fields. It might mean, in certain fields, that point is coming earlier.

In biotech and AI, for example, there might be cases depending also on one aspect that is market risk, that we have talked about, but I think it's very crucial to differentiate between technological risk and market risk in how you run programs.
We try to think about that kind of handover point before we start a program, knowing that if we learn that other funders would not be able to take over funding in cases where we see great potential, then we would have the means, the instruments, and the budget to further support those programs, those teams. That's also what we did do in the past. For example, you might see teams coming out of challenges but having trouble raising the necessary funds due to excessive market risk. And then we will be able to catalyze funding rounds getting together by being able to fund actually up to 70% of a funding round.

Eric: And I'd love to ask you — take the time and ask you three niche metascience questions that you could have quick answers to. Or you could dive into them if you think they're important! And so the first one would be, in understanding the opportunities and constraints of the innovation manager position, are there scenarios in which they might really want to do something where somebody will stand in their way, or them wanting to do it is not sufficient for SPRIN-D to do it?

Jano: That might be the case. For example, what we have in our group of innovation managers is very strong experience and knowledge from the investment community.
There might be innovation managers out there or at SPRIN-D who have less experience in that regard when it comes to structuring funding rounds, clearing up cap tables, and things like that. So there might be a case where we go into that discussion and decide to make sure that we need to see certain requirements fulfilled before we can do follow-on funding, for example.

And that is something that, due to the experience that we have in our group of innovation managers, we also can very much lean on. Therefore, I'd rather see it as arriving at a better result due to our internal processes, rather than forbidding a program manager to do something.

Eric: And who picks the juries that decide on program applications? Who makes up the juries? Are they looking for unanimous approval from the jury? Or can high-variance reviews on applications be good?

Jano: Juries are one thing that we only do in challenges. And it's typically a combination of the innovation manager and me looking at jury members and selecting them.

What is very important to me in that decision-making process is, for one, that juries are somewhat small, meaning typically around six to eight people. What I tried to do when we set that up was try to combine a broad set of perspectives and experiences, both in tech and commercialization, while avoiding having a large committee.

Because large committees typically tend to agree on the smallest common denominator. And that is exactly not what we want. We want to take those bets — those high-risk bets that might actually be able to really pay off. Therefore, the decision-making process within that jury very much embraces that variance in opinion.

It also very much depends on the field, on how much variance we might have in there. One measure that we take to embrace that variance is to have a system of golden tickets. So, for example, for the initial selection for stage one teams, the average jury member has a golden ticket, and if he or she thinks that team X is really so exciting in one category, “I really want to see them perform, I want to see that thing demonstrated” — then we fund them.

We do not have a majority vote typically for the decision-making where we say, “Okay, the majority is for the team, we take it on, and the majority is against, we don't.” We try to arrive at a consensus within the jury, where we have a shared opinion on the strengths and weaknesses and risks of a certain team, and decide why we want to give that team the chance to perform or why that's not the case.

Because that also gives us a basis to evaluate them against our decision, over the course of that stage.

Eric: And who gets final say, once you incorporate all of that and you say, "Oh, we've generally frozen on an agreed-upon list of strengths and weaknesses," whose job is it to look at that and say, "We're going to do X"?

Jano: It is the jury. The jury will arrive at a consensus, even if it means to agree to disagree on certain aspects of a team. It is my job to moderate these juries to arrive at that consensus. And it's really something where, from afar, it might look somewhat weird.

For example, I was in the US, I was talking to a former office director of DARPA, and I was telling her about our selection procedure. And she was like, "Juries? That's not how it's done," coming from a DARPA background. Then she and several other people joined us for a certain jury session.

And she was in a jury session, saw the discussion, saw how we arrived at that consensus, and saw how we tried to aggregate that information that each and every single jury member brings, and build a consensus about what kind of the best-performing teams might be down the road.

And then came to the conclusion that actually, that is a very nice process.

Eric: And do you have something written down on what you would do in the event of a hung jury?

Jano: Never happened. I don't know. I was like in like — I don't know, like 30 jury sessions — never happened.

Eric: This is fascinating. This is also one of those areas where I can imagine there being cultural differences. Like when I learned how Swiss children vote on what field trips to go on, even from exceptionally young ages. It just wouldn't work in the US. [laughter]

Jano: I don't know. But I mean, we recruit jury members internationally.
It's not just Germans sitting there and agreeing with each other. So we have international juries. European, North American, Asian, it's really diverse. And that is exactly what I mean with tacit knowledge and something that you might only be able to fully grasp with all its pros and cons when you are in the process, because it's highly context-dependent.

And that is something where, from my perspective, we don't have — so far — a very good way of learning. And it's also something where we typically don't have the scale to do proper learning. I was talking about the amount of times that we run challenges, other institutions run challenges.

If you think about the fact that we run or start a new challenge maybe on average three times a year, what is the process where you would think that, on a professional level, you get really really good at things that you only do three times a year? And I think it's something that we do more often than other institutions in the field.

Therefore, I'm really concerned about that. How do we make sure we learn across institutions, especially those aspects of tacit knowledge? Because I think these really matter. All that I said about how the operating system of a funding institution might have a critical influence on success factors such as speed — it's nothing that I think we have a good way of learning from one another.

If I look at other fields where you have a strong dependence on tacit knowledge —think about the culinary world, restaurants. That is something where I would like to compare the way we structure processes to. There is this French term called mise en place that describes the situation where chefs would prep their kitchen and station beforehand, to be able to execute when service time is and execute both with quality and speed. It's a very deliberate process about what you prepare beforehand, what you focus on in the moment, to achieve the highest quality at the highest speed. And that is something that we try to do when we try to achieve those 14 days time-to-money. And it's something that you cannot easily copy.

But in that culinary world, with all that tacit knowledge and hands-on experience, we have systems like apprenticeships, mentorships, where you go through stations, you go from restaurant to restaurant to learn all that tacit knowledge. I think we don't have that to that degree in our world.

That's also part of why I think that collaboration with the BITS program, with NSF, to really go down to that level of operations is really exciting. Because that might be one step of trying to do that learning and then transferring of insights on those elements of tacit knowledge.

Eric: Yeah. And so, I'm going to move us along a little, skip a few questions because we're running out of time, but I have a few I really want to get to with you. So one of them would be — we're going to have a lot of West Coast listeners in the US, and Boston and the UK, but I have a pretty West Coast center of gravity. And they're going to have a lot of curiosity about the German R&D ecosystem in general. And also, you might want innovation managers and things in cohorts coming from American institutions and things like that! So I would love to tee you up with something and I would love to force you to step out of your own skin slightly here.

So — I get the sense that Germans don't really brag enough. For example, I read the Fraunhofer-Gesellschaft annual reports that come out, and as an American, I see a lot in there that makes me jealous. And I think there are things in there that make the UK folks jealous as well. Like, I think Catapults and initiatives like that were overtly Fraunhofer-based.

But what I've brought up Fraunhofer to — and for those who don't know, Fraunhofer is the applied version of the Max Planck Institutes, in very simplified form, in Germany, and they have a whole ecosystem of them. But when I bring up Fraunhofer to Germans, and like really fantastic metrics from annual reports, like they have an 80 or 90% spin-off survival rate after five or 10 years, German folks often bring up to me all the things Fraunhofer's not doing or could be doing better. But if I was going to force you to put on your West Coast of the US hat and brag for a bit, could I hear your most optimistic pitch for the sorts of ARPA-like programs that are possible in Germany, that are difficult elsewhere?

'Cause you have a different set of institutions, often with very different focuses or that exist in places that are the valley of death in the US in certain fields.

Jano: Yeah, if you think about the innovation that led to what now is ASML and EUV lithography — that is a degree of engineering that I think is very hard to find elsewhere. If you think about the fact that you have countless technologies at the highest level of engineering, that you then need to integrate to build something like the lithography machines of ASML, that is super hard. If you think about the precision that is necessary to shoot those lasers at those tin droplets that create the radiation that is used for that lithography process, it was probably a 10-year R&D effort of major companies and institutions involved at an unprecedented level. And I think that is something where you really need those sustained levels of engineering at the highest quality in areas that have never been tried before. I think that is where we really have great institutions in Europe.

Eric: And would you care to, for the listeners who won't know precisely what you want to talk about, can you share 3, 5, 10 — whether it's areas, specific centers of engineering excellence — where you would just want people to know if they're thinking of anything in that area? Like a strong argument that Germany is the best game in town, sometimes the only world-class game in town, to do work in that specific area. Like Americans might off the top of their head think of a company like Zeiss and understand that we don't have things like that, but there's probably all sorts of not-famous firms in areas that you think Germany is world-class at. And people don't force you to brag on a podcast like I am right now, often enough to talk about that.

Jano: I think it's really not about those singular, stellar companies or institutions. I think the real strength of Europe is its very high level of quality across the board. It's the diversity that we have with very strong centers of excellence.

For example, life sciences in Denmark and Copenhagen, engineering in Germany, places like Munich and Berlin, AI also in France as well as the southwest of Germany. And I think the real strength is that all these places are really, from American standards, not that far away from one another.

Plus you have a really strong industrial base that could take on things and really scale. And I think that is a thing that we talk not enough about. As I mentioned, our innovation manager, Patrick Rose, came to Europe to run challenges with SPRIN-D because we have that infrastructure in biotech that really enables us to not only develop, but scale things.

Eric: Okay, fantastic. Thank you for that. I have a bunch of questions that you could probably run through each of them in under a minute that I would love to get you on the record talking about, but none of them is that important. This is the list of things I wanted to catch at the end here.

And so one of them was, when you're putting together challenges, you can understand why people think, "Oh, you don't want to condition not on the outcome or constrain things in that way," but it's often essential for practical reasons. Is coming up with those intermediate benchmarks painful, or is it actually often quite straightforward and forces you to think hard about it?

Can you talk about the trade-offs and the emotions involved in coming up with those?

Jano: It's very difficult. Very hard to do well. And from my perspective, it cannot be done purely based on quantifiable metrics and KPIs, because the technological uncertainty is too high to be able to exactly pinpoint both the exact KPIs that you want to use and the right threshold levels. But it also brings in a factor of appreciating all the work that the teams do in different dimensions. The combination of seeing teams in parallel gives us then the opportunity to not just like a team because we like the people, but really compare that to competitors and really show who is ahead.

Eric: And so I talked with Ilan Gur about the idea of one ARIA program leading to another, and he said, "Oh, I actually really hope we do something like that." Is that a tool that SPRIN-D thinks about, or is that a bit far away? 'Cause you're giving yourself a lot of leeway to not think in translation.

I'd love to know how you consider that, 'cause there are obviously pros and cons to the approach.

Jano: Yeah, no, absolutely. And we do sequels. For example, when we look at autonomous systems research and development, we did a sequel to a challenge that we ran last year and run version two this year. Not unlike the famous DARPA Grand Challenge, where pretty much everyone failed in the first attempt, in the first challenge, and then you have a shot at it again in a slightly different way. But no, that's certainly something that we do.

Eric: And so to sit on that DARPA Grand Challenge point, that's an area where DARPA was a workhorse funder of autonomous vehicles from the early eighties up until the Grand Challenges. So that's almost two decades where they had a typical like, "gardening, we don't know what is there in this space yet, we have practical outcomes we're looking for, but also we're developing a tech branch here," that ended in a challenge.

If you had an area that felt like the one approach was ideal for one decade, and then a completely different approach would be ideal a decade or two from then, how might SPRIN-D go about attacking that field in terms of its program structure? If you were going to do sequels, what would that look like, if you all were trying to do the same thing?

Jano: You're talking to an institution that is five years old. [laughter] So I think most of it is still to be developed. Let me give you another example, and that is nuclear fusion. We have been funding technologies for nuclear fusion for a couple of years now, and that certainly will not be finished next year or the year afterwards.

So we continuously evaluate what the right next step is. That depends on tech development. That depends on the whole ecosystem developing, how other funding efforts are structured. And that is certainly something where we think a certain field of technology is both super important and would have a great impact, but wouldn't be able to move along without us.

Then we are certainly willing to continuously support that development.

Eric: And SPRIN-D exists in part to fund stuff that kind of presently ends up falling into the valley of death more than it should. Let's say there was some contractor where it maybe wasn't worth any given program or challenge officer funding the org into existence, but SPRIN-D itself would benefit from some contractor existing and being good for the rest of the ecosystem.

And a specific example from DARPA history — I think of early MOSIS, where DARPA kind of created their own silicon broker at the USC Information Sciences Institute. What they did was enable like really small chip runs for all of DARPA's researchers. I don't know if you had to spend that out of your program, if you would do it as a program manager, but like from the office level, it makes sense. Structurally, how would you all go about doing that?

Is there some straightforward mechanism? For example, ARIA has activation partnerships to do things like that. How would you all handle it?

Jano: So it would be something that we would steer from within SPRIN-D. We don't rely as much on activation partners as ARIA does. We do more of that work SPRIN-D internally, but it certainly would be something that we would look at and would be willing to do.

When we figure out that there is a certain bottleneck that is in the way of achieving an important breakthrough, then we would address that bottleneck. Even if it would mean that that work in isolation would not be a breakthrough. We're just aware that oftentimes those breakthroughs are different puzzle pieces fitting together, things building on top of each other.

It's not just the one problem that we solve and then it's a straight line to a breakthrough.

Eric: And if SPRIN-D's budget 10x'd, would we see 10 times as many challenges and programs, or is it not so linear?

Jano: So I think the one question is, do our ticket sizes get bigger? And the other question is, how broad are we going to be? I think there is a limited scope for having bigger ticket sizes. The amount of funding, the ticket sizes in a challenge — I think that will not change a lot. What we will look at is, if we have a 10x budget, are there technologies out there that need bigger ticket sizes?

Because, for example, we might need to fund phase two clinical trials in certain cases because no one else is going to do that, participate in Series B rounds because they would otherwise not come together. Really, I think there is a natural limit of what certain institutions or team startups at a given time can productively work through in terms of money. Having money is necessary, but it can make people lazy and slow if they have too much money at the wrong point in time. So it would rather be probably a combination where we have more programs and are able to address more challenges.

Eric: And if SPRIN-D was the only group funding some area for X amount of time, how long can X be before you start getting uncomfortable and saying, "Are we just making ourselves feel good here, and nobody actually values this area as much as we do?"

Jano: I think that can be a long time, especially when…

Eric: Is it five years? Is it 50 years?

Jano: Asking a five-year-old institution about what we are going to do when we run things for 50 years is like looking into a crystal ball.

[laughter]

But I think we have the commitment and the willingness to really follow paths as long as we are convinced that's the right way to do it. Because we start programs when everyone else would be against that. So we can continue programs when everyone else would be against that.

Eric: Yeah. And okay, just three more and then I'll let you go. And thank you for all the time you've spent with me today! First one, if you have a really wildly successful challenge, whether you had a huge technical success or you learned something that showed you the promise in this area and you said, "Oh, the expected value is even higher than we thought," or we got a good outcome — to what extent, what does doubling down for SPRIN-D look like?

And at what point is some percent of your budget maybe too much? Like I constantly bring up this Warren Weaver at the Rockefeller Foundation example, because it was 80% of Rockefeller's natural sciences budget. And I think a lot of orgs would say, "Oh, that's a bit too much." But I would love to know where expansion is natural and good if you're doing a good job, and where it might be too much of your budget or there's a sort of soft threshold that you wouldn't want to pass.

Jano: Yeah. That's a very tough question. I mean, if we spend a budget share of like 80% on a given technology, it would really warrant a question of why is no one else committing that amount of money into that project? I think it's very important to think about opportunity costs because spending 80% of money on a given project means not spending that money on a range of other topics and teams.

And I think that is something that we should be aware of and where we should have somewhat of a balance, especially given the broad mandate that we have. I think it's important to be aware that when we spend 10 million on contributing to some Series A, this might mean not funding 10 teams in any other fashion.

Eric: Second to last question. SPRIN-D is a metascience experiment in and of itself. And that being the case, I'm sure you look at all sorts of hyperparameters you can tune or things that could be improved. Are you all currently dealing with any bottlenecks for which you'd love some kind of organizational innovation to address? Or maybe you have one in mind and you're about to test it?

Jano: What I'm really concerned with is really the learning of that tacit knowledge and setting up ways to do that across institutions. I think that is something that really has great, untapped potential. Continuing that work with other ARPAs out there is really super important and I think can be greatly beneficial to the work that we all do.

Eric: Okay. And last question. So SPRIN-D as an organizational experiment, a government R&D experiment, and a scientific experiment — what's an example of a success of each where if in 20 years X happened, you would say, “We've succeeded. This has been a great experiment. I'm so happy I spent that portion of my life working on that, or this whole group of people spent this portion of their life working on that.”

Jano: I think one thing would be if we still exist and still work differently than the rest of government, that would be one, in terms of that government dimension. One thing that I start to see is that people change, for example, other funders changed their reporting requirements because teams that we fund would go to them and say, "SPRIN-D is not requiring that. Apparently, it's the German government, that seems legit, so why do you require it?" Therefore we see other funders in other countries lower their reporting burden on teams, especially that input monitoring.

That is something — when we can really have that push in the ecosystem, in the funders' ecosystem — that would be something where I would say that experiment went well, when we have that push to change how we operate, also beyond the ARPAs.

Eric: Yeah. There are a lot of scientists in the US who desperately hope you're right and that does come to pass. And then on the technical end, is it a Turing award? Is it a Nobel Prize? Is it another ASML?

Jano: It certainly would rather be the ASML than the Nobel Prize and the Turing Award. You know, one example that might be world-changing if it happens is, if any one of our teams from our very first challenge actually succeeds. Because that is about creating a new technological paradigm to treat virus infections. That's something that is really striking me in the sense of, we apparently just gave up on the fact and don't even notice it, that we are typically sick for half the year in the cold season due to some kind of virus infection. And we do not have the tools in antiviral therapeutics as we have with bacterial infections where we're just used to having broad-spectrum antibiotics, but we don't have that same thing for viruses.

We're just used to being sick, having a cold, having the flu. Not to speak of pandemics. And if we could solve that, really make those new technologies in that field successful, we would not only be able to avoid future pandemics but also do away with the flu, with the cold season.

Not suffer from not being at full performance for a couple of weeks in winter. And I think that would be something that might really be world-changing.

Eric: All right. Jano Costard, thank you so much. And I'll stop the recording there.

Jano: Thank you.

Subscribe now

Share

I met Alex last month at the ARIA Summit. At the Summit, I got to spend several days talking with ARIA staff, incoming PDs, and ARIA ‘Creators’ — ARIA’s word for R&D contractors. My discussions with the ARIA creators and incoming PDs who read FreakTakes were particularly informative. In these discussions, we were able to quickly get into the weeds on ideas explored on FreakTakes that these researchers had already read, considered, and about which they had practical thoughts they wanted to discuss.

My brief interaction with Alex was one example. Alex attended a session at the Summit in which we discussed not just the possibilities for BBNs in the ARIA ecosystem, but the practical financial problems BBNs might encounter. Several days after the session, Alex reached out because he had drafted a blog post outlining his understanding of the financial problems and several concrete ways they might be overcome.

I loved Alex’s framing of the problem and shared his excitement to move the brainstorm into a public forum. I quickly shared a summary of Tom Kalil’s thoughts on the four biggest financial hurdles great BBN founders would need to deal with, and, with that, told Alex I’d be happy to hand over the reins to FreakTakes for the day.

I’ll let Alex take it from here!

Subscribe now

Share

Wanted: New Instruments to Fund BBNs

By: Alex Obadia

BBNs are entities that provide services critical to R&D breakthroughs. The acronym ‘BBN’ (coined in this blog!) comes from a famous example — Bolt, Beranek, & Newman, which was instrumental to the creation of ARPAnet. BBNs are often compared to Focused Research Organizations (FROs) because they’d attract similar people. But while FROs require significant funding, BBNs would be the consulting-ish approach to FROs, contributing to R&D by servicing others in need of R&D support.

Like FROs, BBNs do not fit into purely the academic funding bucket, nor do they fit into the VC funding bucket as they’re not aiming to make heaps and heaps of money. Rather, they want to do impactful R&D in an organization that can stand on its own. The closest type of org that I use to conceptualize BBNs is simply a traditional business, but with a focus on frontier tech and R&D, which is not traditional — hence the need for a new name for them!

In a sense, BBNs are a more realistic FRO — one where someone can get started building ~today instead of looking for altruistic funding. One (imperfect) analogy to think of them: FROs are like venture-backed startups, BBNs are bootstrapped startups.

At last week’s ARIA Summit, I attended a workshop on the topic of BBNs led by Eric. While the conversation was initially focused on unearthing what BBN-shaped problems exist out there, it quickly moved to a more “horizontal” focus on whether there are any organizational or coordination innovations that could enable more BBNs — much like the YC SAFE was a coordination technology that allowed early stage startups to raise unpriced rounds easily on standardized founder friendly~ish terms. (And similarly, their series A template).

A few ideas flew around, such as the possibilities of new legal entities like Asterisk Labs’ cooperative model or a standardized university grant contract to make it easier to fund academic researchers rather than the lengthy back & forth with universities on terms — sometimes creating a period of multiple months between getting the “yes” on funding to getting any money.

One idea that stuck with me — and hence why I am writing this post — is the idea of an R&D lending facility [1]. While BBNs seek sustainability (i.e., to make more money than they spend), just like other businesses, they have capital needs that sometimes do not match their inflows from contracts and grants. Four foreseeable categories of problem that might stand between exciting BBNs being founded or continuing to exist include:

• (Temporary) Cash Flow Problems. The BBN has won some contracts/grants, but will have a cash flow problem for at least several months.

• Startup Capital Needs. Some BBNs might be very solid operations if somebody could buy them some expensive equipment they need to get started. E.g., a BBN that brings in contracts via a CRO-style service might need a $500k mass spectrometer to get going.

• Rainy Day Fund. Many BBNs might require rainy day funds in the case that contracts or grants in some area temporarily dry up, for one reason or another.

• Risk Capital Needs. Just about all BBN founders could use a more robust risk capital budget to pursue their ambitious technical visions. For a large percentage of existing BBNs, something like ~30% of their budget existing as undirected funds for ambitious investigations or core technology development could make them something like twice as ambitious. [2]

I can already hear you say, “We have SME loans already.” That’s true. Yet it turns out that today’s system may not be fully equipped to lend to this type of org. [3]

And so the question becomes: Do there exist 3-4 key financial products that are widely needed among budding BBNs and require an R&D-knowledgeable lender to price risk (and therefore premiums) appropriately? [4] If the answer is yes, and if we can spec out those products, it could allow us to fund BBNs outside of solely philanthropy, and to potentially create a set of new financial products (R&D bonds anyone?), that can be bought by the public, allowing a thousand BBNs to bloom.

Once upon a time, dividend-style arrangements fueled the ambitious building and expansion of things like canals and electric utilities in early America. While times and the state of finance have changed, the lessons of these groups in financing work that was ambitious might be evergreen.

To get the brainstorm started, Eric and I put together some concrete ideas, past experiments, and additional context in the footnotes. Have a look! Eric is hoping to figure out what it might look like to raise funds and pilot solutions to each of these problems, and he's eager for your input.

If you’re a would-be BBN who could use some of this funding, a BBN who wishes these products had existed when it started, a funder interested in piloting solutions to these problems, a lender who groks deep-tech risk and has a good idea of what these products could look like, or are simply enthusiastic about the topic, Eric and I would love to hear from you! Reach out at gilliam@renphil.org or on Twitter.

Thanks to Eric for letting me guest post on his blog and helping shape the post, and to Tom Kalil, Ilan Gur, and Tom Milton for their suggestions and reviews!

Notes

[1] From Ilan Gur, the CEO of ARIA, founding PM at ARPA-E, and founder of Activate:

We prototyped this at Activate — we called it a “working capital fund” for early science-based startups. The implication for this sort of working capital fund is much bigger than BBNs — it’s any research based startup that is pre-venture or not venture aligned, and especially low risk capital for those that have been awarded a grant from a reputable org (that pays in arrears) — that’s one of the big reasons we did this at Activate i.e. because our fellows were winning lots of grants but didn’t have working capital because they hadn’t raised yet.

[2] From Tom Milton, CEO of Amodo Design. Amodo is a BBN-style org doing exceptional bioengineering and electrical engineering work for ARIA. Tom says:

Today, the paths for this are A) retained earnings/profits, B) traditional external funding, and C) sometimes raising VC funds. Some examples: A) Argon&Co used its retained earnings to develop its own IP and spends ~30% of their time running their own projects, B) applying to an ARIA funding call, C) Cambridge Consultants launching a $10m spinout fund that invests exclusively in the technologies they’ve developed in-house

[3] From Eric, purveyor of FreakTakes:

I view the question of whether SME loans can be used to found new BBNs to be a hole in our knowledge; it’s the kind of hole we hope this piece might help fill. To this point, no BBN founders I’ve come across (that I can recall) have leaned on SME loans. Whether this has been because a founder did not wish to put personal assets at risk or there did not exist a lender technically savvy enough to price the deal appropriately, these instruments have simply not yet been an enabler of new BBNs thus far. If there’s someone out there eager to help us figure out how to use existing instruments/loans differently to act as enablers for BBNs, that’s great! E.g. I can imagine some Will Manidis-type, obsessed with bespoke banking stuff, maybe having an idea.

[4] From Tom Kalil, CEO of Renaissance Philanthropy. The following are some of Tom’s (paraphrased) suggestions to the problems listed above, to get your creative juices flowing:

• Revenue-Based Finance. Instead of fixed payments that may hamper a young BBN or prove too risky, loans are repaid as a small percentage of monthly revenue. This aligns the incentives of the lender and the BBN — the lender obtains revenue if the BBN does. These loans would often include a grace period to allow the BBN to find its footing. There may also be some reasonable repayment cap (e.g. 5X the principal) or time horizon in which payments are no longer owed (e.g. 8 years), depending on what is desirable for both founders and investors.

• Low-Interest Loan. A below-market rate, low-interest loan from a philanthropy or government agency. The purpose of the loan would be to help fund one-time, start-up costs involved in creating a new BBN or a new R&D capability within a BBN. While this pool of capital would not achieve the financial returns a private market investor might look for, it’s an exciting way to stretch a pool of philanthropic capital much further than it might go as individual grants. Given a BBN’s goal to both pursue ambitious work and substantially offset their costs in the process, low or no-interest loans from philanthropic entities might be a natural, capital-efficient instrument to catalyze new BBNs.

• Revolving Door Loan Fund. This instrument would help solve the 'grant-to-cash' gap. A BBN may win a major grant, but the first payment is months away. This fund provides immediate access to capital to cover salaries, equipment, a lease, and other expenses. When the grant money arrives, the BBN repays the loan, and that capital is “revolved” back into the fund for the next BBN facing a similar problem. This instrument acts as a crucial liquidity bridge, smoothing out the lumpy reality of R&D funding.

Subscribe now

Share

Some time in 2023, Tom Kalil told me he thought it would be a good idea to carve out a chunk of time and get to work on some ARPA project histories. The ARPA model was proliferating, and Tom felt these pieces might find a ready audience of ARPA emulators and fans eager to make use of the actionable information. Tom, who was then at Schmidt Sciences and is now President of Renaissance Philanthropy, has good taste. When he suggests something, I’ve learned you should listen. So I trusted him and threw myself into the work.

If any of you have enjoyed this series and wondered how the sausage is made, “How does one figure out how a 60-year-old R&D project — with no book written about it — was managed?” The answer is often: oral histories. The documentation that enabled many of these histories came to be because, at some point, committed historians sat down with a set of DARPA Directors, officer directors, staff, program managers, and funded researchers to record interviews with them on the practical details of their work.

Usually, I’m just a consumer of these oral histories. In today’s piece, in a departure from my usual role, I get to deliver you all an oral history — one I think is ideally suited to FreakTakes readers. It’s an oral history with possibly the best guest I could have asked for: Ilan Gur. Ilan is the founding CEO of the UK’s new Advanced Research and Invention Agency (ARIA). And, better yet, he’s a huge metascience nerd! The two-hour interview, recorded in Berkeley, attempts to unpack the metascience experiment that is ARIA.

I can’t thank Asimov Press enough for funding the interview and facilitating its recording. The full audio, video, and transcript are here on FreakTakes. And I also worked with Asimov to write up a much snappier, abridged version of the interview. You can find that piece in Asimov Press today.

Enjoy!

Share

Subscribe now

If you prefer to listen to podcasts on Spotify, the Spotify version of this interview can be found here.

And since the text formatting for podcast transcripts on Substack can be annoying, I’ve also published the written transcript as its own piece, in addition to including it below.

Eric: Ilan, the CEO of the [UK] Advanced Research and Invention Agency. Thank you so much for being here and doing the Asimov interview today.

Ilan: Thanks for having me. I don’t think I have ever been interviewed by someone who geeks out about the same things I do as much as you.

Eric: Well that’s exhilarating, because this is my first interview. We’ll see how it goes! [Laughter]

First question. We’ll start with a fun one. As grandiose as it sounds, ARIA’s mandate is to aim big and try to create technologies as important as the ARPANET or spark new fields the way the Rockefeller Foundation did with molecular biology. Do you consciously think about that to yourself — that you have to be or find the next Warren Weaver or JCR Licklider? Does that keep you up at night?

Ilan: [Laughter] What keeps me and our team up at night is actually having a massive enough impact. Do we think about trying to be or find the next Weaver or Licklider? Probably not. I use an analogy often of being a catalyst or an enzyme. Whether it's me or our team, we're one thing you introduce into the system that hopefully mobilizes a bunch of other stuff to make an impact.

Eric: In that case, if you think about being a catalyst, what makes a reaction big enough?

Ilan: That’s a great question. You’ll appreciate this: when we have program directors that come into ARIA, some of the pre-reads we give them before they start are interviews with Bob Taylor or stories from DARPA or otherwise. And the main thing is to try and get in their heads, “What does a win look like?” For a lot of places if the research you fund leads to a new product that makes an impact, that's a big win. I often say, “That would be a loss for ARIA.” Because what we really need to do is catalyze something that is bigger than a product, bigger than a company. It should essentially be like a movement, an entirely new technology platform that didn't exist, an entirely new industry that didn't exist. And I think the way you catalyze that can be really different for different people. So if I think about our program directors, they're all different. They all have different superpowers. Just like if you think about Licklider...I mean you're a history geek, right? So just like if you think about Licklider and Bob Taylor, right, you could talk about them each as having catalyzed this massive transformation, but in very different ways.

Eric: Are there any reactions that have been catalyzed in the past 10 or 20 years or so where you look at them and you say “Wow that's the kind of thing we should be going for”? I’m definitely guilty of thinking too far in the past on some of these things, so I’d love to hear your examples.

Ilan: One of our advisors for ARIA is Aslem Turechi, one of the founders of BioNTech. And so you think about their story of having sort of worked so hard on mRNA-based vaccines and systems — initially for cancer. And then you think of Dan Wattendorf at DARPA who basically started the program thinking about vaccines for pandemics. All of that was in the mix, and then all of a sudden you had the pandemic event that actually turned it all into the world-changing impact. So I think that's such an easy one to go to. I also want us to be thinking differently at ARIA around how those transformations might take place.

Interestingly, for example, you think of ASML as a company. But it really is a company that manifests, in that one company, a massive transformation in the field, in terms of UV lithography. Especially with ARIA being something based in the UK...you can't be picky in terms of how you're gonna get that world changing of a transformation within an ecosystem that is bound like the UK. I mean, we’re working globally, but I think ASML is such a compelling example of, “Out of some fodder and set of reactants, back to the catalyst, something catalyzed this massively transformative company and then field shift.”

Eric: Do you have any program in the current cohort that you think, like, their way of scaling to impact might be in an ASML-shaped box?

Ilan: That’s a great question. I think it may be for...any of them. [Laughter] In the sense that, and we may talk about this more later on, one of the big things about ARIA that we're trying to build in...that maybe differentiates it from, say, ARPA agencies or DARPA (which started seventy years ago) is recognizing just how important entrepreneurship and startups are to cutting-edge research in today's world. Our bet with ARIA is that probably the biggest driving force for impact will be through entrepreneurs and entrepreneurship, in some way. Whether you end up catalyzing a whole ecosystem of entrepreneurs and different companies, or entrepreneurs that are focused on government, or whatever else. Or what emerges from that is one singularity of a company that changes the world, or like an “Ozempic moment.” Again, I don’t think we can be picky. I think you could imagine that happening in just about any of our spaces. For example, I think about neurotech, I think about programmable plants, and certainly in robotics. It's just a question of do we get lucky and nucleate that right thing at the right time.

Eric: On that point. So I read your House of Lords testimony and a lot of things of that sort…

[Laughter]

Eric: I’d love to parrot back to you how I understand ARIA, from those materials. You can tell me what’s right, what’s wrong, what I need to say differently, etc. Also, if you could use examples from some of the current programs, that would be great. I’m sure there are many listeners who may have seen a program announcement, but that’s all they know. So as I understand it: ARIA programs must be 1) “big if true” and 2) differentiated. You don't want to fund programs that somebody else in the ecosystem would fund otherwise. And while each project doesn't have to succeed — the vast majority won't in the traditional sense — they should all succeed in materially changing the conversation regarding what's possible in a given space. In terms of your mandate in the UK — in the long term, these programs are meant to help drive economic output, quality of life improvements, health benefits, etc. Is that more or less accurate, how you'd conceptualize it? If someone says, “Isn’t that what deep tech VCs do?” what would you say?

Ilan: First of all, is that right? I think that’s right. We're meant to take bold bets that kind of amplify different parts of the research system in new ways. I think about our programs as funding like a constellation of teams that can lead to massively transformative outcomes at the intersection of quality of life and economic growth — not just for the UK, but for the world. I think that ‘for the world’ is really about the scale of ambition that we are talking about.

You were asking, “How’s that different from a VC?”

Eric: If somebody says, “How’s that different from a VC? Or “...than if the NIH spins up a new study section?” Or something of that sort.

Ilan: I think those are two very different questions. Let’s talk about VCs. There are a lot of VCs who are just in it to make money...but in terms of the mindset and the motivations...a lot of VCs, I think, do see venture capital as a platform to change the trajectory of the future and world in positive ways. I think the big difference with VC is — when talking about the idea of ‘big-if-true’ — there’s a sense of scale, right? You have to think about the scale of “how big is ‘big’?” — and how speculative are you willing to get on the ‘if-true’?

In talking to venture capitalist friends about what I do and what we do at ARIA, sometimes you hear this metaphor around startups and VCs: “surfing a wave.”

That’s one way to think about this. What does a VC do? A VC is meant to be like a surfer in the water, to look out at the wave set that’s coming and ask themselves, “Oh, okay, is that wave forming? Is that going to be a really big one?” The size of the wave is, kind of, how big of a market they think might emerge. And the bet they’re trying to make is, “I bet that’s a big enough wave.” And then they’re trying to bet on the timing, like, “Okay, this is when I should grab a surfboard and start paddling.” And you can think of the surfboards as the companies they’re investing in.

I think the big difference [between ARIA and VC] is, that’s being very reactive to the system. You’re trying to find the trends you can hop on and surf, so that you can maximize the ROI of your investment in a very concrete, direct way.

I see our job as more of putting the energy in to create the wave. A venture capitalist can only fund into a thesis once there’s enough momentum where they can start reading, “I see the market forming, I think there are technologies that can work here.” And most importantly, “There are companies.” Like, most VCs are waiting for companies to come find money. So there need to be enough entrepreneurs and companies doing that.

If you look at the spaces that we’ve carved out [at ARIA], we’ve said, “Right now everyone looks out and just sees flat water, but we believe that by pushing a little bit here or there we can start to build a wave.” And if we can catalyze the formation of a wave that’s big enough in one of these spaces, then actually that’s going to catalyze all the VCs to want to jump on that wave and to want to invest in companies. Then you get a win-win and, ultimately, change the world. [Laughter]

Eric: And so…

Ilan: Was that helpful, that analogy? I probably went a little too much on that.

Eric: No no no, it was great! So I have some questions. Sometimes, in terms of creating the wave...I’m sure there are times where you put a certain amount of money in a space, and there's enough teams out there that already care about this and have the proper background and the right incentives to just throw themselves into it once the capital is there. And then there's probably other cases where more legwork is necessary. Because that's not quite what a university would do — or usually a startup would do X task, but this isn’t quite a VC-scale market. How do you approach those two different types of areas?

Ilan: There’s so much we can talk about here. I’m glad this is a long form interview, because we’ll have time to get into a bunch of this. Part of this has to do with how we organize what we focus on, and maybe we’ll come back to that. I think it’s worth just talking about an example here. What came to mind as we were talking about the wave and VCs, I was thinking about one of our program directors, Angie [Burnett]. This might be particularly interesting to this audience. Angie’s a plant physiologist by background, had worked in academia, a national lab, she worked for the UN for a little bit. She came into ARIA initially thinking about food security and what we could do in food security. Fast forward — where did she end up with her opportunity space, which is what we call our focus areas, and then the program that she launched? The opportunity space that she launched is called Programmable Plants.

We tend to think for all of these [opportunity spaces], there’s some insight there. For us, an opportunity space needs to be, like you said, big-if-true. And we need to be able to make an argument relative to the potential impact. So if it’s highly consequential for society and relative to that impact you can argue it’s underexplored...you know, not enough funding or not enough of the right type of ideas or bets being made in that space. And that for some reason, it’s ripe for transformation. That defines a very big [opportunity] space that we are now going to start working in — and basically buying options through the early research that we’re funding to learn more and see if we find something that could become one of those big waves.

Angie's insight on this space was that if you just think from first principles, when you look at some of the biggest problems and opportunities we have in the world, obviously, agriculture is one of them. You think about food security, you think about climate change. The tough part about these segments is that they're massive problems, right? You need to talk in terms of gigatons of CO2 or carbon, in either case. And her view is that, actually, you can think of plants as a technology platform, and plants are one of the few technology platforms we know of that, actually, we have a system that operates at that scale — of gigatons of carbon.

And what do I mean by technology platform? A plant is a piece of hardware. It has certain functionality. Interestingly, in many plants, like you go to the store and you buy a cob of corn, we have actually engineered the functionality of that plant over many years. And you have distribution channels and the ability to deliver these things at a massive global scale, commensurate with those problems — food security, climate, etc.

The problem is...it's like a really shitty technology platform, right? We're very limited in what functionality we can build in. Instead of 18 months for a new iPhone, you take 18 years to get a new crop that's actually viable. Interestingly, if you talk to most investors, they will say, “Well, in the ag space...you've got the top [incumbent] ag companies and [given the incumbents and structure] it's basically the hardest, most entrenched space to innovate in.” It's just not a great place to invest. Massive capital, massive entrenched interests, there’s regulations, etc.

The bet that we're making is that actually...on first principles, when you look at what’s happening in synthetic biology, when you look at the problems and the forcing functions that are coming, requiring a change to how we do agriculture...there’s a pretty strong argument to be made that, sometime in the next few decades, there will be a shift in how we approach global agriculture. You can think about it as something like, “There will be an ASML-type company that completely disrupts the big five ag companies in the world, and does things radically differently.”

Eric: And what's Angie's job over the course of five years? And I guess also tie in, what is a program director? And how important is it that they have a point of view? And does that come out as you're picking them? ...I asked you a lot of questions.

Ilan: No, that’s fine. Angie’s job was first — together with me and the cohort we had — beating up the question of, “How do you define a space that’s interesting and in line with this idea of ‘plants as a technology platform that’s underappreciated’?” Synthetic biology has been this massive vector of technology progress, which is not being leveraged for plants and ag to anywhere near the extent it’s being leveraged for human health. So now there’s like this arbitrage [opportunity]: “We’re making all this progress in synbio, we have this massive problem opportunity in ag.” But they’re not really connecting the dots.

Eric: And how many years behind do people perceive ag as, in terms of the cutting edge in synbio? I’ve heard 10 or 20 years, but you’d have a better...

Ilan: Well, let’s fast forward. So, what is Angie’s job? Her first job is to lay out that space. The next job was to say, “Okay, what’s a funding program we’re going to launch?”

She had a bunch of different ideas. Where she ended up is a program that we’ve now launched — and we’ve just announced the people and groups we’re funding. That program is called Synthetic Plants. [See this link for more information.]

And the idea was, if you look at the real frontier edge of synthetic biology, and we're talking about things like de novo genome synthesis, like actually thinking about synthetic organelles. You have, in the UK and other parts of the world, really amazing progress happening. But when you go and talk to those researchers working in those areas, and you say, “Well, we're thinking about doing a plant program,” the reaction Angie got from all of them was, “Why? There's so much to be done in mammalian systems, we're making all this progress...like, plants are hard! They have multiple copies of the genome, it's impossible to figure out how to transform them, it’s XYZ...”

And instead of getting discouraged by that, Angie got more excited by that. She said, “Well, wait a second, let me tell you a little bit about the potential impact we could make if you start working on plants.” Long story short, she held a workshop. And I got to be at this workshop, where she brought together a combination of top synbio people, top ag people in the UK and otherwise. There were probably 50 people in the room. One of the questions I asked was, “How many other people in the room did you know when you walked in?” And I had said [gesturing his hand upward], “Raise your hand if you only knew one person, raise your hand if you only knew X...”

And I think most people knew like five out of the 50...and none of the synbio people had talked to any of the plant people.

What we’ve found is, now, there are all these new collaborations. We have synbio folks who are basically saying, “I’m shifting my work to plants because I’m just convinced this is the most important thing I can do.”

Eric: And when you say ‘folks,’ how many are thinking of shifting their work to plants?

Ilan: Well, you’ll see how many synbio people we fund in this program when it gets announced. But I know of at least two specific cases Angie’s mentioned where, when she started the process, the person was so averse to the idea of doing anything in plants and thought it was a waste of time...who have built really meaningful collaborations and — whether they get our funding or not — are going to start working on that. So, it’s cool.

Eric: When is high variance [in reactions] among the adjacent experts exciting? When is it less exciting?

Ilan: Oh yeah. How do you know when people just say, “Oh, that's a horrible idea.” How do you know when it actually is a horrible idea? [Laughter]

Yeah, I mean this is an interesting thing. You talk about variance, right? Someone from DARPA once shared this with me, and it's a framework that I really love, which is...if you think about the role DARPA plays, more important than anything else, it is a mechanism to increase variance in the system.

It’s actually something I think a lot about for ARIA. The idea there is, more than anything else, ARIA was created to do things differently. The idea is: do things differently, and by doing things differently, by shifting the modality of research, you can get different outcomes. We can spend a lot of time coming back and talking about that.

But if you want to do things differently, first of all, you need people who see things differently. It’s one of the reasons the PD [program director] model is very helpful. But secondly, you have to break what is, I think, a pretty strong and homogeneous sort of truism of how we (largely governments) fund research. Which is something like, “We’re pretty good at funding the well-known disciplines. And we’re pretty good at funding the obvious problems or opportunities.” But that's all very linear, like, “More funding for biology, or synthetic biology, or semiconductors, because we see what's happening geopolitically.”

If you want to do something different, you need to find some way to cut and slice through that in a different way. Interestingly, the ARPA model says, “Okay, you find a program director, like Angie. You have them look across the system and say, ‘Well, what’s an actual thesis of something new that we could do, that’s not a single discipline? Maybe it is, but it cuts across TRLs [Technology Readiness Levels].” That’s the other thing. People sometimes ask, “Is this low TRL or high TRL?” It’s like, no. We’re mixing these modes in new ways.

Eric: You’re ambitious and applied...

Ilan: Yeah, and you have sort of a spectrum of portfolio. What that means is, what we end up funding on the backend of one of these theses we’ve developed, is going to be a very different set of people with a very different set of incentives — in terms of what they are going to do — than would otherwise get created.

And the variance is...if everyone thought it was a good idea, we would be introducing no variance into the system. If no one thought it was a good idea, we probably shouldn’t do it because it probably is not a good idea. But if you get spiky reactions to anything you’re doing, it’s the sort of thing where...okay, you have some minority that actually think this could work. But in a consensus panel, peer review, that minority voice would probably not win out or be there. So it would never happen otherwise. And, actually, I think the hardest job of a program director is to be able to actually face the criticism of so many people saying, “You don’t know what you’re talking about!”

Eric: You need to have courage to live with your bet, for years on end...

Ilan: You need to have courage to live with your bet, yeah.

Eric: And so some people...

Ilan: Wait, I’ve been talking a lot! I feel like it’ll be good for this audience...as we’ve been talking, what comes up for you as you think about some of the history that you’ve learned about for program managers at DARPA, variance, etc.?

Eric: So, I guess what I’ve been really impressed by, in working with the ARIA PDs, and ARIA in general, is when I read about early ARPA, 1960s and 1970s in particular, it seems like they really did trust the PMs to have an opinion. Or to say, “Here’s why this contractor’s best. You have to see it [the vision], and here’s why there’s only one group to do X thing.”

It seems like ARIA has given its PDs a long enough leash to have the point of view and follow it wherever it may lead. When I talk to folks at a place like DARPA, or something, I often get the sense that the PMs do have a strong point of view, but it’s an older organization. Like, it’s a 60 year old government agency. I think a part of the magic of DARPA is, so many government organizations only get more bureaucratic over time. They’ve [DARPA] been phenomenal at, like every five or ten years, finding ways to clear out a lot of the red tape, scar tissue, etc. But it does still build up a little bit.

So it’s been really cool. Something I try to impress upon people is that 2025 DARPA is not 1965 DARPA. You need to pursue different programs and do different things [today]. And when people ask me, “Well, do we have an early ARPA [today]?” Before ARIA, I guess I’d say, “Maybe OpenPhil.” OpenPhil lets people be free like this. But as ARIA has come into existence, it seems like you all live in the vein of early ARPA much more closely than anyone else. But, saying that, I also know you want to do something new...

Ilan: No, it’s interesting though, because I think this relates to the variance point. I’m trying to keep this dear to ARIA in the early days. It’s not about, “Did we find the right bet?” Because actually, given the variance point, in the early days, it’s not going to be obvious which is the right bet.

So it’s really about, “Are we being true to a commitment to increasing the variance of what gets funded?” And then, “Are we going to have the muscle to see the right bets emerge from that work?”

I actually had a [ARIA] board member recently say, “Isn’t one of our biggest risks right now that we created the wrong programs — like we’re funding the wrong programs?” And I said, “No, I don’t think that’s a risk at all, in part because we decided to create these opportunity spaces.” The whole idea behind an opportunity space for us, in these focus areas, is that our whole job is to take speculative bets. What that means is we can’t know upfront whether this is going to lead to impact. We can’t engineer that. The only way we get it wrong is if we take a speculative bet in an unproductive direction. So the point of an opportunity space is to say, “Let’s define what we think is a productive direction.”

An opportunity space is going to have certain beliefs that bound it. And the point is, if you can get yourself to believe these things...e.g. “Pressure on food security and climate is going to force us to do some things different in agriculture,” “Synthetic biology is moving...” Right? If you believe those things, you have to imagine there is the potential for enormous value — economic value and social impact — that can happen in this space. And now that we’ve bound that space and set that opportunity space, now we can take steps, big and small, and make bets in that area without having to worry, like, “Is synthetic plants the right program?” I’m not worried about it at all. Because we’re making a bet in a direction that’s productive.

The only thing we now have to worry about is that we don’t do what we were made to do. What makes ARIA unique — and what’s meant to make a place like DARPA unique, certainly in the early DARPA days — is that you make a bet, you take a step, and then you learn something, and you can pivot and say, “Oh! Actually it’s this 5% of what we’ve done so far that is starting to feel really valuable. Let’s minimize the rest and 10X that 5%.” That’s the thing that...I think that’ll be one of the big muscles we have to create for ARIA in this next phase.

Eric: So, you’ve moved to the UK, and ARIA is...

Ilan: I’m sorry, I have to stop! I’m like, I’m sitting here wondering if people watching this are excited about it. It’s just like — the idea that there’s now a community that’s excited to hear this conversation — that’s something I’ve been searching for for a while.

Eric: [Excited laughter] Okay, so, in that case...

Ilan: How many subscribers do you have?

Eric: Like 4,000. But there’s people like Matt Clancy who have like 15,000, or something like that. [Correction: Matt has over 18,000.]

Ilan: But those are high impact people.

Eric: Yes, I am continually wowed by the subscribers I have...and when I write something, the email outreach I get. It’s very much a dream for someone like me who...I wish I could be a great researcher [gestures at Ilan], but it was not in the cards for me.

Ilan: [Laughter] I wish I could be a great researcher too. Wasn’t in the cards for me. Definitely more of a Bob Taylor than a Licklider.

Eric: So you’ve been doing this for so much longer than many people. A lot of people have maybe gotten interested in this space in the past 5–6 years or something.

Ilan: Wait, what’s “this space”?

Eric: We’ll call it this “applied metascience” space. People who want to really consciously experiment with how you build R&D ecosystems differently, or research labs differently. In many ways, you’ve been doing this for possibly the majority of your professional career. Can you tell people about the different stops you’ve taken? Maybe different things you’ve learned along the way? Different gripes you’ve picked up? ... [I imagine] you only throw yourself into this if there’s stuff to be fixed and you have ideas.

Ilan: Yeah, I mean, I sometimes describe it as, like, I got thrown into this just because I felt like a misfit in all of the different environments to do research. And we can talk more about that. But I think that is a motivating factor, right? What motivates me, and probably what motivates most of the folks who would listen to a podcast like this, is just realizing that you can take the things we’re learning at the cutting edge of science and discovery, and turn them into stuff that’s useful, real, and awesome.

I mean I had — actually, you’ll probably appreciate this story — I was one of a rare set of people who was an undergraduate major in material science. Material science is kind of a weird field that most people, at least when I went to college, didn’t really know about.

Eric: They’d pick it up as a masters...

Ilan: Yeah, exactly. I think there were maybe like five people at Berkeley who came into the undergrad material science program. But the reason I got into material science...I grew up in Pittsburgh and went to a public school, which just happened to have a couple of phenomenal science teachers. And one of them encouraged me in 10th grade to put in an application for this program that CMU [Carnegie Mellon] did, where they would host high school students in a lab.

And I still remember, like, they gave you a questionnaire, and it was like, “What are your interests?” And I was in 10th grade, you know, I was not a polymath who was way ahead of their time. So my interests are like, ”I like physics [shrugs]...maybe chemistry?” You know, you just say pretty banal, normal stuff. Anyway, they paired me with a material science lab. And I remember showing up and saying, “Why did I get the oddball thing? Why couldn’t I just be paired with a normal lab?” And I didn’t think it was going to be cool or exciting.

The project — and this is the piece that I think you’ll appreciate — the project that I got paired with was a lab that worked in magnetics. What they wanted to do was come up with a new magnetic storage medium. At the time, you know, a hard drive was, what, 16 megabytes? You didn’t have flash drives, etc. And so the question was like, “How could you get much higher density?” So what they were doing is they were taking a material called MCM 41 — this is the part you’ll love — MCM 41 was one of a class of what are called molecular sieves. It’s a material that has a microporous structure, a lot of core space, sort of a zeolite material used as a molecular sieve/filter/etc. Basically it had like honeycomb pockets. The project was, through different means — chemically, sputtering, etc. — to fill these honeycomb pores with magnetic material, with some iron compound. And then the idea is that the ceramic honeycomb borders would be the separations between domains of a magnetic storage medium.

I still remember, at the time — you’d go around and say things like, “You could imagine putting a gigabyte on something the size of your hand!” But it was mind blowing! Just the whole idea that you have these nano honeycombs, and then you’d have a head that would sort of scan them.

Eric: Did you believe it? Or did you feel like that was crazy?

Ilan: No, I totally believed it! And the reason you’ll like the story is because MCM 41 stands for “Mobile Corporation Material #41.” So I’m still waiting for you to do a post — maybe you’ve done one and I’ve missed it — on all of the amazing discoveries, fundamental and otherwise, that came out of the oil and gas research labs. Mobile Corporation did all this incredible material science work. And, of course, they named all the compounds, like, “MCM 3, MCM 5.” So MCM 41 was, like, something everybody knows about...or at the time did!

Eric: That actually would be an interesting post too. Because I was just yelling [injecting historical stories into random conversations] at some people on Twitter the other day about the origins of TI [Texas Instruments], as just like, one of the early oil and gas firms that had a ton of early MIT ties. They just got very good at making their own instrumentation. And that’s a pretty high margin business. It’s a little less speculative. And they just like...I forget if it [TI] was a spinoff or if the firm [the original oil and gas firm] entirely pivoted. But yeah, maybe I should. If you have ideas!

Ilan: Oh yeah! Materials, lithium ion batteries, came out of an oil and gas lab. A bunch of solar cell work. I think there’s something cool there, but...

Videographer: Can we take a quick break?

[Camera Break]

Ilan: You were asking about the trajectory of ARIA?

Eric: Yeah, it would be good if we could discuss how...you were at ARPA-E. What in the world is Activate? All of these things, probably like, being dissatisfied in different ways. Or being happy with what you did, but dissatisfied drove you...?

Ilan: I think there’s a story of navigating, you know, continuing to basically try and do bigger and bigger metascience experiments, if you call it that. There’s a version of that which is, like, me being disillusioned by different institutional structures, or different silos, and how we do research. But maybe a more positive or interesting version is that, as I think over that time, is...I talked about being at CMU. When I did that research project at CMU, what flipped on for me was this idea of, “Holy shit, the wonder of the ways that we can take scientific discoveries and turn them into massive impact!” For me, the ultimate [thing] at the time — and this probably went all the way through my undergrad and into my PhD — the most valued currency for me in terms of changing the world with science was the ideas. And the instruments and the processes and the methods. I was so obsessed and in love with that, and I saw that as the big driver.

Interestingly, I had this experience 18 months into my PhD where we had written a Science paper. We were on the cover of Forbes for this new approach to creating solar cells much cheaper, you know, II-VI nanoparticles that we were going to print like newspaper. There was all this excitement. And I spent a couple of days with some business school folks with technical backgrounds, and realized it was all BS. We actually weren’t solving the right problems.

And I became...you could think about it as being disillusioned, but actually I think it’s more [that] I recognized that the world of ideas that I was in, where that was the currency, there was something missing there, in terms of how you drive impact.

And then I ended up in startups, VC-backed startups. My mindset moved from like, “Actually, ideas are pretty cheap, you know? Ideas are pretty easy to come across.” It’s more about, “How quickly can you iterate through them?”

But I also got obsessed with currency as the currency for driving impact. Because my job running a startup was, like, “Raise money.” And you could see that the more money you could raise, the more of an opportunity you had to drive and get to faster outcomes relative to competitors, or whatever else. So I really was in this mindset of money.

And having a chance to go to ARPA-E, one of the things I found very compelling was that it was a different mode from venture capitalists. I thought venture capital was sort of too narrow of an impact model for doing early-stage science, especially for industrial markets. But I also found compelling, like, “Oh wow. We’re going to be the biggest Skunk Works innovation funder in the US, for anything climate-related.” And I was attracted to this big money question. Money is like the driving force you have for reactions.

And now I think, through that time for me, at ARPA-E, my biggest recognition was that we had...

We had an incredible mission: to drive translational science into big impact in climate and energy. We had access to a lot of funding to do it, so we had the driving force. We had access to all these brilliant ideas, meaning every researcher who worked in any of these areas in the country would apply to our programs if we could just pick and choose the ideas. And the thing I realized was that, actually, none of those was actually the thing that mattered. What really mattered was people and institutions and incentives. And you can have all the money you want, and you can have all the ideas you want,  but if you don't have the right people in the right institutional environments with the right incentives to drive progress, you're pushing on a rope.

Eric: At ARPA-E, were you ever wowed by some set of people and institutions and incentives that you got to deal with?

Ilan: Well, in the end, I came back to...I left my startup feeling somehow like I had been misled. The idea of, “Venture Capital-backed startups are going to change the world.” We were working on batteries for EVs, industrial markets, very low margin business. Probably our timing was off. But I was like, I don’t see how we can develop the technology, get it to scale, and actually have it converge for someone like Vinod Khosla, who was like, “Either this needs to be a trillion dollar…” it was at the time billion dollar, now like, “...trillion dollar company, or it’s not worth doing.”

And so I kind of had a feeling when I went to ARPA-E, like, “Actually venture-backed startups are not the way to do this. And startups might not be the way to do this. Let’s figure out all the other institutional modes.” What I found in the end was that, to your question, of all the environments that I was funding research, the ones that felt most resonant and most productive were either the startups or the folks within academic labs that were already operating as though they were spinning out a company.

And what I realized was that the venture capital funding model has constraints, but if you don’t worry about the funding model for a second and you just think, “Well, what is a startup?” A startup is a vehicle for getting the right people into the right environment with the right incentives completely aligned with some translational research mission.

If you think about it, [for] everyone in an early-stage startup, the goal is to create value in a very concrete way. You’re probably doing something very speculative, if it’s science-based. Everyone who’s there in the early days has decided they’re going to commit the next chapter of their life to that pursuit. The incentive is clear. The alignment is clear. You often are getting people from different backgrounds, in interdisciplinary ways, to do it. So I think startups are massively valuable and important vehicles for R&D.

And the question becomes, and we can have a conversation about this, the question becomes, “How can we fund them? How can we fund them as R&D centers?” Venture capital is not a great way to do that. Some VCs actually do fund very applied R&D and take speculative bets. And I think those are companies that tend to change the world. But relative to the, like, $70 billion of US funding to applied and basic research — that number is probably dated right now — you don’t have anywhere near that going into super science-y startups.

Eric: To understand the problem...You love the shape of VC-funded startup groups — the people, environment, incentives they create — for the problems they want to attack. But there’s all sorts of problems that...

Ilan: They can do it if they see the wave coming and they’re just hopping on the wave. So when that happens, they work great. But why can’t you have the same aligned incentives, full-on commitment, before the wave even exists? And couldn’t that be a mechanism to create some of those waves?

Eric: And how do you think about making that happen…

Ilan: I have no idea.

Eric: ...at ARIA?

Ilan: Oh, at ARIA!

Eric: Yes. And of course all of my readers will know I’m biased. I’ve spent this huge amount of time writing about the best early ARPA contractors, like BBN or the CMU autonomous vehicle groups...groups that were very startup-related, but really embraced technical ambition over market size. Their constraint was that they need some amount of grants [or contracts] to fund it. But I’m sure you have all sorts of theories on how to make this work for you [ARIA] in spots where there are parts of the R&D ecosystem that could use a little bolstering — at least in terms of ARIA’s incentives.

Ilan: Yeah. So one of the things we’re doing at ARIA, which I think is probably different than DARPA, is being really open-minded and attuned to, “How do we try and fund not just the best people and ideas, but with resonance in terms of the environments and incentives where they are doing the work?”

What does that mean? That means a few things. One is that we are very comfortable with the idea that, using ARIA funding, a researcher might decide to go start a company. They might decide to go, leave their academic lab, and just do the research as an independent researcher, because that’s what they think is the right thing.

We have something interesting, which is we do these seed funding awards. I mentioned those opportunity spaces where we say, “This is just the direction that we think is fertile.” A program director will create a program, which is their thesis, and that’s where they’re going to focus a lot of energy and where we’ll focus a lot of funding initially. But then what we do is we say, “Well, we’re not the only smart people around. And it’s all about increasing variance. So why don’t we find other people and let them start seeding ideas that could lead to the massive breakthroughs...[unintelligible]”

And this is like, you know, give us a three-page application, tell us why you’re obsessed with this idea, and [why] no one else will fund it. And we’ll give you up to £500k. So just start running at it. And if you come back with something compelling, we can double down with you. [We can] either make it a bigger project or actually have it inspire a whole program.

Eric: So Jenny said something in one of her interviews. Jenny is the PD on the robotic dexterity program. She said at one , “I was almost disappointed that I didn’t get the chance to fund, like, some random person in their garage.” Because the funds are open to them. Can...

Ilan: Anything.

Eric: Can you explain what it would look like to be...like if I’m a random guy in Birmingham and I want to…?

Ilan: [Gestures at Eric] As opposed to a random guy in…

Eric: Yeah, in Chicago. [Laughter] And if I wanted to run it out of my garage, how much paperwork is involved? It seems to be a very fair [reasonable] amount.

Ilan: So this is really interesting. What we started with at ARIA is, we said, we wanted to make clear that we are completely agnostic. We wanted to fund the best people in the best environments. Apply. We don’t care if you’re at a university. Just tell us you’re obsessed with the idea, what the idea is, and show us that with our funds you can get access to the equipment you need to make it work. And we wanted to be really agnostic.

Now, there’s a piece of this, which is we have a brilliant, awesome, founding CFO for ARIA. Her name is Antonia [Jenkinson]. She and her team, I went to them and I said, “This is a little hard to do as a government agency. You got to do due diligence on these people. How are they using their money?” And they said, “Yeah, but this is what we’re built to do, so we’ll just figure out how to do it.”

Eric: And is it true that she walks around with the [ARIA] founding mandate in her pocket?

Ilan: [Laughter] She has been known to carry the ARIA Act around, which I love. So we started and said, “Yes, we can fund this [alternative applicants].” And seeds were a three-page application. We would tell you within three weeks whether you got the funding.

Eric: Three pages? Like...[makes a show of counting out three pages]

Ilan: Three pages, yeah. It was a three-page application, three weeks to funding decisions. And when we did it, we did have this. We had people apply. And then we had a few people who said...we had one person at a well-known university in the UK who said, “I’m really glad I’m getting this seed. To be honest, I don’t think I want to do this in my academic lab. For a number of reasons.” I think they were actually a postdoc. They were, sort of, earlier [career]. And they said, “What do you think about me, just like, finding and renting some space to do this work as an independent researcher?” The program director talked through it with them and said, “Actually, that makes sense. You’ll probably do a lot better. You’ll probably be more motivated.” Whatever it was. And so we just figured out how to give that person the award as an individual to do it in this way.

And we saw a few of those. Then we paused, and we said, “Well, wait a second. People are funded and excited to work on these projects.” And when we tell them we’re open to wherever you do it, they’re telling us, “Well, maybe I’ll do it some other way.” So the next time we did a seed call, as part of the questions we said, “What’s the institution you’re in now? If we award you the seed, where would you like to do the work? It doesn’t have to be the same place.” And the options were, “Will you do it in an academic environment?” You know, “my current university, another university, a company,” wherever else. And then we [included], “still undecided”.

First of all, I think we’re the only government funding agency in the world that gives that option. Secondly, the fact that there was an undecided bucket means, “We’re cool...like, we’ll talk through it.” I think we had like 23% of applicants in the next seed call say “Undecided.”

Which for me is such a big deal, because it suggests that being prompted with the question of like, “Eric, there’s a big project...it could be the most important thing you do in your life. Stop and think about, ‘What is the best environment for you to do this work and have it succeed and have a chance to change the world?’” And what we learned is like 23% of people [think], “Maybe it should be a startup, maybe this is my moment.” Or, “Maybe I should move to the UK and try to do this at a university in the UK,” which has happened now with ARIA grants. So, that was pretty cool.

Eric: Yeah…

Ilan: That felt like a...I warned you that if you get me excited about something, I’m just going to keep talking about it!

Eric: No, no, no! That was great! So I have a question [from my notes] lingering here, I don’t know where to put it. So I’ll just ask you now.

Ilan: Go ahead!

Eric: So this one comes from Tom Kalil. He said, “In what specific ways do you desire to transcend the DARPA model?” Because you’re not looking to set up some version of an American ARPA in the UK?

Ilan: I think that’s right.

Eric: There’s lessons you’re taking, but…

Ilan: Yeah, I mean look...I think DARPA is such incredible inspiration. You know what I mean. Like people argue about, “Is DARPA past its heyday?” “Is it doing good or bad?” But we have this beautiful gift, which is just the history and the reality and the myth of DARPA all wrapped in. And, honestly, if you can’t be inspired by that, like, forget you! Right? And I think that inspiration is so important and true.

We’ve touched on some of the things that I’m hoping with ARIA...ARPA, like you said, established 70 years ago in a very different time. It’s changed in many ways. I think for me, the most important things — one of them is one of the things we talked about earlier, which is having the intellectual humility to, and the eye on part of our job as just to increase variance and learn and pick up on threads that are valuable. That is really important to me.

One of the things I’ve noticed — and this is good and bad — but when our program directors...we pair them up with DARPA PMs, former and current, to give advice and whatever else. And oftentimes I will hear a DARPA PM say, “Wait, 37 performers for this program? You can’t do that. That’s not how we do things. That’s not the right way to do it.” What we’ve had the chance to do is kind of go to first principles on everything. And that’s a real case. I don’t think it’s 37. But davidad, who runs our Safeguarded AI program...for the first technical area of his program, he wants to, as quickly as possible, answer some theory questions and develop some frameworks which he thinks are best done as open source. And, actually, where that led him is we’re funding a lot of teams with small awards, to which normally people would say, “It’s not worth the management,” or “It’s not cohesive enough.” But the first principles suggest that, yeah, that’s the right thing to do there.

So, it’s kind of like having the intellectual humility to say, “Okay, I don’t know if this is exactly right. But it feels like it’s matching the first principles. And across what we’re doing, it gives us variance. We’ll learn more on the go.” I think that’s one thing that I’m hoping gets preserved at ARIA.

This other thing we’re talking about is probably the other most important one, which is...DARPA, rightfully, has a mantra of, “We don’t have our own labs. We don’t create institutions. We fund research projects.” And DARPA increasingly has done more with startups, which I think is great and one of the biggest impact vectors it has. But I think for us, for ARIA, being able to say from the beginning, “Actually, the people and the institutions matter. And there may be new institutions that we need to form — or that we catalyze forming. And we’re not going to be, sort of, scared about that,” is important.

Eric: Yeah. This is also something I think early DARPA was a bit friendlier to. They wouldn’t necessarily found an org, but they would write a pretty big check to an org that, like, didn’t seem to have an office yet or something like that.

Ilan: Was RAND DARPA?

Eric: Was RAND DARPA...

Ilan: I remember reading something that...

Eric: No, I think RAND would’ve been...I do think the USC ISI [Information Sciences Institute] was very clearly heavily early DARPA funded [from its earliest days, which helped create the organization of largely RAND alums]. They ran DARPA’s early MOSIS initiative, which was one of the first fabless [services]. That’s definitely a case of...those guys may have left RAND to form...I’d have to go double check some of my notes. But that’s a case of them [DARPA] writing [very early checks into an org] and making some promises, and it was an institution they needed.

Ilan: Yeah, it’s actually interesting! MOSIS came up recently at ARIA because, one of the programs that we’ve launched...we’re already starting to see that, within this...

[Camera Break]

Eric: You were talking about MOSIS and how it fits into one of your programs that you were talking about recently.

Ilan: Yeah, the opportunity space is Nature Computes Better. One of the theses there is that there have been lots of attempts to think about how you disrupt general computing. But it’s such an impossible thing to think about doing because of the massive supply chain and all the different forms of general compute. Suraj Bramhavar, one of our program directors, his insight was, “Actually, AI is very unique in the history of computing because you have a compute mode with a pretty narrow set of mathematical primitives, but potentially massive applicability, value, and impact. And that narrow mode of AI could be a new foothold to think about alternative types of computation — with alternative physics, alternative hardware, alternative substrates, etc.”

And the other piece of this is...when we think about, “Is there a program or opportunity space here,” one of the important questions to ask is, “What is the fundamental limit of performance that could happen in this space? And where are we now?” And one of the things we know about computation is,[even though] we’ve had Moore’s law for as long as we have, we are still orders of magnitude from the fundamental limits of computation. Whether it’s [the fundamental limit based on] Shannon’s information theory or what we see happening in nature. So the idea behind the space is, “Can we find new approaches inspired by physical systems, natural systems, other ideas that researchers have, etc. to use AI as a foothold for next gen compute?” The program that we’ve shaped aims to demonstrate that you can show, as a first step, AI training at a thousandth the cost and energy consumption than state of the art today.

And there are a bunch of really interesting systems questions around, like, “How do you even prove that in sort of a demo system?” We’ve now funded a set of teams across academia, startups, big companies, and a number of approaches that we think, in coordination, can get there. And from that work we’re already seeing, “Wow, there might be a next phase of this given some of the early excitement that we have.”

How would you take advantage of it? Maybe you need some MOSIS-like institution that is focused on this area. It’s still very early, but it’s cool because, like, you had your post on MOSIS and it actually is making a difference, being able to give us [ARIA] a catalyst of things to chew on.

Eric: And can you paint a picture of what this MOSIS would be doing? Like if a lot of these contractors are point solutions or different approaches, what would you need them [this MOSIS] to do?

Ilan: [Laughter] I don’t think I could paint a picture of that for Suraj’s program because he’s literally just in the early stages of thinking about it and I don’t think I could actually talk through it without disclosing things that he probably wouldn’t want me to. But I do think, you know, one of the things we talked about earlier was the question of, “Where are there new institutions, and how could they help ARIA?” We talked about our Creators. (DARPA talks about the people they fund as ‘performers.’) We thought that was a little odd. Actually, you might know the origin story of where ‘performers’ came from. Maybe it’s like: you want performance, so you have performers? Anyway, we call the people and institutions we fund ‘Creators.’ The idea being, the work we do at ARIA, I think you need equal doses of creativity and creation. This word ‘create’ is interesting because you can think about it in terms of creativity — which is like pie in the sky thinking — or you can think about it as like, “No! Go create something, like building!” And our view is you need both of those to come together.

Anyway, we talked about new institutions around Creators. We’ve also been thinking about new institutions to help ARIA do better. You might have seen our Activation Partners call.

Eric: Yes. Can you explain what that is?

Ilan: I can. [Laughter] This might be a divergence, but...

Eric: We can come back to it later if you prefer.

Ilan: No, let’s do it! When we think about the ARIA model as a whole, we talked about finding the opportunity spaces, being able to inject energy, catalyze these big waves — which are essentially new movements of technology...but then talent and capital that can transform the world in one of these spaces.

There is a question. We’re not DARPA. One of the things that allows DARPA to get things from early idea/technology into practical systems in the field is that they have this massive lever that is the Department of Defense budget. [Laughter]

Eric: Yeah, they have a built-in customer.

Ilan: Yeah, they have a built-in customer. And there is no bigger driving force than the procurement power of the Department of Defense, if wielded in the correct way. ARIA’s mandate is much broader. We’re not just defense, we’re actually not doing any defense right now — it’s quality of life and economic growth, broadly speaking. We don’t expect to have a built-in customer from government. There might actually be cases where government, like the NHS [National Health Service] could be an amazing partner and uptake vehicle for technologies we develop. But outside of having purchasing power as the driving force, our bet is that the biggest driving force that exists to get new ideas into the world at scale is going to be entrepreneurship.

It’s that whole idea of, “If you, as the researcher and a team of people are willing to commit their life for a stretch of time, yes, you could change the world.” We know it. We’ve seen that happen.

So, one of the questions we’re trying to grapple with is, we’ve created these opportunity spaces specifically to be places where the wave doesn’t yet exist. So, there probably aren’t a lot of startups in these spaces because venture capitalists aren’t super active in all these spaces. Maybe there are some. What can we do to increase the amount of reactants that are entrepreneurial in the opportunity spaces we have?

And we thought about a few things. When I was at ARPA-E, actually, we ran an experiment that I helped launch, which is called the technology to market program. We basically built, within a government funding agency, a business analysis/biz dev type function. The idea being, “Let’s figure out how to inject into our programs some thinking around how you translate the technologies.” DARPA does some of this.

Our view is actually, for ARIA, there are some things we can do internally...but to think about how technology’s transition to market, especially through startups and entrepreneurship, the best people in the world doing that are going to be people doing it in the field. And you might be able to attract one or two of them, here and there, into a government funding agency, but you’re not going to be the best in the world at doing that. So what we did was we put out a call, and we said, “Listen, we think there is going to be massive opportunities and value created in these spaces. It’s still early. If you are an organization that deals with venture talent creation, entrepreneurship, rapid prototyping...anything related to tech translation, transitions, and value creation...and you’re amazing at what you do, could you imagine focusing your energy on some of our spaces? And just do what you do really well, but inject more reactants into that system.”

My colleague Pippy [James], who’s our Chief Product Officer, was really the visionary behind driving this forward. Where we ended up was, we have a handful of organizations. Some of them are venture capital firms that are saying, “We will, for the first time, focus on the UK and look at opportunity creation in these spaces.” They’re doing things like running fellowship programs for scientists who want to be entrepreneurs.

Renaissance Philanthropy is another Activation Partner thinking about, “How do we create a two-sided marketplace between the technologies that start to emerge and philanthropic goals/philanthropic funding?”

Eric: A lot of people don’t think about it, philanthropic demand is demand all the same. They [philanthropists] often have particular technologies they want to bring into existence. But it’s often unclear how to solicit demand from philanthropic...the mass of philanthropies. It’s hard to understand what they want.

Ilan: Yeah, and in a lot of these spaces it’s not clear that when we’re done with our program, the commercial value is going to be right there. It may be that we’re still in a tragedy of the commons, market failure mode, and we need to band together with others to do more in those spaces. So I think that’s going to be important.

But one of the things that came to mind as we were talking earlier is one of the Activation Partners we have, I think you’ve spent time with them, is this company Amodo [Design].

Eric: Amodo, yeah! Love Tom [Milton]!

Ilan: Amodo is such an interesting story. They’re based in Sheffield. It’s a small team. It just reminds me of the things you write about! It’s a small team that, you know, they’ve been involved in a spin out of a university and they basically recognize that, like, “Man, universities don’t do a very good job of prototyping things, having a rapid iteration mindset, or understanding how to get things into a more commercial place. Why don’t we just set up a shop to do that as a service — for startups, academics, and otherwise?”

Eric: So you don’t have to shoehorn your experiments into off-the-shelf Thermo-Fisher equipment. Tom and them can make what you need.

Ilan: Exactly! “Let’s go solve it!” They’re like mechanical, electrical, prototyping fixers!

Eric: And they’re hiring!

Ilan: And they’re hiring! [Laughter] So, interestingly, they applied to be an Activation Partner, alongside...you know, our other Activation Partners are, like, Google DeepMind...but they applied. And first you look at it, and you’re like, “Oh you’re five people in a room in Sheffield?” And then when we talked to them and we saw what they were doing and the vision, it was like, “Actually, this is a massive asset for ARIA to be successful.”

And maybe it won’t just be Amodo over time, maybe we’ll find others. But already, through the arrangement that we’ve built, they’re actually doing this as a service for a bunch of Creators that we’re funding, who are saying like, “Oh my goodness, I thought it was going to take us six months to do this and I got a much better version of it in a week!” So I think that’s super cool.

Eric: And the PDs have seemed very impressed with them. Like when I was talking to some of the PDs [and other ARIA staff] about the BBN thing, the concept that I used was, “Oh, do you want an Amodo for your area?”

Ilan: And that’s one of the things we’re doing through Activation Partners, which, again, is sort of an experiment around this idea of, “The best people, in the right environment, with [the right] incentives.” We’re working with Convergent Research on what I think will be the first experiment of them saying, “Wide open to the spaces,” which will be our opportunity spaces. Instead of being a kind of two-sided marketplace and connector of talent with philanthropic or other funders for FROs (Focused Research Organizations), they’re basically saying, “If you’re someone talented who wants to commit your life to building a focused research organization in one of ARIA’s]opportunity spaces, apply! Then we’re going to find one, maybe even more, to go launch.” And it’s exactly what you said...

Eric: To people in the metascience space, I think Convergent is maybe one of the most exciting...

Ilan: Yeah.

Eric: ...kinds of Activation Partnerships. Because I think everybody [metascience nerds] was around when they first kicked up. And they also always knew [suspected] the vision was probably that Convergent might like to win NIH funds, or something like that, and set up more every single year. So to have an official [Activation] partnership was exciting, for somebody who’s just been a fan for a while.

Ilan: I think the biggest — we might agree on this — but I think probably the most important thing to accelerate progress in the research ecosystem is to increase the diversity of institutional types. Convergent is one of the few examples we have of a mode that is taking root, that’s saying, “Here’s a new institutional type. Let’s figure out how to scale it!” You talk to Adam [Marblestone] and he talks about...you know, he sees a thousand — I think he says either a hundred or a thousand — FRO-shaped things that need to be created in the world. I think it’s a thousand. I imagine there are a thousand. If he thinks it’s a hundred, I can probably debate it. [Laughter]

Eric: It was one of those two numbers. One more question, and then I’m going to do a little pivot. You seem to have a very good grasp of early/mid-20th Century R&D, the institutions, some of the stories, etc. Is there any organization or group from that period you find yourself thinking a lot about? Either because you want somebody to emulate it or you just think it’s cool?

Ilan: That’s a good question. I think the ones I think about are maybe, not surprisingly, the ones that feel like startups before their time. You wrote the piece about Edison.

Eric: Yeah.

Ilan: That was a story I was already obsessed with...but the whole idea of, like, Edison in the workshop. There’s an organ in the back, there are bunk beds, they’re sleeping there. They’re just all in, committed. You know, he’s raising money based on a dream. You think it’s bullshit, but then he makes it true by doing the work to close the loop.

Eric: The financiers come to visit and he looks crazy...

Ilan: That’s one piece. I love the story of Philo Farnsworth. I don’t know how much you know about that one?

Eric: I don’t know that one.

Ilan: 15-year-old farm boy who has a vision of making television. And basically through some serendipitous set of events, because he’s so passionate about it, ends up moving to San Francisco, getting funded by like an LA financier from Hollywood, and is competing with RCA in a race for the first television. And you just look at that, and look at the lab, and look at the talent times intrinsic motivation in the right place and the right time...

And I think Curie — this is more academic — but I think Marie Curie is another example of this. It doesn’t seem like it, but when you look at how she and Pierre Curie worked, it definitely feels like a startup.

Eric: What about it is very startup-y to you?

Ilan: Oh, the combination of the singular focus and the idea of, “We’re going to be open-minded and try to get to the goal through all means.” I always love the story of — I might have this wrong — but my impression is that Pierre Curie discovered piezoelectricity as part of trying to figure out how to very sensitively measure the radioactivity in the materials that he and Marie were looking at. So, basically, you have an academic pursuit to discover and understand this phenomenon of radium that no one understood, that leads to stumbling into discovering a whole other phenomenon, piezoelectricity — which has massive implications technologically. And it’s all because you had this intensity of spirit in one place.

[MIT’s] Rad Lab is probably another example of this. I don’t know if you’ve written a piece on Rad Lab.

Eric: I’ve never written a piece on the Rad Lab. There’s one book that’s continually over $100. And I check Amazon like once a month to see if the price will come down and I can finally get it. [Laughter]

Ilan: I’m going to buy you that book, whatever it is.

Eric: You don’t have to do that!

Ilan: If it means you’re going to write a piece about Rad Lab, then I think we all...we’ll do like a GoFundMe campaign to buy you the book!

Eric: [Laughter] I can promise to find a way to have it out in the next year, somehow! I have a few other Rad Lab oral histories sitting on my shelf that I’ve been letting pile up, so I do have to do this.

Ilan: Yeah, but the whole idea that because we got a group of physicists together to try and build radar systems, what fell out of that was, like, MRI...and the basic theoretical and practical things around that is just awesome!

Eric: Yeah, and also in terms of funding something for five years [as FROs tend to be], and then it's set off in a direction of its own...people at Rad Lab seem to have thought of some of Weiner’s early cybernetics writing, and things of that sort, as helping set the agenda for the RLE [Research Laboratory of Electronics, which the Rad Lab morphed into postwar] going into peacetime. It’s very clear that [the Rad Lab] was catalytic in extreme ways. The kind of thing that would be ARIA ‘returning the fund’ level [results].

Ilan: Yeah, it’s funny, one of the things that’s coming to mind here, as we think about FROs...one of the things I think a lot about is, for so many of these institutions, and I feel this way about ARIA right now. We’re two years in from starting everything from scratch. It feels like we have the environment where these amazing things can come out of it. When I think of our program directors, when I think about the Creators they’re starting to fund, the dynamic of the team, I’m like, “Wow. This has the energy and the characteristics you would want.”

The big fear is, I think, “Okay, this is founder mode.” [Laughter] These are early days! If you look at Rad Lab, it was this intensive thing, and MRI and these other things came out of it in the early days. And more came out of it later, but that question of, “How do you capture that founding period and not lose it?” I mean, what I love about FROs is the idea is, “Don’t even try.” Imagine that you’re going to have an intensive thing for five years. And then it turns into something else, like it metamorphizes. I think there’s something really important about that.

Eric: I remember hearing Adam muse at one point about the idea of, “Oh, it would be great if we had funders who could accept or were excited about the idea of maybe one FRO maybe leading into another FRO.” That is the outcome. You [ARIA] are playing a bit more of a repeated game. How do you think about that, one ARIA program leading to another ARIA program?

Ilan: Oh, 100%. I think most likely that is how we will make an impact, which is we’ll have a program that inspires the next program. And within that next program, we see something and we double down on it...and that’s the thing that happens, being adaptive in that way.

One of the interesting things that I’m noticing is...we haven’t talked much about how we recruit program directors. But the combination of our broad mandate and the fact that we are bringing in, as cohorts, program directors from such different backgrounds...we have these opportunity spaces that the program directors have shaped, and, yet, we’re now starting to see as we dig into them, these really interesting intersections between them. And I’m just wondering what’s going to happen. What are we going to do about that? Are we going to have programs that sit across two opportunity spaces? Are they going to merge into one? You know?

Eric: And can we talk about your new program directors? Who are they? Why did you choose to light up flares around these particular people and their expertise, or their point of view?

Ilan: Yeah. I think this recording is going to come out right after we have announced them. It’s probably worth first saying, we made a bet early on at ARIA. And my colleague Pippy had a really big role in this, which is we basically decided that one of the hardest parts for a research organization like this in the UK was going to be to find program directors or program managers, because, you know, it’s a weird thing to do. You’re on some amazing career trajectory and you decide to, like, step out of that for three to five years to do this other weird thing with a brand new agency that no one has ever heard of. Probably the listeners of this realize that one of the core parts of the DARPA model, which we’ve adopted at ARIA, is this idea of term limits. The people who are developing the theses are people that are coming in from the front lines of doing research in some way. They’ve got to sprint for three to five years to develop a thesis, fund a program, have an impulse function that hopefully catalyzes change, and then they go out and we refresh with some new program director. And part of that is to keep the ideas fresh, but actually, a big piece of it is to keep it incentive-aligned.

I’m never worried that a program director is going to make a decision on their program based on worrying about their career trajectory and what their boss at ARIA thinks, because they’re going to expire in three years. So they’re all just purely focused on, “How do I use this time to make something happen?”

One of the bets we made was, we knew it was going to be a challenge to find people. So we decided, rather than just hire people one-off, as we found them, let’s actually do what we believe is the most open call of its kind to find program directors. Let’s imagine that we can think of a set of characteristics that matter and that we think are going to make for a good program director. And that’s going to be more important than specifically their resume or what they’ve done before. And then we decided that we’ll bring them in as a cohort. So, part of the idea here was, if it’s going to be a risky thing for somebody to come join ARIA at the early stages, it becomes a lot less risky if, when they’re thinking about joining, they can look around and see five or six other people that they think are pretty awesome, who are also going to jump into that pursuit.

And I think that worked out really well. If you look at the first set of program directors, we’ve got someone like davidad who, I think he got his masters degree from MIT at age 15. I don’t think he ever finished his PhD. He ended up working at places like Twitter doing independent research. And he sits in our office next to Jenny Read, who is like a seasoned senior professor at the University of Newcastle — in different fields.

And you have all these other people, and they’re so spiritually aligned, and respect each other, and push each other. It’s really cool. [pause] We were talking about new program directors, but I didn’t get to it.

Eric: No, no, it’s okay!

Ilan: What would you want to see from ARIA’s next program directors?

Eric: [Laughter] What would I want to see?

Ilan: Trying to figure out how to...oh you’ve met some of our, you’ve met our existing program directors! If I were to say, like, “Okay, you have the chance to double the size of the group of program directors and make it an even stronger group. What would you try and sprinkle into the mix?”

Eric: So, that’s tough for me [laughter]...because I’m not a technical genius or a polymath in any way. I’ve been very intrigued by the energy of everybody involved [as an ARIA program director] and I’ve always left excited. But I guess, I don’t know. Engaging in this work as a non-technical person...

Ilan: Yeah.

Eric: ...I more just go in with the assumption of, “Somebody sees genius or a spark of something in this person. I’d love to help them any way I could!” I guess, reading the history... it’s unclear that it would be obvious to anybody, or me going into the past that, like, Frank Heart would be Frank Heart, or J.C.R. Licklider would be Licklider. So, I guess I don’t know...I attempt not to answer these questions or go down this line of thinking because I think I would be biased by random things, probably. For example, I go talk to [ARIA-related] people, and most of them are British. And it’s like, this is a new cultural context for me. Sometimes, with people in my personal work, I go based on spark or energy or something like that. But I don’t think that’s quite how a British scientist would do it. So I don’t really know. I won’t answer your question, essentially!

[Post-interview note: As I transcribe this interview, I wish I would have flagged that some social science program like a computational law program could be cool!]

Ilan: Yeah, interestingly, it goes to the intellectual humility point. You said something that resonated with me, which is like, “I imagine someone saw a spark of genius or brilliance in them.” So we designed our process to recruit program directors — I think we started, the first time we did it, we had seven qualities we were looking for. And they have names, like, obviously like ‘technical depth,’ but we have one called ‘vision to action,’ ‘adaptability,’ etc. We basically put all the applicants through the paces in different formats. We were probing them in different ways to test along those axes. And the idea is not to just have some aggregate score for someone who’s great, but the idea is to find the spikes. We talk a lot about how each of our program directors has a different set of spikes, meaning like superpowers that they’re going to leverage to do the job in their own ray, recognizing that a Licklider has very different strengths than a Bob Taylor, right?

And I think that’s worked out really well. We did the same thing with this new cohort of program directors. But we looked at our first set of program directors and what we learned from them, and we realized a few things. One was, there was something that they were showing in their behavior that we hadn’t tested against, which is something like ‘value recognition’ or ‘opportunity recognition.’ The ability to see the dots and find the constellation within them. So we added that to the mix. We also deliberately tried to recruit...our first set of program directors, roughly 2/3rds to 3/4s of them have more academic backgrounds. They tend to be sort of — I don’t want to offend anyone — misfits? They tend to have different mindsets than your typical academic, that’s why they’re attracted to this role. One thing we tried to do deliberately with the new program directors is say, “Let’s try and bias towards more people with industry or startup experience,” which is the case. So, what can I tell you about them?

Eric: Can you give us a couple examples of program directors who clearly have overlap with an opportunity space from the first cohort? And any examples of people who are, kind of, breaking new ground?

Ilan: Yeah, so a lot of them are breaking new ground. Actually, we’re already starting to see a bunch of different connectivity between them.

I’ll share one. One of our incoming program directors, Rico [Chandra], is Swiss. He studied at CERN, he’s a nuclear physicist. He then decided to go do a startup in the nuclear security space. He’s coming into ARIA, and the thing he’s become obsessed with is this question of...the fact that with all of our modern technology, flight and the ability to transport things in the air is still a big challenge, especially sustainably. And his view is that, “Actually, we know that there are plenty of birds, albatrosses and others, that can go vast distances, thousands of kilometers, without any source of power, in perpetual flight. Why is it that we can’t beat them with modern technology?”

So, you know, with our incoming program directors, we basically say, “Listen, just have a starting point, and then we’ll go from there.” We don’t know if it’ll turn into a program, if it might pivot or not.

Eric: It’s also interesting because more recent people in aeronautical engineering who we think of as geniuses, like Kelly Johnson, [often] thought in wind tunnels. But when you go back to the Wright brothers, they were obsessed with birds. Like there were all these stories of them on the beach where they were apparently — I forget if it was Wilbur or Orville or both — they were impeccable at imitating the motion of the birds’ wings.

Ilan: Oh I love that.

Eric: [gesturing to imitate the Wright brothers imitating the birds] As they were trying to figure it out. So it’s fun that...to find a really ambitious program, he’s also kind of gone back to the source.

Ilan: So one of the interesting things about Rico is he also happens to be, like, a world-record holding long distance hang glider. So he’s been thinking about this a lot. We don’t know whether it’s a program, but one of the things we realized...we have an opportunity space called Scoping Our Planet. And the thesis there is, “With climate change, the thing we really care about is the climate changing. We obviously care about and know that fossil fuels and greenhouse gas emissions are causing the climate to change. But there’s very little we know and very little energy has been put in by the innovation community to understand how we monitor the climate.”

And you can think about why that’s important. Everything from measurement and verification of things like carbon removal, which is sort of a big and unsolved market failure problem. But you can also think about weather prediction. You can think about understanding climate tipping points, which is what our program is focused on. Rico came in and we realized, “Oh, is this like a new aeronautics opportunity space?” And then we realized, “Well, maybe this is just the capability that can be very valuable in scoping our planet.” So now he’s talking to Sarah [Bohndiek] and Gemma [Bale] [the Scoping Our Planet program directors] and trying to understand, “Is this something that can be game-changing around how we have new sensors for parameterizing the earth and communication protocols?” So that’s kind of an interesting, surprising one [new program idea] that connects to something we’ve done.

Eric: So, it seems like you really do hire people first, and find the specific programs later. A lot of your first cohort of program directors are working on programs that, like, it’s maybe not what their academic lab focuses on. Are there any people you’ve hired in this new cohort where you think, “Oh, what they might end up working on is exceptionally broad.” You’ve hired them with a guess, but it’s hard to tell where precisely it will go.

Ilan: I think that’s true of a number of them. I was just chatting with one of our incoming program directors, Alex [Obadia]. Alex is really interesting. He’s a mathematician by training, who then got involved in cryptography. He’s been doing essentially crypto and blockchain stuff, but he’s really passionate about thinking about how the kernels of innovation in real cryptography, that’s being applied to financial systems and otherwise, are going to be consequential to the future of society.

He turned me on to — folks should look this up if they haven’t — this paper on programmable cryptography. The idea there is, right now you think of cryptography as, “Okay, I have a certain piece of data in a discreet way. It’s encrypted or it’s not. I can see it or not.” There’s a whole community of people, now, thinking about, “Could I have a whole program that’s encrypted? Could I have a computer program that goes and does programming on your data without actually seeing your data, because of the encryption?” Or even a computer program that is doing computation and getting answers based on data where no one actually knows what the program is. And you think, “Like, what are we talking about here?” But clearly, as we think about AI, and AI agents playing a bigger role in our lives, there is a question of, “How do we optimize the efficiency of what we can learn and the value we can create with AI, where one of the big barriers is privacy of data?”

So, you can imagine — and I’m not an expert here — but the sense I get is, you can imagine programmable cryptography allowing me to, say, have a set of preferences, my values, things that are very sensitive to me, where I can tell an AI agent that is going and engaging in some democratic process on my behalf. Or I can have an election process that runs a program where it queries, “What are my really sensitive personal preferences?” and has it feed into an answer of a program without ever seeing my data.

So I got very excited about this! And I said, “Alex, this is massive, this is definitely an opportunity space!” And his view was, “Yeah, but I think enough people are already working on this, it might happen anyway.” And we got in this big argument. So now he’s thinking about, “Actually, what’s the next stage beyond that?” We have this massive convergence that’s coming, where we’re going to be fusing humans with technology in new ways. You can think of neurotech. You can think of, you know, just the way we’re much closer to devices. What does preserving privacy and trust look like when your devices are not purely digital, purely physical, or purely biological, but somehow sit at the intersection? I don’t know whether there’s a ‘there’ there, but that’s what he’s starting to explore.

Eric: You seem to have a number of life sciences-adjacent folks in the new cohort.

Ilan: We do.

Eric: What’s the thinking behind that?

Ilan: It’s a good question. Again, it’s another indicator that we really were people-first. When we got the first cohort, we realized there were really only two program directors working on things related to life sciences and biology — Angie, on Programmable Plants, and then Jacques [Carolan], who has the Precision Neurotech program within the Scalable Neurotechnology space. But if you look at the UK, it has incredible strengths in biology. If you look at technology vectors that are progressing extremely quickly — synthetic biology, right? You have all these pieces. So, shouldn’t we have more?

We do have more program directors thinking about biology in the new cohort. Brian [Wang] is an organic chemist, but then moved into thinking about pandemic preparedness. And he still cares about that very deeply, and one of the questions he’s asking is, “Can we utilize new learnings about the innate immune system?” The innate immune system is the part of your body that reacts to pathogens and toxic in broad spectrum ways, right? So it’s not creating antibodies to specifically attack a specific vector, it’s just that initial response. It turns out, actually, the way plants deal with pathogens is entirely innate immune system. And his view is, “Well, maybe we can be inspired by that to figure out how to create therapeutics that can be effective against new pathogens, new bugs, that we haven’t seen before.” And there could be a number of implications to that work where, through that same inspiration, basically the question is, “How do you get ahead of things that are evolving?” You know, you think about the Covid experience. But cancer is the same way, right? How do you get out ahead of something that is a foreign...

Eric: So, when you’re selecting program directors, you’re in a room and you have some exceptional electrical engineer who wants to work on some project nobody has ever done before. You might have a biophysicist interested in some medical area. You might have somebody from a field that doesn’t even have a name. How do you go about...? You have to come out of there with eight people. What does that prioritization even look like? Is it painful, because there truly is no apples-to-apples comparison? How does it feel? What do you do?

Ilan: No, this is actually where the approach...Historically, I think I always felt like I had a pretty good instinct on talent. You know, you hear about this in research and otherwise. I mean, you’ve written about this, which is just, like, there’s a taste and there’s ability to spot talent. And that’s something I always try to be attuned to, and refine.

I think over time I’ve come to realize that one of the best ways to find talent is actually to just create a product that really talented people want, that doesn’t exist. If you think about the ‘product’ we have at ARIA, you have, in this very open way, an ability to take £50 million and figure out how to create a wave that changes the world. That’s really compelling if people believe you. So that’s one thing that I think has changed where you say, “Okay, that’s a great way to attract talent.”

The other thing is, and the big lesson for me around our program director recruitment, by trying to strip away to first principles around, like, “What do we really care [about] in terms of the characteristics of these people?”...and this is very much like a Kahneman-based approach of recruiting and selecting people, which is just like, “We’re going to have the criteria. We’re going to probe the system a bunch of ways. We’re going to see how they spike against the criteria.” And ideally where that ends up, to your point...when we went through that process based on paper applications, phone call screens, technical interviews, we ended up with this set of people — this year I think it was 16 — where we said, “Based on everything we’ve seen, any one of these 16 people could be a program director.” Because who are we to know? Like again, the intellectual humility. I could guess, but I’d probably be wrong.

So then we said, “Forget it. Rather than just trying to pick the right ones out of the 16, let’s actually bring the 16 people together.” One thing that’ll happen is...these are awesome people who will go drive impact in the world, and they’ll probably all do more if they’re connected to each other. But it’ll [also] give us one more way to look at how they show up, not just as individuals, but also how they sort of connect to each other.

Basically, at the end of the process, we’re just thinking about the portfolio. We’re thinking about the portfolio of people, portfolio of ideas, based on what we think they might do, what they might bring to the cohort. Just curating based on that.

Eric: If I can probe you with a bit of an edge case from history, I’d love to get your...

Ilan: Oh boy, I might not...

Eric: So, speaking of intellectual humility, I’m a big Warren Weaver fan.

Ilan: Yes!

Eric: And something that comes out all throughout his writing...he is an exceptionally...he’s a Midwesterner with a lot of humility. It comes out in all sorts of his [writing]. So when he first came to the Rockefeller Foundation, even though he had humility, he had one of the more extreme bets in philanthropic history. The [Rockefeller] Natural Sciences Division, which gave out on the order of what ARIA gives out per year, maybe a little less. It’s hard to do a one-to-one, because science was cheaper back then. But, essentially, they [Weaver’s predecessors] said, “Oh we fund electrical engineering, zoology, etc. And we fund the very best people who come in.” And when he came in, he said, “That’s fantastic. We’re not going to do that anymore.”

When he’d gone around and talked to people, he said, “There’s something right there in that area between biology and physics. So, yes, our budget is really big, but it’s very finite. We’re going to press on that.” And he essentially put 80% of his budget towards — it was 1932 — five years later, he would name it ‘molecular biology.’

Ilan: Out of curiosity, do you know how long he was in the foundation before he made that bet? You probably do.

Eric: I think he came in with the bet.

Ilan: Oh okay!

Eric: He took the interview with them to say, “I don’t think I’m your guy. I’m just an applied mathematician. But there’s this thing..."

Ilan: “Here’s the thing we would do!”

Eric: Yeah. He read very broadly, for sure. And he kept at it [funding molecular biology] for 20 years before moving on to something else. But if you all felt you had an ARPAnet or a molecular biology on your hands, how much is too much of your funds focused on that one area? There’s maybe a limit. 80% is a lot, for example.

Ilan: Yeah. This is interesting. This is something we talk about a lot. First of all, the question of, “We have these seven opportunity spaces. Why are we recruiting more program directors? You know, aren’t we doing enough? Like the goal isn’t just to boil the ocean.” And actually, we tend to think really deliberately about, “We need to get to this one outcome that is so massive. Do we think we’re touching enough surface area right now to fast forward ten years, and realize we got lucky and we just noticed that thing that was going to be so valuable?”

And when we looked at our budget, we looked at the size of the UK ecosystem, and we looked at what we were doing, we said, “Actually, no. We’re probably not touching enough surface area.” So we now have more program directors coming in. There will be some new opportunity spaces. We’ve also said, “There’s a limit.”

It’s likely that every other year ARIA will recruit a set of program directors. It’s a three to five year term. So that gets you into a steady state of 15–20 program directors. And that’ll be it. So we’ll have a first set of opportunity spaces. That’ll be the surface area that we touch. And really what success needs to look like is...out of that, we’ve found something that we not just want to double down on, but 10X on. And so if a big portion of ARIA’s budget does not end up in one of those areas over the others, there’s a problem.

Eric: Oh wow.

Ilan: That’s my take! You know, I’m on a term limit, too! I probably won’t be the CEO when they have to figure this out. But is that 80%? I could imagine it. We talk a lot about…

Eric: And could the person in charge of ARIA...

Ilan: Decide?

Eric: ...if they were a brave person, if they thought it was right, is that that? Are there some structural or institutional barriers that would make bravery even tougher, beyond the pure pressure of people looking at you like you’re a little crazy or something?

Ilan: One of the nice things about ARIA, it’s worth mentioning, the UK — like a number of people who were in UK government, civil servants, others, Parliament — they got this right. In the sense of, ARIA really does have the right mandate, freedom, flexibilities. There’s not political intervention, there’s not a lot of BS processes...they’re the right processes to make sure we’re responsible stewards of taxpayer funds. But one of those things is the CEO of ARIA makes programmatic decisions; that’s very much modeled after DARPA. I think that’s 100% true. And that’s good to have in place from a governance perspective. The question is, “Will the organization have a culture that leans towards the bold bet?

We’ve been talking about this a lot. So far in ARIA’s history, the things we’re most proud of are when we’ve gotten exposed to something and then we just said, “Ok, let’s do the bolder thing here.” And it happens in big and small ways. Sometimes it’s about a program director we decided to bring in. Sometimes it’s about a program that we launched. Sometimes it’s like, “Do we use Slack?” Because there are potentially any number of issues from having your Creator community on a Slack system. We just had this conversation and said, “We should bias towards doing the bold thing.” [Laughter]

And my hope is that we build into the agency that a leader in the future will be celebrated by pushing that bold bet.

Eric: I had a question written down, and I think maybe you just answered it. I was going to ask, “How do you deal with a program that you’re exceptionally excited about, but maybe the follow-on funder doesn’t get it? Like the VCs aren’t sure what to think about it five years in advance. Which is relevant, because with early autonomous vehicles, it’s not that the Army desperately wanted...they didn’t really know what to do with it. But it sounds like your answer is, maybe, “If it’s a bold enough bet, we can be the follow-on funder. For a while at least.”

Ilan: Yeah, that’s definitely our mindset. And we’ve even tried to build that in. Some ARPA agencies require cost share in their projects: “Oh, you have to show us that someone else has skin in the game by funding 20%.” We basically said, “We’re not doing that, because that actually could as easily could be a counter-indicator for us.” In the future, we think we’ll probably leverage other funders as follow-on partners, and everything else. But let’s make sure there’s nothing that prevents us from doing the bold thing.

That said, it can’t be as easy as, you know, in your scenario, “What’s going to get us to one day have 80% of our funding in one of these spaces?” It can’t just be someone walked in, looked around, and said, “Okay, yeah, let’s just do that.” We should be evidence-based. [Laughter]

We have a pretty good set of principles. A lot of this is, “What are the principles? And then what are the processes that allow us to make sure we’re holding true to those principles?” We have that for creating opportunity spaces. We have that for approving budget for programs. The next thing is to have that for portfolio allocations. Meaning, “We’ve run these opportunity spaces. Where do we think we are seeing the bigger bets emerging versus not?” So that we can say, “This opportunity space is going to get bigger or more focused, and this other one might go into hibernation because we don’t think the elements are there to create the wave.”

Eric: Yeah, no, that’s very interesting. I didn’t know a lot of that, so that was great to hear! So a bit of a question very relevant to...

Ilan: How many questions do you have?! [Laughter] It’s like an endless amount. Are you just making them up and the papers [Eric’s notes] are just like a...

Eric: Well I’ve skipped some [gestures at discarded paper pile], I’ve come up with some new ones [waves papers]. They’re not all off the page! They [Asimov Press] brought me in for a reason.

Ilan: {Laughter] He’s just got like blank pieces of paper just to make it look official.

Eric: I could show blanks! [shows the blank side of his stack of papers]

Do you spend a significant amount of your time thinking about setting up a portfolio to take advantage of AI? Obviously a lot of big breakthroughs in science history come from using the big thing from last year or 5 years ago to apply to some new area?

Ilan: Yeah, it’s a really hard one. I mean, the simple answer is, “Yes.” We are spending a lot of time thinking about how AI — the fact that we’re starting an agency like ARIA at this moment in time, given what’s happening in AI development. One of the easy inspirations for new programs at places like DARPA or ARPA-E is, “What are the technology vectors that are making incredible progress? What are the learning curves you can ride?”

So, I remember at ARPA-E, we had a number of projects we funded based on the fact that fiber optic lasers were getting much cheaper and more powerful year after year. So we had a project we funded on basically using that for geothermal — like drilling hard rock — and other things. AI is like that, just massively.

I think we’re trying to be a little bit deliberate before we figure out exactly what that means for ARIA. For instance, we haven’t jumped in and built, for ARIA, a major compute cluster. We’re starting to think about, “What does an AI program director look like?” You know, how to actually do a lot of the soundboarding that we do. Our program directors are actually using AI. It turns out AI is actually, like, an incredible tool as a program director. Because often you’re looking at things outside of your area of expertise. Oftentimes you’re like, “Oh, if only I had an expert that I could bounce this off of!” Having a thousand experts in all the areas at your fingertips makes a big difference.

I think the much more powerful part is, when we think about these opportunity spaces, one of the questions becomes, “Relative to the impact we think AI could have on this space, how much activity is there to start to deploy that?” And I think that’s going to be one of the important...especially for this new cohort, we’ll end up with some programs that really have AI at the center.

But I think for all of them, there will be this question of, “In this space, where are we in terms of the max benefit that AI is going to build by integrating with this discipline or set of fields, versus now?” And I think that will help dictate how much of an AI flavor we have in the space, and what we’re doing to be differentiated. I actually don’t know if I believe that answer that I just gave, but it sounded good.

Eric: Before we move on, is there any other technology you spend a lot of time thinking about? The world of science and engineering is broad. There might be something else that you think people don’t pay enough attention to.

Ilan: I mean, there are little examples. Like expansion microscopy, that’s such a cool thing! Most people don’t know! It’s popping up in different places.

Eric: For people who don’t know, the TLDR on expansion microscopy is, “What if you made the sample bigger!” And...

Ilan: By literally just putting it in a gel and expanding it. And it works! It’s unbelievable.

I don’t know. I think synbio comes up a lot. I think data, beyond just AI, data comes up a lot as an area where, when you look at some of these disciplines...this is something we’ve found in some of our climate work. You look at the best repositories and the most valuable repositories of data around climate and weather, and realize that none of them are engineered to be accessed in a way that’s compatible with modern data techniques. So there are just infrastructure gaps where you think, “Okay, there’s something cross-cutting around data that we can do.” Or a barrier that we need to get over.

Eric: All right. So I have a suite of metascience questions for you.

Ilan: Okay.

Eric: So, you can go rapid-fire responses, or spend time on them!

Ilan: [Laughter] Okay!

Eric: Do you consider now a great time to work on metascience?

Ilan: Yes. It’s probably the best time to work on metascience that has existed in my life.

Eric: You’ve done this for a few decades, depending on how you want to look at it. Were some periods a lot more frustrating than others?

Ilan: When I left ARPA-E, I wanted to go start...actually, I may have sent you the paper, I had a paper...

Eric: ‘ARPA Lab.’

Ilan: Yes, ARPA Lab. I wanted to create an ARPA Lab. And the point was, like, “ARPA-E is great as a funding agency. The problem is I don’t have resonant institutions to fund. So let’s imagine a lab that is ARPA-minded, that is very entrepreneurial.” And I sort of mapped it out. It was probably like a lab that would have been a compilation of FROs.

And I thought maybe we’d get philanthropy to fund it. And, you know, who was I? So maybe someone else could have done it. But like, crickets, you know, nobody! You know, philanthropy funding science, it sounds like it was a long time ago; it wasn’t that long ago. But the state of affairs, the default for philanthropy and science historically, probably for all time — you can tell me if I’m wrong — has been like, “Oh, I have a family member who got this disease, and I’m going to fund research on this disease.” Or, “I went to this university, so I’m going to give them money to do scientific research.” The idea that you’d have philanthropists making big bets like Convergent Research or Arc [Institute], in terms of new modalities for R&D, that is a new phenomenon and a really exciting one.

Eric: Speaking of something like Arc...

Ilan: This is meant to be rapid fire! [Laughter] You let me go on!

Eric: A lot of people conceptualize the new science orgs as experiments, in and of themselves, in how to do science differently. ARIA is also some version of this. Is it difficult to see what’s going on at a place like Arc, and, if there’s a useful learning, fold it into your operation? Am I not thinking about that right?

Ilan: I think you are thinking about it right. I think the biggest thing — as someone who grew out of venture capital, startups, and Silicon Valley — the thing I’ve realized is, the power of a vibrant ecosystem of startups is you create evolutionary pressure for institutional change. Meaning, nobody talked about OKRs until Google came around. And then everyone needed to do OKRs! And then Stripe came around, and I don’t know whether Stripe does OKRs, but they do things differently. And then all of a sudden it’s like, “Oh, that’s the way to do things.”

It drives you to say, “Oh there are competitors, or peer organizations, that are figuring out better ways to do things institutionally.” And that’s keeping the system super fresh. We don’t have that in research. The institutions are so stagnant for so long that you don’t have the evolutionary pressure to change.

For me, what metascience means is: go run those experiments and find ways to start diversifying and creating a more vibrant institutional ecosystem. Partially so you can get that institutional pressure! You need a critical mass of those things to get that institutional pressure.

So you asked, “What is ARIA learning from Arc?” I love Arc as an experiment, and I think it’s going to change the world. I initially was skeptical that Arc was going to be so close to Stanford as a university. And that they were going to hire people that were still in the academic incentive structure. I think, actually, it’s probably proving me wrong on that. And that’s a great learning. But it’s a very high level, like from a distance, learning.

The beauty would be if we had enough of these things where I could tell you that, “With a 90% certainty, that some one in ARIA’s team, in three years, will come from Arc.” Like if you look at startups in the Valley, right, like, “What are the chances that someone who was at Tesla will end up at another one of these companies?” Absolutely, right? That’s one of the things that drives that learning. It would be awesome if, within the R&D ecosystem, we had that kind of vibrancy and mobility, because I think then you really get the learning.

Eric: And if you personally had the funds, for whatever reason, to fund additional orgs to complement ARIA, like completely different metascience experiments, does anything come up in your head?

Ilan: [Laughter] Yeah, what came to mind, just thinking about your audience, was a conversation with Michael Nielsen when we started ARIA, who was basically giving me a hard time: “What you’re talking about doesn’t feel that differentiated. Where’s the gap? What about a research organization that gives 100-year grants?” The idea of, “We don’t have long term modes.” If you think of time constants as one of the axes of the portfolio, there’s not a lot of diversity on time constants, to like 100 years. Which I loved! I basically said, “Yeah, but if I look at ARIA’s mandate, actually, ARIA’s mandate is not to create new fields.” That’s important. It’s not like [to create] molecular biology. We are built to get something to a new technology platform or industry base. And I don’t think the hundred year timescale is going to work for that.

But I think long duration...basically modes of getting talent focused on either creative research or creation research for long periods, I think there are far too few modes for that, and fewer than there used to be.

Eric: So, that’s obviously ambitious. It’s also expensive. ARIA is pretty expensive, Arc is expensive. Do you think you need a minimum amount of funds to do a good metascience experiment? Do you have low-cost ideas?

Ilan: Well, I don’t know if the long-term thing is expensive. I mean, look, what is expensive? It’s all relative to the potential impact. You have to normalize it, right? So, one of the things I’d love to see someone do is...I think a lot about, “How do we train scientists?” Translational scientists in particular...there aren’t great training environments anymore for translational scientists. One thing I’d love to see someone do is basically say, especially with AI, the training is going to be less and less about the knowledge, and more and more about the taste, the tacit knowledge. and the instincts. So apprenticeship is really important. Why don’t we take kids that are high potential — and when I say kids, I don’t know whether I mean age 15 or 22, who knows? But take high potential people and basically say, “We are going to engineer for you a program where the next 15 years of your life you are constantly being taught how to do research, translational research, through a series of apprenticeships with incredible people.”

And maybe that’s in conjunction with a university, maybe it’s not. To pay for that, you can basically say, “We’ll give you fellowships,” or “You’ll work part of it.” That sounds very expensive, but my instinct is that if you did something like that, you could create the super researchers of the next generation that change the world. So probably not that expensive. [Laughter]

Eric: I’d love to poke on that...

Ilan: Maybe that’s too expensive.

Eric: ...but I’m going rapid-fire.

Ilan: Let me give you one more!

Eric: Yeah, for sure!

Ilan: Very cheap metascience experiment, probably the cheapest one I can think of. I noticed in my PhD, and actually when I hired for my startup, I found myself hiring people...a mode that I would hire was someone, who went into their PhD, they totally butt heads with their advisor, and they walked out. They ended up having to change advisors, or even complete disciplines, and then they thrived. And they come out with this like, “Oh, that was a horrible experience. But it made me change and I’m in a good place.”

Anyway, I keep thinking, I found an amazing PhD advisor that was really resonant. I didn’t waste my time. I felt like I hit the ground running. And I had this incredible experience. So a simple idea: a matchmaking app for PhDs and their advisors, when they come in. If you could increase and improve the compatibility of a PhD student and their advisor, on whatever axes, I think that ends up being a big deal. And it’s basically free.

Eric: Okay, great. That’s fantastic. Thinking of ARIA as a kind of experiment in and of itself, what’s a current bottleneck you all have that you’re very eager to find a way around or work on? There’s always small things in working on an organization.

Ilan: I’ll tell you what comes to mind. It’s kind of a hard one, but I think we’ll probably have to solve for it. We want to move fast in our programs. The view is, “You’re doing something speculative, so time to the next learning cycle is really important and valuable.” And we want to be funding people with diverse skillsets in diverse institutional types. Some institutions...the kinetics of the institution are very slow. And yet, we still want to fund people in those institutions. So one of the questions becomes, “If the kinetics of the institution, just in terms of how fast they can get stuff done, are generally very slow, but you have a Creator in that institution who wants to move fast, how can we help them move faster?”

And I actually think the only answer is we need, like, a fixer. We need an organization whose job it is to de-bottleneck activities within other institutions.

So my imagination is, Eric, you start the Research Speed Fixing Company, and we contract you, and you know, Jenny has found this great performer, but things are moving too slow in this institution. And Jenny gives them a phone number, that’s like a magic phone number. And they call it and you say, “Eric’s Fixing Services! We’ll send someone over right away!” And someone shows up in that institution, and they're running around the admin, just like putting pressure on things and getting people to move faster!

Eric: There are actually things like this in DARPA history.

Ilan: Are there? Amazing!

Eric: I can’t think of one from a performer, but I can think of one from central office. So, for example, when you read through all the oral histories, there’s this guy from early IPTO — which is computing office — Al Blue. And his name just keeps coming up as the guy who makes any bid you want to put out legal, workable, straightforward. I don’t know if he was an engineer or scientist or anything like that. He might have been one of the military guys who finds a way in and has to make his way in the office.

But that kind of stuff makes a big difference! Like in the same breath where they’ll be like, “Oh Licklider was a god,” they’ll also be like, “Al Blue made X thing happen. You need to talk to him.”

Ilan: Yeah! Well, we’re trying to build it into the culture! Right? Like the program directors know...it’s my [Ilan’s] job to have some pressure around, like, learning cycles, it’s their job to do the same. Hopefully we’ll create a Creator community...picking people that are intrinsically motivated, so they want to be pushing the kinetics of what they’re doing. And, yet, it would be really nice if you had more help on that.

Videographer: Time for a battery swap.

[Camera Break]

Eric: Alright, next in the rapid-fire questions…

Ilan: Can we, can we just pause for a bit. I love that we’re doing this long form thing, but people have to realize that this camera right here — there’s a camera with like a bag of ice on top of it because we’re going so long that it’s overheating. Which is great!

Eric: We’re making it work, though. I think the camera looks great, you know? It’s like when you go in the locker room at the end of the game and everybody’s got ice on their knees.

Ilan: Yeah, totally.

Eric: Both teams played hard.

Ilan: We’ll feel like we played hard today. [Laughter]

Eric: Is there anything you personally think people who write about metascience spend too little or too much time on? Like is there some hypothetical Substack where, as the CEO of ARIA, you would hoover up if somebody wanted to spend the time on it? I already have homework where I have to write a Rad Lab piece.

Ilan: That’s right! You do have to write a Rad Lab piece. Multiple! There’s like a series there. Oh, you also need to write a piece about how, actually, DARPA ended up emerging in part because of the UK’s radar effort. Which is important for me because it brings things full-circle to ARIA.

Metascience...honestly, what I would love is a blog that just chronicles metascience experiments and their learnings, and just keeps tabs on them. And that that grows and grows.

Eric: And what are variables that you think should be...some of would be qualitative, you meet them where they are...but would there be any underlying variables you’d want if somebody was writing about Convergent or Arc or ARIA, to constantly revisit?

Ilan: Incentives.

Eric: Incentives?

Ilan: Incentives. I think that is the only theme that matters. [Laughter]

Eric: Yeah, I think that makes sense. Do you have any call to action for any groups involved in metascience, or who would want to be involved in metascience, that you think would be useful to put out there? That can be researchers, engineers, ops people, policy people, whatever it is I am.

Ilan: I’m going to say something controversial, which has been on my mind a lot, which is...I think metascience is, at the same time, one of the most important movements, in terms of driving more progress out of research, and also in some ways one of its most dangerous movements.

The reason I say that is, when I was at ARPA-E, I was really interested in the question of, “How does ARPA-E understand its impact, map its impact, and measure it?” And I tried to dig into how we could do that. And both from my explorations of the community of people thinking about that and from the many activities we had to go through at ARPA-E to try and show and prove our impact, how things work...the amount that was “useful” was probably 10%, and the amount that was, “Actually, you’re trying to map something that doesn’t make any sense onto what we’re doing. You don’t really understand the context of how we work, and you’re trying to measure it. And you have a number of theories, but even when I tell you they don’t map on here, you don’t believe me,” actually led to a lot of inefficiency.

So I think what I would say is, if you’re in metascience, recognize that first of all, if you can do something experimental which pushes that evolutionary pressure on the system, great! If you can show there are better ways to do things by doing them and then having results, amazing! If you’re working more on the theoretical side or the evaluation side, just recognize you have a big responsibility, which is to make sure that when you add up all the hours of people who are engaged with that — from agencies doing the work, startups doing the work, or whatever else — that you have high conviction it’s going to be net benefit as opposed to net cost. Is that...?

Eric: No, that’s perfect!

Ilan: I feel like I’ve said that to people, and I think I’ve offended them in some ways, but I think it’s a really important thing to be thinking through.

Eric: No, that's perfect. We’ll end the official questions there. That’s Ilan Gur! Thank you so much for doing this and how long you were willing to spend with me today.

Some time in 2023, Tom Kalil told me he thought it would be a good idea to carve out a chunk of time and get to work on some ARPA project histories. The ARPA model was proliferating, and Tom felt these pieces might find a ready audience of ARPA emulators and fans eager to make use of the actionable information. Tom, who was then at Schmidt Sciences and is now President of Renaissance Philanthropy, has good taste. When he suggests something, I’ve learned you should listen. So I trusted him and threw myself into the work.

If any of you have enjoyed this series and wondered how the sausage is made, “How does one figure out how a 60-year-old R&D project — with no book written about it — was managed?” The answer is often: oral histories. The documentation that enabled many of these histories came to be because, at some point, committed historians sat down with a set of DARPA Directors, officer directors, staff, program managers, and funded researchers to record interviews with them on the practical details of their work.

Usually, I’m just a consumer of these oral histories. In today’s piece, in a departure from my usual role, I get to deliver you all an oral history — one I think is ideally suited to FreakTakes readers. It’s an oral history with possibly the best guest I could have asked for: Ilan Gur. Ilan is the founding CEO of the UK’s new Advanced Research and Invention Agency (ARIA). And, better yet, he’s a huge metascience nerd! The two-hour interview, recorded in Berkeley, attempts to unpack the metascience experiment that is ARIA.

I can’t thank Asimov Press enough for funding the interview and facilitating its recording. The full audio, video, and transcript are here on FreakTakes. And I also worked with Asimov to write up a much snappier, abridged version of the interview. You can find that piece in Asimov Press today.

Enjoy!

[Spotify]

Subscribe now

Share

For those who’d like to listen along while they read, I’ve embedded the Spotify version of the podcast. Full transcript below. Special thanks to my girlfriend Katherine for helping me edit the full transcript:)

Eric: Ilan, the CEO of the [UK] Advanced Research and Invention Agency. Thank you so much for being here and doing the Asimov interview today.

Ilan: Thanks for having me. I don’t think I have ever been interviewed by someone who geeks out about the same things I do as much as you.

Eric: Well that’s exhilarating, because this is my first interview. We’ll see how it goes! [Laughter]

First question. We’ll start with a fun one. As grandiose as it sounds, ARIA’s mandate is to aim big and try to create technologies as important as the ARPANET or spark new fields the way the Rockefeller Foundation did with molecular biology. Do you consciously think about that to yourself — that you have to be or find the next Warren Weaver or JCR Licklider? Does that keep you up at night?

Ilan: [Laughter] What keeps me and our team up at night is actually having a massive enough impact. Do we think about trying to be or find the next Weaver or Licklider? Probably not. I use an analogy often of being a catalyst or an enzyme. Whether it's me or our team, we're one thing you introduce into the system that hopefully mobilizes a bunch of other stuff to make an impact.

Eric: In that case, if you think about being a catalyst, what makes a reaction big enough?

Ilan: That’s a great question. You’ll appreciate this: when we have program directors that come into ARIA, some of the pre-reads we give them before they start are interviews with Bob Taylor or stories from DARPA or otherwise. And the main thing is to try and get in their heads, “What does a win look like?” For a lot of places if the research you fund leads to a new product that makes an impact, that's a big win. I often say, “That would be a loss for ARIA.” Because what we really need to do is catalyze something that is bigger than a product, bigger than a company. It should essentially be like a movement, an entirely new technology platform that didn't exist, an entirely new industry that didn't exist. And I think the way you catalyze that can be really different for different people. So if I think about our program directors, they're all different. They all have different superpowers. Just like if you think about Licklider...I mean you're a history geek, right? So just like if you think about Licklider and Bob Taylor, right, you could talk about them each as having catalyzed this massive transformation, but in very different ways.

Eric: Are there any reactions that have been catalyzed in the past 10 or 20 years or so where you look at them and you say “Wow that's the kind of thing we should be going for”? I’m definitely guilty of thinking too far in the past on some of these things, so I’d love to hear your examples.

Ilan: One of our advisors for ARIA is Aslem Turechi, one of the founders of BioNTech. And so you think about their story of having sort of worked so hard on mRNA-based vaccines and systems — initially for cancer. And then you think of Dan Wattendorf at DARPA who basically started the program thinking about vaccines for pandemics. All of that was in the mix, and then all of a sudden you had the pandemic event that actually turned it all into the world-changing impact. So I think that's such an easy one to go to. I also want us to be thinking differently at ARIA around how those transformations might take place.

Interestingly, for example, you think of ASML as a company. But it really is a company that manifests, in that one company, a massive transformation in the field, in terms of UV lithography. Especially with ARIA being something based in the UK...you can't be picky in terms of how you're gonna get that world changing of a transformation within an ecosystem that is bound like the UK. I mean, we’re working globally, but I think ASML is such a compelling example of, “Out of some fodder and set of reactants, back to the catalyst, something catalyzed this massively transformative company and then field shift.”

Eric: Do you have any program in the current cohort that you think, like, their way of scaling to impact might be in an ASML-shaped box?

Ilan: That’s a great question. I think it may be for...any of them. [Laughter] In the sense that, and we may talk about this more later on, one of the big things about ARIA that we're trying to build in...that maybe differentiates it from, say, ARPA agencies or DARPA (which started seventy years ago) is recognizing just how important entrepreneurship and startups are to cutting-edge research in today's world. Our bet with ARIA is that probably the biggest driving force for impact will be through entrepreneurs and entrepreneurship, in some way. Whether you end up catalyzing a whole ecosystem of entrepreneurs and different companies, or entrepreneurs that are focused on government, or whatever else. Or what emerges from that is one singularity of a company that changes the world, or like an “Ozempic moment.” Again, I don’t think we can be picky. I think you could imagine that happening in just about any of our spaces. For example, I think about neurotech, I think about programmable plants, and certainly in robotics. It's just a question of do we get lucky and nucleate that right thing at the right time.

Eric: On that point. So I read your House of Lords testimony and a lot of things of that sort…

[Laughter]

Eric: I’d love to parrot back to you how I understand ARIA, from those materials. You can tell me what’s right, what’s wrong, what I need to say differently, etc. Also, if you could use examples from some of the current programs, that would be great. I’m sure there are many listeners who may have seen a program announcement, but that’s all they know. So as I understand it: ARIA programs must be 1) “big if true” and 2) differentiated. You don't want to fund programs that somebody else in the ecosystem would fund otherwise. And while each project doesn't have to succeed — the vast majority won't in the traditional sense — they should all succeed in materially changing the conversation regarding what's possible in a given space. In terms of your mandate in the UK — in the long term, these programs are meant to help drive economic output, quality of life improvements, health benefits, etc. Is that more or less accurate, how you'd conceptualize it? If someone says, “Isn’t that what deep tech VCs do?” what would you say?

Ilan: First of all, is that right? I think that’s right. We're meant to take bold bets that kind of amplify different parts of the research system in new ways. I think about our programs as funding like a constellation of teams that can lead to massively transformative outcomes at the intersection of quality of life and economic growth — not just for the UK, but for the world. I think that ‘for the world’ is really about the scale of ambition that we are talking about.

You were asking, “How’s that different from a VC?”

Eric: If somebody says, “How’s that different from a VC? Or “...than if the NIH spins up a new study section?” Or something of that sort.

Ilan: I think those are two very different questions. Let’s talk about VCs. There are a lot of VCs who are just in it to make money...but in terms of the mindset and the motivations...a lot of VCs, I think, do see venture capital as a platform to change the trajectory of the future and world in positive ways. I think the big difference with VC is — when talking about the idea of ‘big-if-true’ — there’s a sense of scale, right? You have to think about the scale of “how big is ‘big’?” — and how speculative are you willing to get on the ‘if-true’?

In talking to venture capitalist friends about what I do and what we do at ARIA, sometimes you hear this metaphor around startups and VCs: “surfing a wave.”

That’s one way to think about this. What does a VC do? A VC is meant to be like a surfer in the water, to look out at the wave set that’s coming and ask themselves, “Oh, okay, is that wave forming? Is that going to be a really big one?” The size of the wave is, kind of, how big of a market they think might emerge. And the bet they’re trying to make is, “I bet that’s a big enough wave.” And then they’re trying to bet on the timing, like, “Okay, this is when I should grab a surfboard and start paddling.” And you can think of the surfboards as the companies they’re investing in.

I think the big difference [between ARIA and VC] is, that’s being very reactive to the system. You’re trying to find the trends you can hop on and surf, so that you can maximize the ROI of your investment in a very concrete, direct way.

I see our job as more of putting the energy in to create the wave. A venture capitalist can only fund into a thesis once there’s enough momentum where they can start reading, “I see the market forming, I think there are technologies that can work here.” And most importantly, “There are companies.” Like, most VCs are waiting for companies to come find money. So there need to be enough entrepreneurs and companies doing that.

If you look at the spaces that we’ve carved out [at ARIA], we’ve said, “Right now everyone looks out and just sees flat water, but we believe that by pushing a little bit here or there we can start to build a wave.” And if we can catalyze the formation of a wave that’s big enough in one of these spaces, then actually that’s going to catalyze all the VCs to want to jump on that wave and to want to invest in companies. Then you get a win-win and, ultimately, change the world. [Laughter]

Eric: And so…

Ilan: Was that helpful, that analogy? I probably went a little too much on that.

Eric: No no no, it was great! So I have some questions. Sometimes, in terms of creating the wave...I’m sure there are times where you put a certain amount of money in a space, and there's enough teams out there that already care about this and have the proper background and the right incentives to just throw themselves into it once the capital is there. And then there's probably other cases where more legwork is necessary. Because that's not quite what a university would do — or usually a startup would do X task, but this isn’t quite a VC-scale market. How do you approach those two different types of areas?

Ilan: There’s so much we can talk about here. I’m glad this is a long form interview, because we’ll have time to get into a bunch of this. Part of this has to do with how we organize what we focus on, and maybe we’ll come back to that. I think it’s worth just talking about an example here. What came to mind as we were talking about the wave and VCs, I was thinking about one of our program directors, Angie [Burnett]. This might be particularly interesting to this audience. Angie’s a plant physiologist by background, had worked in academia, a national lab, she worked for the UN for a little bit. She came into ARIA initially thinking about food security and what we could do in food security. Fast forward — where did she end up with her opportunity space, which is what we call our focus areas, and then the program that she launched? The opportunity space that she launched is called Programmable Plants.

We tend to think for all of these [opportunity spaces], there’s some insight there. For us, an opportunity space needs to be, like you said, big-if-true. And we need to be able to make an argument relative to the potential impact. So if it’s highly consequential for society and relative to that impact you can argue it’s underexplored...you know, not enough funding or not enough of the right type of ideas or bets being made in that space. And that for some reason, it’s ripe for transformation. That defines a very big [opportunity] space that we are now going to start working in — and basically buying options through the early research that we’re funding to learn more and see if we find something that could become one of those big waves.

Angie's insight on this space was that if you just think from first principles, when you look at some of the biggest problems and opportunities we have in the world, obviously, agriculture is one of them. You think about food security, you think about climate change. The tough part about these segments is that they're massive problems, right? You need to talk in terms of gigatons of CO2 or carbon, in either case. And her view is that, actually, you can think of plants as a technology platform, and plants are one of the few technology platforms we know of that, actually, we have a system that operates at that scale — of gigatons of carbon.

And what do I mean by technology platform? A plant is a piece of hardware. It has certain functionality. Interestingly, in many plants, like you go to the store and you buy a cob of corn, we have actually engineered the functionality of that plant over many years. And you have distribution channels and the ability to deliver these things at a massive global scale, commensurate with those problems — food security, climate, etc.

The problem is...it's like a really shitty technology platform, right? We're very limited in what functionality we can build in. Instead of 18 months for a new iPhone, you take 18 years to get a new crop that's actually viable. Interestingly, if you talk to most investors, they will say, “Well, in the ag space...you've got the top [incumbent] ag companies and [given the incumbents and structure] it's basically the hardest, most entrenched space to innovate in.” It's just not a great place to invest. Massive capital, massive entrenched interests, there’s regulations, etc.

The bet that we're making is that actually...on first principles, when you look at what’s happening in synthetic biology, when you look at the problems and the forcing functions that are coming, requiring a change to how we do agriculture...there’s a pretty strong argument to be made that, sometime in the next few decades, there will be a shift in how we approach global agriculture. You can think about it as something like, “There will be an ASML-type company that completely disrupts the big five ag companies in the world, and does things radically differently.”

Eric: And what's Angie's job over the course of five years? And I guess also tie in, what is a program director? And how important is it that they have a point of view? And does that come out as you're picking them? ...I asked you a lot of questions.

Ilan: No, that’s fine. Angie’s job was first — together with me and the cohort we had — beating up the question of, “How do you define a space that’s interesting and in line with this idea of ‘plants as a technology platform that’s underappreciated’?” Synthetic biology has been this massive vector of technology progress, which is not being leveraged for plants and ag to anywhere near the extent it’s being leveraged for human health. So now there’s like this arbitrage [opportunity]: “We’re making all this progress in synbio, we have this massive problem opportunity in ag.” But they’re not really connecting the dots.

Eric: And how many years behind do people perceive ag as, in terms of the cutting edge in synbio? I’ve heard 10 or 20 years, but you’d have a better...

Ilan: Well, let’s fast forward. So, what is Angie’s job? Her first job is to lay out that space. The next job was to say, “Okay, what’s a funding program we’re going to launch?”

She had a bunch of different ideas. Where she ended up is a program that we’ve now launched — and we’ve just announced the people and groups we’re funding. That program is called Synthetic Plants. [See this link for more information.]

And the idea was, if you look at the real frontier edge of synthetic biology, and we're talking about things like de novo genome synthesis, like actually thinking about synthetic organelles. You have, in the UK and other parts of the world, really amazing progress happening. But when you go and talk to those researchers working in those areas, and you say, “Well, we're thinking about doing a plant program,” the reaction Angie got from all of them was, “Why? There's so much to be done in mammalian systems, we're making all this progress...like, plants are hard! They have multiple copies of the genome, it's impossible to figure out how to transform them, it’s XYZ...”

And instead of getting discouraged by that, Angie got more excited by that. She said, “Well, wait a second, let me tell you a little bit about the potential impact we could make if you start working on plants.” Long story short, she held a workshop. And I got to be at this workshop, where she brought together a combination of top synbio people, top ag people in the UK and otherwise. There were probably 50 people in the room. One of the questions I asked was, “How many other people in the room did you know when you walked in?” And I had said [gesturing his hand upward], “Raise your hand if you only knew one person, raise your hand if you only knew X...”

And I think most people knew like five out of the 50...and none of the synbio people had talked to any of the plant people.

What we’ve found is, now, there are all these new collaborations. We have synbio folks who are basically saying, “I’m shifting my work to plants because I’m just convinced this is the most important thing I can do.”

Eric: And when you say ‘folks,’ how many are thinking of shifting their work to plants?

Ilan: Well, you’ll see how many synbio people we fund in this program when it gets announced. But I know of at least two specific cases Angie’s mentioned where, when she started the process, the person was so averse to the idea of doing anything in plants and thought it was a waste of time...who have built really meaningful collaborations and — whether they get our funding or not — are going to start working on that. So, it’s cool.

Eric: When is high variance [in reactions] among the adjacent experts exciting? When is it less exciting?

Ilan: Oh yeah. How do you know when people just say, “Oh, that's a horrible idea.” How do you know when it actually is a horrible idea? [Laughter]

Yeah, I mean this is an interesting thing. You talk about variance, right? Someone from DARPA once shared this with me, and it's a framework that I really love, which is...if you think about the role DARPA plays, more important than anything else, it is a mechanism to increase variance in the system.

It’s actually something I think a lot about for ARIA. The idea there is, more than anything else, ARIA was created to do things differently. The idea is: do things differently, and by doing things differently, by shifting the modality of research, you can get different outcomes. We can spend a lot of time coming back and talking about that.

But if you want to do things differently, first of all, you need people who see things differently. It’s one of the reasons the PD [program director] model is very helpful. But secondly, you have to break what is, I think, a pretty strong and homogeneous sort of truism of how we (largely governments) fund research. Which is something like, “We’re pretty good at funding the well-known disciplines. And we’re pretty good at funding the obvious problems or opportunities.” But that's all very linear, like, “More funding for biology, or synthetic biology, or semiconductors, because we see what's happening geopolitically.”

If you want to do something different, you need to find some way to cut and slice through that in a different way. Interestingly, the ARPA model says, “Okay, you find a program director, like Angie. You have them look across the system and say, ‘Well, what’s an actual thesis of something new that we could do, that’s not a single discipline? Maybe it is, but it cuts across TRLs [Technology Readiness Levels].” That’s the other thing. People sometimes ask, “Is this low TRL or high TRL?” It’s like, no. We’re mixing these modes in new ways.

Eric: You’re ambitious and applied...

Ilan: Yeah, and you have sort of a spectrum of portfolio. What that means is, what we end up funding on the backend of one of these theses we’ve developed, is going to be a very different set of people with a very different set of incentives — in terms of what they are going to do — than would otherwise get created.

And the variance is...if everyone thought it was a good idea, we would be introducing no variance into the system. If no one thought it was a good idea, we probably shouldn’t do it because it probably is not a good idea. But if you get spiky reactions to anything you’re doing, it’s the sort of thing where...okay, you have some minority that actually think this could work. But in a consensus panel, peer review, that minority voice would probably not win out or be there. So it would never happen otherwise. And, actually, I think the hardest job of a program director is to be able to actually face the criticism of so many people saying, “You don’t know what you’re talking about!”

Eric: You need to have courage to live with your bet, for years on end...

Ilan: You need to have courage to live with your bet, yeah.

Eric: And so some people...

Ilan: Wait, I’ve been talking a lot! I feel like it’ll be good for this audience...as we’ve been talking, what comes up for you as you think about some of the history that you’ve learned about for program managers at DARPA, variance, etc.?

Eric: So, I guess what I’ve been really impressed by, in working with the ARIA PDs, and ARIA in general, is when I read about early ARPA, 1960s and 1970s in particular, it seems like they really did trust the PMs to have an opinion. Or to say, “Here’s why this contractor’s best. You have to see it [the vision], and here’s why there’s only one group to do X thing.”

It seems like ARIA has given its PDs a long enough leash to have the point of view and follow it wherever it may lead. When I talk to folks at a place like DARPA, or something, I often get the sense that the PMs do have a strong point of view, but it’s an older organization. Like, it’s a 60 year old government agency. I think a part of the magic of DARPA is, so many government organizations only get more bureaucratic over time. They’ve [DARPA] been phenomenal at, like every five or ten years, finding ways to clear out a lot of the red tape, scar tissue, etc. But it does still build up a little bit.

So it’s been really cool. Something I try to impress upon people is that 2025 DARPA is not 1965 DARPA. You need to pursue different programs and do different things [today]. And when people ask me, “Well, do we have an early ARPA [today]?” Before ARIA, I guess I’d say, “Maybe OpenPhil.” OpenPhil lets people be free like this. But as ARIA has come into existence, it seems like you all live in the vein of early ARPA much more closely than anyone else. But, saying that, I also know you want to do something new...

Ilan: No, it’s interesting though, because I think this relates to the variance point. I’m trying to keep this dear to ARIA in the early days. It’s not about, “Did we find the right bet?” Because actually, given the variance point, in the early days, it’s not going to be obvious which is the right bet.

So it’s really about, “Are we being true to a commitment to increasing the variance of what gets funded?” And then, “Are we going to have the muscle to see the right bets emerge from that work?”

I actually had a [ARIA] board member recently say, “Isn’t one of our biggest risks right now that we created the wrong programs — like we’re funding the wrong programs?” And I said, “No, I don’t think that’s a risk at all, in part because we decided to create these opportunity spaces.” The whole idea behind an opportunity space for us, in these focus areas, is that our whole job is to take speculative bets. What that means is we can’t know upfront whether this is going to lead to impact. We can’t engineer that. The only way we get it wrong is if we take a speculative bet in an unproductive direction. So the point of an opportunity space is to say, “Let’s define what we think is a productive direction.”

An opportunity space is going to have certain beliefs that bound it. And the point is, if you can get yourself to believe these things...e.g. “Pressure on food security and climate is going to force us to do some things different in agriculture,” “Synthetic biology is moving...” Right? If you believe those things, you have to imagine there is the potential for enormous value — economic value and social impact — that can happen in this space. And now that we’ve bound that space and set that opportunity space, now we can take steps, big and small, and make bets in that area without having to worry, like, “Is synthetic plants the right program?” I’m not worried about it at all. Because we’re making a bet in a direction that’s productive.

The only thing we now have to worry about is that we don’t do what we were made to do. What makes ARIA unique — and what’s meant to make a place like DARPA unique, certainly in the early DARPA days — is that you make a bet, you take a step, and then you learn something, and you can pivot and say, “Oh! Actually it’s this 5% of what we’ve done so far that is starting to feel really valuable. Let’s minimize the rest and 10X that 5%.” That’s the thing that...I think that’ll be one of the big muscles we have to create for ARIA in this next phase.

Eric: So, you’ve moved to the UK, and ARIA is...

Ilan: I’m sorry, I have to stop! I’m like, I’m sitting here wondering if people watching this are excited about it. It’s just like — the idea that there’s now a community that’s excited to hear this conversation — that’s something I’ve been searching for for a while.

Eric: [Excited laughter] Okay, so, in that case...

Ilan: How many subscribers do you have?

Eric: Like 4,000. But there’s people like Matt Clancy who have like 15,000, or something like that. [Correction: Matt has over 18,000.]

Ilan: But those are high impact people.

Eric: Yes, I am continually wowed by the subscribers I have...and when I write something, the email outreach I get. It’s very much a dream for someone like me who...I wish I could be a great researcher [gestures at Ilan], but it was not in the cards for me.

Ilan: [Laughter] I wish I could be a great researcher too. Wasn’t in the cards for me. Definitely more of a Bob Taylor than a Licklider.

Eric: So you’ve been doing this for so much longer than many people. A lot of people have maybe gotten interested in this space in the past 5–6 years or something.

Ilan: Wait, what’s “this space”?

Eric: We’ll call it this “applied metascience” space. People who want to really consciously experiment with how you build R&D ecosystems differently, or research labs differently. In many ways, you’ve been doing this for possibly the majority of your professional career. Can you tell people about the different stops you’ve taken? Maybe different things you’ve learned along the way? Different gripes you’ve picked up? ... [I imagine] you only throw yourself into this if there’s stuff to be fixed and you have ideas.

Ilan: Yeah, I mean, I sometimes describe it as, like, I got thrown into this just because I felt like a misfit in all of the different environments to do research. And we can talk more about that. But I think that is a motivating factor, right? What motivates me, and probably what motivates most of the folks who would listen to a podcast like this, is just realizing that you can take the things we’re learning at the cutting edge of science and discovery, and turn them into stuff that’s useful, real, and awesome.

I mean I had — actually, you’ll probably appreciate this story — I was one of a rare set of people who was an undergraduate major in material science. Material science is kind of a weird field that most people, at least when I went to college, didn’t really know about.

Eric: They’d pick it up as a masters...

Ilan: Yeah, exactly. I think there were maybe like five people at Berkeley who came into the undergrad material science program. But the reason I got into material science...I grew up in Pittsburgh and went to a public school, which just happened to have a couple of phenomenal science teachers. And one of them encouraged me in 10th grade to put in an application for this program that CMU [Carnegie Mellon] did, where they would host high school students in a lab.

And I still remember, like, they gave you a questionnaire, and it was like, “What are your interests?” And I was in 10th grade, you know, I was not a polymath who was way ahead of their time. So my interests are like, ”I like physics [shrugs]...maybe chemistry?” You know, you just say pretty banal, normal stuff. Anyway, they paired me with a material science lab. And I remember showing up and saying, “Why did I get the oddball thing? Why couldn’t I just be paired with a normal lab?” And I didn’t think it was going to be cool or exciting.

The project — and this is the piece that I think you’ll appreciate — the project that I got paired with was a lab that worked in magnetics. What they wanted to do was come up with a new magnetic storage medium. At the time, you know, a hard drive was, what, 16 megabytes? You didn’t have flash drives, etc. And so the question was like, “How could you get much higher density?” So what they were doing is they were taking a material called MCM 41 — this is the part you’ll love — MCM 41 was one of a class of what are called molecular sieves. It’s a material that has a microporous structure, a lot of core space, sort of a zeolite material used as a molecular sieve/filter/etc. Basically it had like honeycomb pockets. The project was, through different means — chemically, sputtering, etc. — to fill these honeycomb pores with magnetic material, with some iron compound. And then the idea is that the ceramic honeycomb borders would be the separations between domains of a magnetic storage medium.

I still remember, at the time — you’d go around and say things like, “You could imagine putting a gigabyte on something the size of your hand!” But it was mind blowing! Just the whole idea that you have these nano honeycombs, and then you’d have a head that would sort of scan them.

Eric: Did you believe it? Or did you feel like that was crazy?

Ilan: No, I totally believed it! And the reason you’ll like the story is because MCM 41 stands for “Mobile Corporation Material #41.” So I’m still waiting for you to do a post — maybe you’ve done one and I’ve missed it — on all of the amazing discoveries, fundamental and otherwise, that came out of the oil and gas research labs. Mobile Corporation did all this incredible material science work. And, of course, they named all the compounds, like, “MCM 3, MCM 5.” So MCM 41 was, like, something everybody knows about...or at the time did!

Eric: That actually would be an interesting post too. Because I was just yelling [injecting historical stories into random conversations] at some people on Twitter the other day about the origins of TI [Texas Instruments], as just like, one of the early oil and gas firms that had a ton of early MIT ties. They just got very good at making their own instrumentation. And that’s a pretty high margin business. It’s a little less speculative. And they just like...I forget if it [TI] was a spinoff or if the firm [the original oil and gas firm] entirely pivoted. But yeah, maybe I should. If you have ideas!

Ilan: Oh yeah! Materials, lithium ion batteries, came out of an oil and gas lab. A bunch of solar cell work. I think there’s something cool there, but...

Videographer: Can we take a quick break?

[Camera Break]

Ilan: You were asking about the trajectory of ARIA?

Eric: Yeah, it would be good if we could discuss how...you were at ARPA-E. What in the world is Activate? All of these things, probably like, being dissatisfied in different ways. Or being happy with what you did, but dissatisfied drove you...?

Ilan: I think there’s a story of navigating, you know, continuing to basically try and do bigger and bigger metascience experiments, if you call it that. There’s a version of that which is, like, me being disillusioned by different institutional structures, or different silos, and how we do research. But maybe a more positive or interesting version is that, as I think over that time, is...I talked about being at CMU. When I did that research project at CMU, what flipped on for me was this idea of, “Holy shit, the wonder of the ways that we can take scientific discoveries and turn them into massive impact!” For me, the ultimate [thing] at the time — and this probably went all the way through my undergrad and into my PhD — the most valued currency for me in terms of changing the world with science was the ideas. And the instruments and the processes and the methods. I was so obsessed and in love with that, and I saw that as the big driver.

Interestingly, I had this experience 18 months into my PhD where we had written a Science paper. We were on the cover of Forbes for this new approach to creating solar cells much cheaper, you know, II-VI nanoparticles that we were going to print like newspaper. There was all this excitement. And I spent a couple of days with some business school folks with technical backgrounds, and realized it was all BS. We actually weren’t solving the right problems.

And I became...you could think about it as being disillusioned, but actually I think it’s more [that] I recognized that the world of ideas that I was in, where that was the currency, there was something missing there, in terms of how you drive impact.

And then I ended up in startups, VC-backed startups. My mindset moved from like, “Actually, ideas are pretty cheap, you know? Ideas are pretty easy to come across.” It’s more about, “How quickly can you iterate through them?”

But I also got obsessed with currency as the currency for driving impact. Because my job running a startup was, like, “Raise money.” And you could see that the more money you could raise, the more of an opportunity you had to drive and get to faster outcomes relative to competitors, or whatever else. So I really was in this mindset of money.

And having a chance to go to ARPA-E, one of the things I found very compelling was that it was a different mode from venture capitalists. I thought venture capital was sort of too narrow of an impact model for doing early-stage science, especially for industrial markets. But I also found compelling, like, “Oh wow. We’re going to be the biggest Skunk Works innovation funder in the US, for anything climate-related.” And I was attracted to this big money question. Money is like the driving force you have for reactions.

And now I think, through that time for me, at ARPA-E, my biggest recognition was that we had...

We had an incredible mission: to drive translational science into big impact in climate and energy. We had access to a lot of funding to do it, so we had the driving force. We had access to all these brilliant ideas, meaning every researcher who worked in any of these areas in the country would apply to our programs if we could just pick and choose the ideas. And the thing I realized was that, actually, none of those was actually the thing that mattered. What really mattered was people and institutions and incentives. And you can have all the money you want, and you can have all the ideas you want,  but if you don't have the right people in the right institutional environments with the right incentives to drive progress, you're pushing on a rope.

Eric: At ARPA-E, were you ever wowed by some set of people and institutions and incentives that you got to deal with?

Ilan: Well, in the end, I came back to...I left my startup feeling somehow like I had been misled. The idea of, “Venture Capital-backed startups are going to change the world.” We were working on batteries for EVs, industrial markets, very low margin business. Probably our timing was off. But I was like, I don’t see how we can develop the technology, get it to scale, and actually have it converge for someone like Vinod Khosla, who was like, “Either this needs to be a trillion dollar…” it was at the time billion dollar, now like, “...trillion dollar company, or it’s not worth doing.”

And so I kind of had a feeling when I went to ARPA-E, like, “Actually venture-backed startups are not the way to do this. And startups might not be the way to do this. Let’s figure out all the other institutional modes.” What I found in the end was that, to your question, of all the environments that I was funding research, the ones that felt most resonant and most productive were either the startups or the folks within academic labs that were already operating as though they were spinning out a company.

And what I realized was that the venture capital funding model has constraints, but if you don’t worry about the funding model for a second and you just think, “Well, what is a startup?” A startup is a vehicle for getting the right people into the right environment with the right incentives completely aligned with some translational research mission.

If you think about it, [for] everyone in an early-stage startup, the goal is to create value in a very concrete way. You’re probably doing something very speculative, if it’s science-based. Everyone who’s there in the early days has decided they’re going to commit the next chapter of their life to that pursuit. The incentive is clear. The alignment is clear. You often are getting people from different backgrounds, in interdisciplinary ways, to do it. So I think startups are massively valuable and important vehicles for R&D.

And the question becomes, and we can have a conversation about this, the question becomes, “How can we fund them? How can we fund them as R&D centers?” Venture capital is not a great way to do that. Some VCs actually do fund very applied R&D and take speculative bets. And I think those are companies that tend to change the world. But relative to the, like, $70 billion of US funding to applied and basic research — that number is probably dated right now — you don’t have anywhere near that going into super science-y startups.

Eric: To understand the problem...You love the shape of VC-funded startup groups — the people, environment, incentives they create — for the problems they want to attack. But there’s all sorts of problems that...

Ilan: They can do it if they see the wave coming and they’re just hopping on the wave. So when that happens, they work great. But why can’t you have the same aligned incentives, full-on commitment, before the wave even exists? And couldn’t that be a mechanism to create some of those waves?

Eric: And how do you think about making that happen...…

Ilan: I have no idea.

Eric: ...at ARIA?

Ilan: Oh, at ARIA!

Eric: Yes. And of course all of my readers will know I’m biased. I’ve spent this huge amount of time writing about the best early ARPA contractors, like BBN or the CMU autonomous vehicle groups...groups that were very startup-related, but really embraced technical ambition over market size. Their constraint was that they need some amount of grants [or contracts] to fund it. But I’m sure you have all sorts of theories on how to make this work for you [ARIA] in spots where there are parts of the R&D ecosystem that could use a little bolstering — at least in terms of ARIA’s incentives.

Ilan: Yeah. So one of the things we’re doing at ARIA, which I think is probably different than DARPA, is being really open-minded and attuned to, “How do we try and fund not just the best people and ideas, but with resonance in terms of the environments and incentives where they are doing the work?”

What does that mean? That means a few things. One is that we are very comfortable with the idea that, using ARIA funding, a researcher might decide to go start a company. They might decide to go, leave their academic lab, and just do the research as an independent researcher, because that’s what they think is the right thing.

We have something interesting, which is we do these seed funding awards. I mentioned those opportunity spaces where we say, “This is just the direction that we think is fertile.” A program director will create a program, which is their thesis, and that’s where they’re going to focus a lot of energy and where we’ll focus a lot of funding initially. But then what we do is we say, “Well, we’re not the only smart people around. And it’s all about increasing variance. So why don’t we find other people and let them start seeding ideas that could lead to the massive breakthroughs...[unintelligible]”

And this is like, you know, give us a three-page application, tell us why you’re obsessed with this idea, and [why] no one else will fund it. And we’ll give you up to £500k. So just start running at it. And if you come back with something compelling, we can double down with you. [We can] either make it a bigger project or actually have it inspire a whole program.

Eric: So Jenny said something in one of her interviews. — Jenny is the PD on the robotic dexterity program. She said at one , “I was almost disappointed that I didn’t get the chance to fund, like, some random person in their garage.” Because the funds are open to them. Can...

Ilan: Anything.

Eric: Can you explain what it would look like to be...like if I’m a random guy in Birmingham and I want to…?

Ilan: [Gestures at Eric] As opposed to a random guy in…

Eric: Yeah, in Chicago. [Laughter] And if I wanted to run it out of my garage, how much paperwork is involved? It seems to be a very fair [reasonable] amount.

Ilan: So this is really interesting. What we started with at ARIA is, we said, we wanted to make clear that we are completely agnostic. We wanted to fund the best people in the best environments. Apply. We don’t care if you’re at a university. Just tell us you’re obsessed with the idea, what the idea is, and show us that with our funds you can get access to the equipment you need to make it work. And we wanted to be really agnostic.

Now, there’s a piece of this, which is we have a brilliant, awesome, founding CFO for ARIA. Her name is Antonia [Jenkinson]. She and her team, I went to them and I said, “This is a little hard to do as a government agency. You got to do due diligence on these people. How are they using their money?” And they said, “Yeah, but this is what we’re built to do, so we’ll just figure out how to do it.”

Eric: And is it true that she walks around with the [ARIA] founding mandate in her pocket?

Ilan: [Laughter] She has been known to carry the ARIA Act around, which I love. So we started and said, “Yes, we can fund this [alternative applicants].” And seeds were a three-page application. We would tell you within three weeks whether you got the funding.

Eric: Three pages? Like...[makes a show of counting out three pages]

Ilan: Three pages, yeah. It was a three-page application, three weeks to funding decisions. And when we did it, we did have this. We had people apply. And then we had a few people who said...we had one person at a well-known university in the UK who said, “I’m really glad I’m getting this seed. To be honest, I don’t think I want to do this in my academic lab. For a number of reasons.” I think they were actually a postdoc. They were, sort of, earlier [career]. And they said, “What do you think about me, just like, finding and renting some space to do this work as an independent researcher?” The program director talked through it with them and said, “Actually, that makes sense. You’ll probably do a lot better. You’ll probably be more motivated.” Whatever it was. And so we just figured out how to give that person the award as an individual to do it in this way.

And we saw a few of those. Then we paused, and we said, “Well, wait a second. People are funded and excited to work on these projects.” And when we tell them we’re open to wherever you do it, they’re telling us, “Well, maybe I’ll do it some other way.” So the next time we did a seed call, as part of the questions we said, “What’s the institution you’re in now? If we award you the seed, where would you like to do the work? It doesn’t have to be the same place.” And the options were, “Will you do it in an academic environment?” You know, “my current university, another university, a company,” wherever else. And then we [included], “sStill undecided”.

First of all, I think we’re the only government funding agency in the world that gives that option. Secondly, the fact that there was an undecided bucket means, “We’re cool...like, we’ll talk through it.” I think we had like 23% of applicants in the next seed call say ‘“Undecided.”’

Which for me is such a big deal, because it suggests that being prompted with the question of like, “Eric, there’s a big project...it could be the most important thing you do in your life. Stop and think about, ‘What is the best environment for you to do this work and have it succeed and have a chance to change the world?’” And what we learned is like 23% of people [think], “Maybe it should be a startup, maybe this is my moment.” Or, “Maybe I should move to the UK and try to do this at a university in the UK,” which has happened now with ARIA grants. So, that was pretty cool.

Eric: Yeah…

Ilan: That felt like a...I warned you that if you get me excited about something, I’m just going to keep talking about it!

Eric: No, no, no! That was great! So I have a question [from my notes] lingering here, I don’t know where to put it. So I’ll just ask you now.

Ilan: Go ahead!

Eric: So this one comes from Tom Kalil. He said, “In what specific ways do you desire to transcend the DARPA model?” Because you’re not looking to set up some version of an American ARPA in the UK?

Ilan: I think that’s right.

Eric: There’s lessons you’re taking, but…

Ilan: Yeah, I mean look...I think DARPA is such incredible inspiration. You know what I mean. Like people argue about, “Is DARPA past its heyday?” “Is it doing good or bad?” But we have this beautiful gift, which is just the history and the reality and the myth of DARPA all wrapped in. And, honestly, if you can’t be inspired by that, like, forget you! Right? And I think that inspiration is so important and true.

We’ve touched on some of the things that I’m hoping with ARIA...ARPA, like you said, established 70 years ago in a very different time. It’s changed in many ways. I think for me, the most important things — one of them is one of the things we talked about earlier, which is having the intellectual humility to, and the eye on part of our job as just to increase variance and learn and pick up on threads that are valuable. That is really important to me.

One of the things I’ve noticed — and this is good and bad — but when our program directors...we pair them up with DARPA PMs, former and current, to give advice and whatever else. And oftentimes I will hear a DARPA PM say, “Wait, 37 performers for this program? You can’t do that. That’s not how we do things. That’s not the right way to do it.” What we’ve had the chance to do is kind of go to first principles on everything. And that’s a real case. I don’t think it’s 37. But davidad, who runs our Safeguarded AI program...for the first technical area of his program, he wants to, as quickly as possible, answer some theory questions and develop some frameworks which he thinks are best done as open source. And, actually, where that led him is we’re funding a lot of teams with small awards, to which normally people would say, “It’s not worth the management,” or “It’s not cohesive enough.” But the first principles suggest that, yeah, that’s the right thing to do there.

So, it’s kind of like having the intellectual humility to say, “Okay, I don’t know if this is exactly right. But it feels like it’s matching the first principles. And across what we’re doing, it gives us variance. We’ll learn more on the go.” I think that’s one thing that I’m hoping gets preserved at ARIA.

This other thing we’re talking about is probably the other most important one, which is...DARPA, rightfully, has a mantra of, “We don’t have our own labs. We don’t create institutions. We fund research projects.” And DARPA increasingly has done more with startups, which I think is great and one of the biggest impact vectors it has. But I think for us, for ARIA, being able to say from the beginning, “Actually, the people and the institutions matter. And there may be new institutions that we need to form — or that we catalyze forming. And we’re not going to be, sort of, scared about that,” is important.

Eric: Yeah. This is also something I think early DARPA was a bit friendlier to. They wouldn’t necessarily found an org, but they would write a pretty big check to an org that, like, didn’t seem to have an office yet or something like that.

Ilan: Was RAND DARPA?

Eric: Was RAND DARPA...

Ilan: I remember reading something that...

Eric: No, I think RAND would’ve been...I do think the USC ISI [Information Sciences Institute] was very clearly heavily early DARPA funded [from its earliest days, which helped create the organization of largely RAND alums]. They ran DARPA’s early MOSIS initiative, which was one of the first fabless [services]. That’s definitely a case of...those guys may have left RAND to form...I’d have to go double check some of my notes. But that’s a case of them [DARPA] writing [very early checks into an org] and making some promises, and it was an institution they needed.

Ilan: Yeah, it’s actually interesting! MOSIS came up recently at ARIA because, one of the programs that we’ve launched...we’re already starting to see that, within this...

[Camera Break]

Eric: You were talking about MOSIS and how it fits into one of your programs that you were talking about recently.

Ilan: Yeah, the opportunity space is Nature Computes Better. One of the theses there is that there have been lots of attempts to think about how you disrupt general computing. But it’s such an impossible thing to think about doing because of the massive supply chain and all the different forms of general compute. Suraj Bramhavar, one of our program directors, his insight was, “Actually, AI is very unique in the history of computing because you have a compute mode with a pretty narrow set of mathematical primitives, but potentially massive applicability, value, and impact. And that narrow mode of AI could be a new foothold to think about alternative types of computation — with alternative physics, alternative hardware, alternative substrates, etc.”

And the other piece of this is...when we think about, “Is there a program or opportunity space here,” one of the important questions to ask is, “What is the fundamental limit of performance that could happen in this space? And where are we now?” And one of the things we know about computation is,[even though] we’ve had Moore’s law for as long as we have, we are still orders of magnitude from the fundamental limits of computation. Whether it’s [the fundamental limit based on] Shannon’s information theory or what we see happening in nature. So the idea behind the space is, “Can we find new approaches inspired by physical systems, natural systems, other ideas that researchers have, etc. to use AI as a foothold for next gen compute?” The program that we’ve shaped aims to demonstrate that you can show, as a first step, AI training at a thousandth the cost and energy consumption than state of the art today.

And there are a bunch of really interesting systems questions around, like, “How do you even prove that in sort of a demo system?” We’ve now funded a set of teams across academia, startups, big companies, and a number of approaches that we think, in coordination, can get there. And from that work we’re already seeing, “Wow, there might be a next phase of this given some of the early excitement that we have.”

How would you take advantage of it? Maybe you need some MOSIS-like institution that is focused on this area. It’s still very early, but it’s cool because, like, you had your post on MOSIS and it actually is making a difference, being able to give us [ARIA] a catalyst of things to chew on.

Eric: And can you paint a picture of what this MOSIS would be doing? Like if a lot of these contractors are point solutions or different approaches, what would you need them [this MOSIS] to do?

Ilan: [Laughter] I don’t think I could paint a picture of that for Suraj’s program because he’s literally just in the early stages of thinking about it and I don’t think I could actually talk through it without disclosing things that he probably wouldn’t want me to. But I do think, you know, one of the things we talked about earlier was the question of, “Where are there new institutions, and how could they help ARIA?” We talked about our Creators. (DARPA talks about the people they fund as ‘performers.’) We thought that was a little odd. Actually, you might know the origin story of where ‘performers’ came from. Maybe it’s like: you want performance, so you have performers? Anyway, we call the people and institutions we fund ‘Creators.’ The idea being, the work we do at ARIA, I think you need equal doses of creativity and creation. This word ‘create’ is interesting because you can think about it in terms of creativity — which is like pie in the sky thinking — or you can think about it as like, “No! Go create something, like building!” And our view is you need both of those to come together.

Anyway, we talked about new institutions around Creators. We’ve also been thinking about new institutions to help ARIA do better. You might have seen our Activation Partners call.

Eric: Yes. Can you explain what that is?

Ilan: I can. [Laughter] This might be a divergence, but...

Eric: We can come back to it later if you prefer.

Ilan: No, let’s do it! When we think about the ARIA model as a whole, we talked about finding the opportunity spaces, being able to inject energy, catalyze these big waves — which are essentially new movements of technology...but then talent and capital that can transform the world in one of these spaces.

There is a question. We’re not DARPA. One of the things that allows DARPA to get things from early idea/technology into practical systems in the field is that they have this massive lever that is the Department of Defense budget. [Laughter]

Eric: Yeah, they have a built-in customer.

Ilan: Yeah, they have a built-in customer. And there is no bigger driving force than the procurement power of the Department of Defense, if wielded in the correct way. ARIA’s mandate is much broader. We’re not just defense, we’re actually not doing any defense right now — it’s quality of life and economic growth, broadly speaking. We don’t expect to have a built-in customer from government. There might actually be cases where government, like the NHS [National Health Service] could be an amazing partner and uptake vehicle for technologies we develop. But outside of having purchasing power as the driving force, our bet is that the biggest driving force that exists to get new ideas into the world at scale is going to be entrepreneurship.

It’s that whole idea of, “If you, as the researcher and a team of people are willing to commit their life for a stretch of time, yes, you could change the world.” We know it. We’ve seen that happen.

So, one of the questions we’re trying to grapple with is, we’ve created these opportunity spaces specifically to be places where the wave doesn’t yet exist. So, there probably aren’t a lot of startups in these spaces because venture capitalists aren’t super active in all these spaces. Maybe there are some. What can we do to increase the amount of reactants that are entrepreneurial in the opportunity spaces we have?

And we thought about a few things. When I was at ARPA-E, actually, we ran an experiment that I helped launch, which is called the technology to market program. We basically built, within a government funding agency, a business analysis/biz dev type function. The idea being, “Let’s figure out how to inject into our programs some thinking around how you translate the technologies.” DARPA does some of this.

Our view is actually, for ARIA, there are some things we can do internally...but to think about how technology’s transition to market, especially through startups and entrepreneurship, the best people in the world doing that are going to be people doing it in the field. And you might be able to attract one or two of them, here and there, into a government funding agency, but you’re not going to be the best in the world at doing that. So what we did was we put out a call, and we said, “Listen, we think there is going to be massive opportunities and value created in these spaces. It’s still early. If you are an organization that deals with venture talent creation, entrepreneurship, rapid prototyping...anything related to tech translation, transitions, and value creation...and you’re amazing at what you do, could you imagine focusing your energy on some of our spaces? And just do what you do really well, but inject more reactants into that system.”

My colleague Pippy [James], who’s our Chief Product Officer, was really the visionary behind driving this forward. Where we ended up was, we have a handful of organizations. Some of them are venture capital firms that are saying, “We will, for the first time, focus on the UK and look at opportunity creation in these spaces.” They’re doing things like running fellowship programs for scientists who want to be entrepreneurs.

Renaissance Philanthropy is another Activation Partner thinking about, “How do we create a two-sided marketplace between the technologies that start to emerge and philanthropic goals/philanthropic funding?”

Eric: A lot of people don’t think about it, philanthropic demand is demand all the same. They [philanthropists] often have particular technologies they want to bring into existence. But it’s often unclear how to solicit demand from philanthropic...the mass of philanthropies. It’s hard to understand what they want.

Ilan: Yeah, and in a lot of these spaces it’s not clear that when we’re done with our program, the commercial value is going to be right there. It may be that we’re still in a tragedy of the commons, market failure mode, and we need to band together with others to do more in those spaces. So I think that’s going to be important.

But one of the things that came to mind as we were talking earlier is one of the Activation Partners we have, I think you’ve spent time with them, is this company Amodo [Design].

Eric: Amodo, yeah! Love Tom [Milton]!

Ilan: Amodo is such an interesting story. They’re based in Sheffield. It’s a small team. It just reminds me of the things you write about! It’s a small team that, you know, they’ve been involved in a spin out of a university and they basically recognize that, like, “Man, universities don’t do a very good job of prototyping things, having a rapid iteration mindset, or understanding how to get things into a more commercial place. Why don’t we just set up a shop to do that as a service — for startups, academics, and otherwise?”

Eric: So you don’t have to shoehorn your experiments into off-the-shelf Thermo-Fisher equipment. Tom and them can make what you need.

Ilan: Exactly! “Let’s go solve it!” They’re like mechanical, electrical, prototyping fixers!

Eric: And they’re hiring!

Ilan: And they’re hiring! [Laughter] So, interestingly, they applied to be an Activation Partner, alongside...you know, our other Activation Partners are, like, Google DeepMind...but they applied. And first you look at it, and you’re like, “Oh you’re five people in a room in Sheffield?” And then when we talked to them and we saw what they were doing and the vision, it was like, “Actually, this is a massive asset for ARIA to be successful.”

And maybe it won’t just be Amodo over time, maybe we’ll find others. But already, through the arrangement that we’ve built, they’re actually doing this as a service for a bunch of Creators that we’re funding, who are saying like, “Oh my goodness, I thought it was going to take us six months to do this and I got a much better version of it in a week!” So I think that’s super cool.

Eric: And the PDs have seemed very impressed with them. Like when I was talking to some of the PDs [and other ARIA staff] about the BBN thing, the concept that I used was, “Oh, do you want an Amodo for your area?”

Ilan: And that’s one of the things we’re doing through Activation Partners, which, again, is sort of an experiment around this idea of, “The best people, in the right environment, with [the right] incentives.” We’re working with Convergent Research on what I think will be the first experiment of them saying, “Wide open to the spaces,” which will be our opportunity spaces. Instead of being a kind of two-sided marketplace and connector of talent with philanthropic or other funders for FROs (Focused Research Organizations), they’re basically saying, “If you’re someone talented who wants to commit your life to building a focused research organization in one of ARIA’s]opportunity spaces, apply! Then we’re going to find one, maybe even more, to go launch.” And it’s exactly what you said...

Eric: To people in the metascience space, I think Convergent is maybe one of the most exciting...

Ilan: Yeah.

Eric: ...kinds of Activation Partnerships. Because I think everybody [metascience nerds] was around when they first kicked up. And they also always knew [suspected] the vision was probably that Convergent might like to win NIH funds, or something like that, and set up more every single year. So to have an official [Activation] partnership was exciting, for somebody who’s just been a fan for a while.

Ilan: I think the biggest — we might agree on this — but I think probably the most important thing to accelerate progress in the research ecosystem is to increase the diversity of institutional types. Convergent is one of the few examples we have of a mode that is taking root, that’s saying, “Here’s a new institutional type. Let’s figure out how to scale it!” You talk to Adam [Marblestone] and he talks about...you know, he sees a thousand — I think he says either a hundred or a thousand — FRO-shaped things that need to be created in the world. I think it’s a thousand. I imagine there are a thousand. If he thinks it’s a hundred, I can probably debate it. [Laughter]

Eric: It was one of those two numbers. One more question, and then I’m going to do a little pivot. You seem to have a very good grasp of early/mid-20th Century R&D, the institutions, some of the stories, etc. Is there any organization or group from that period you find yourself thinking a lot about? Either because you want somebody to emulate it or you just think it’s cool?

Ilan: That’s a good question. I think the ones I think about are maybe, not surprisingly, the ones that feel like startups before their time. You wrote the piece about Edison.

Eric: Yeah.

Ilan: That was a story I was already obsessed with...but the whole idea of, like, Edison in the workshop. There’s an organ in the back, there are bunk beds, they’re sleeping there. They’re just all in, committed. You know, he’s raising money based on a dream. You think it’s bullshit, but then he makes it true by doing the work to close the loop.

Eric: The financiers come to visit and he looks crazy...

Ilan: That’s one piece. I love the story of Philo Farnsworth. I don’t know how much you know about that one?

Eric: I don’t know that one.

Ilan: 15-year-old farm boy who has a vision of making television. And basically through some serendipitous set of events, because he’s so passionate about it, ends up moving to San Francisco, getting funded by like an LA financier from Hollywood, and is competing with RCA in a race for the first television. And you just look at that, and look at the lab, and look at the talent times intrinsic motivation in the right place and the right time...

And I think Curie — this is more academic — but I think Marie Curie is another example of this. It doesn’t seem like it, but when you look at how she and Pierre Curie worked, it definitely feels like a startup.

Eric: What about it is very startup-y to you?

Ilan: Oh, the combination of the singular focus and the idea of, “We’re going to be open-minded and try to get to the goal through all means.” I always love the story of — I might have this wrong — but my impression is that Pierre Curie discovered piezoelectricity as part of trying to figure out how to very sensitively measure the radioactivity in the materials that he and Marie were looking at. So, basically, you have an academic pursuit to discover and understand this phenomenon of radium that no one understood, that leads to stumbling into discovering a whole other phenomenon, piezoelectricity — which has massive implications technologically. And it’s all because you had this intensity of spirit in one place.

[MIT’s] Rad Lab is probably another example of this. I don’t know if you’ve written a piece on Rad Lab.

Eric: I’ve never written a piece on the Rad Lab. There’s one book that’s continually over $100. And I check Amazon like once a month to see if the price will come down and I can finally get it. [Laughter]

Ilan: I’m going to buy you that book, whatever it is.

Eric: You don’t have to do that!

Ilan: If it means you’re going to write a piece about Rad Lab, then I think we all...we’ll do like a GoFundMe campaign to buy you the book!

Eric: [Laughter] I can promise to find a way to have it out in the next year, somehow! I have a few other Rad Lab oral histories sitting on my shelf that I’ve been letting pile up, so I do have to do this.

Ilan: Yeah, but the whole idea that because we got a group of physicists together to try and build radar systems, what fell out of that was, like, MRI...and the basic theoretical and practical things around that is just awesome!

Eric: Yeah, and also in terms of funding something for five years [as FROs tend to be], and then it's set off in a direction of its own...people at Rad Lab seem to have thought of some of Weiner’s early cybernetics writing, and things of that sort, as helping set the agenda for the RLE [Research Laboratory of Electronics, which the Rad Lab morphed into postwar] going into peacetime. It’s very clear that [the Rad Lab] was catalytic in extreme ways. The kind of thing that would be ARIA ‘returning the fund’ level [results].

Ilan: Yeah, it’s funny, one of the things that’s coming to mind here, as we think about FROs...one of the things I think a lot about is, for so many of these institutions, and I feel this way about ARIA right now. We’re two years in from starting everything from scratch. It feels like we have the environment where these amazing things can come out of it. When I think of our program directors, when I think about the Creators they’re starting to fund, the dynamic of the team, I’m like, “Wow. This has the energy and the characteristics you would want.”

The big fear is, I think, “Okay, this is founder mode.” [Laughter] These are early days! If you look at Rad Lab, it was this intensive thing, and MRI and these other things came out of it in the early days. And more came out of it later, but that question of, “How do you capture that founding period and not lose it?” I mean, what I love about FROs is the idea is, “Don’t even try.” Imagine that you’re going to have an intensive thing for five years. And then it turns into something else, like it metamorphizes. I think there’s something really important about that.

Eric: I remember hearing Adam muse at one point about the idea of, “Oh, it would be great if we had funders who could accept or were excited about the idea of maybe one FRO maybe leading into another FRO.” That is the outcome. You [ARIA] are playing a bit more of a repeated game. How do you think about that, one ARIA program leading to another ARIA program?

Ilan: Oh, 100%. I think most likely that is how we will make an impact, which is we’ll have a program that inspires the next program. And within that next program, we see something and we double down on it...and that’s the thing that happens, being adaptive in that way.

One of the interesting things that I’m noticing is...we haven’t talked much about how we recruit program directors. But the combination of our broad mandate and the fact that we are bringing in, as cohorts, program directors from such different backgrounds...we have these opportunity spaces that the program directors have shaped, and, yet, we’re now starting to see as we dig into them, these really interesting intersections between them. And I’m just wondering what’s going to happen. What are we going to do about that? Are we going to have programs that sit across two opportunity spaces? Are they going to merge into one? You know?

Eric: And can we talk about your new program directors? Who are they? Why did you choose to light up flares around these particular people and their expertise, or their point of view?

Ilan: Yeah. I think this recording is going to come out right after we have announced them. It’s probably worth first saying, we made a bet early on at ARIA. And my colleague Pippy had a really big role in this, which is we basically decided that one of the hardest parts for a research organization like this in the UK was going to be to find program directors or program managers, because, you know, it’s a weird thing to do. You’re on some amazing career trajectory and you decide to, like, step out of that for three to five years to do this other weird thing with a brand new agency that no one has ever heard of. Probably the listeners of this realize that one of the core parts of the DARPA model, which we’ve adopted at ARIA, is this idea of term limits. The people who are developing the theses are people that are coming in from the front lines of doing research in some way. They’ve got to sprint for three to five years to develop a thesis, fund a program, have an impulse function that hopefully catalyzes change, and then they go out and we refresh with some new program director. And part of that is to keep the ideas fresh, but actually, a big piece of it is to keep it incentive-aligned.

I’m never worried that a program director is going to make a decision on their program based on worrying about their career trajectory and what their boss at ARIA thinks, because they’re going to expire in three years. So they’re all just purely focused on, “How do I use this time to make something happen?”

One of the bets we made was, we knew it was going to be a challenge to find people. So we decided, rather than just hire people one-off, as we found them, let’s actually do what we believe is the most open call of its kind to find program directors. Let’s imagine that we can think of a set of characteristics that matter and that we think are going to make for a good program director. And that’s going to be more important than specifically their resume or what they’ve done before. And then we decided that we’ll bring them in as a cohort. So, part of the idea here was, if it’s going to be a risky thing for somebody to come join ARIA at the early stages, it becomes a lot less risky if, when they’re thinking about joining, they can look around and see five or six other people that they think are pretty awesome, who are also going to jump into that pursuit.

And I think that worked out really well. If you look at the first set of program directors, we’ve got someone like davidad who, I think he got his masters degree from MIT at age 15. I don’t think he ever finished his PhD. He ended up working at places like Twitter doing independent research. And he sits in our office next to Jenny Read, who is like a seasoned senior professor at the University of Newcastle — in different fields.

And you have all these other people, and they’re so spiritually aligned, and respect each other, and push each other. It’s really cool. [pause] We were talking about new program directors, but I didn’t get to it.

Eric: No, no, it’s okay!

Ilan: What would you want to see from ARIA’s next program directors?

Eric: [Laughter] What would I want to see?

Ilan: Trying to figure out how to...oh you’ve met some of our, you’ve met our existing program directors! If I were to say, like, “Okay, you have the chance to double the size of the group of program directors and make it an even stronger group. What would you try and sprinkle into the mix?”

Eric: So, that’s tough for me [laughter]...because I’m not a technical genius or a polymath in any way. I’ve been very intrigued by the energy of everybody involved [as an ARIA program director] and I’ve always left excited. But I guess, I don’t know. Engaging in this work as a non-technical person...

Ilan: Yeah.

Eric: ...I more just go in with the assumption of, “Somebody sees genius or a spark of something in this person. I’d love to help them any way I could!” I guess, reading the history... it’s unclear that it would be obvious to anybody, or me going into the past that, like, Frank Heart would be Frank Heart, or J.C.R. Licklider would be Licklider. So, I guess I don’t know...I attempt not to answer these questions or go down this line of thinking because I think I would be biased by random things, probably. For example, I go talk to [ARIA-related] people, and most of them are British. And it’s like, this is a new cultural context for me. Sometimes, with people in my personal work, I go based on spark or energy or something like that. But I don’t think that’s quite how a British scientist would do it. So I don’t really know. I won’t answer your question, essentially!

[Post-interview note: As I transcribe this interview, I wish I would have flagged that some social science program like a computational law program could be cool!]

Ilan: Yeah, interestingly, it goes to the intellectual humility point. You said something that resonated with me, which is like, “I imagine someone saw a spark of genius or brilliance in them.” So we designed our process to recruit program directors — I think we started, the first time we did it, we had seven qualities we were looking for. And they have names, like, obviously like ‘technical depth,’ but we have one called ‘vision to action,’ ‘adaptability,’ etc. We basically put all the applicants through the paces in different formats. We were probing them in different ways to test along those axes. And the idea is not to just have some aggregate score for someone who’s great, but the idea is to find the spikes. We talk a lot about how each of our program directors has a different set of spikes, meaning like superpowers that they’re going to leverage to do the job in their own ray, recognizing that a Licklider has very different strengths than a Bob Taylor, right?

And I think that’s worked out really well. We did the same thing with this new cohort of program directors. But we looked at our first set of program directors and what we learned from them, and we realized a few things. One was, there was something that they were showing in their behavior that we hadn’t tested against, which is something like ‘value recognition’ or ‘opportunity recognition.’ The ability to see the dots and find the constellation within them. So we added that to the mix. We also deliberately tried to recruit...our first set of program directors, roughly 2/3rds to 3/4s of them have more academic backgrounds. They tend to be sort of — I don’t want to offend anyone — misfits? They tend to have different mindsets than your typical academic, that’s why they’re attracted to this role. One thing we tried to do deliberately with the new program directors is say, “Let’s try and bias towards more people with industry or startup experience,” which is the case. So, what can I tell you about them?

Eric: Can you give us a couple examples of program directors who clearly have overlap with an opportunity space from the first cohort? And any examples of people who are, kind of, breaking new ground?

Ilan: Yeah, so a lot of them are breaking new ground. Actually, we’re already starting to see a bunch of different connectivity between them.

I’ll share one. One of our incoming program directors, Rico [Chandra], is Swiss. He studied at CERN, he’s a nuclear physicist. He then decided to go do a startup in the nuclear security space. He’s coming into ARIA, and the thing he’s become obsessed with is this question of...the fact that with all of our modern technology, flight and the ability to transport things in the air is still a big challenge, especially sustainably. And his view is that, “Actually, we know that there are plenty of birds, albatrosses and others, that can go vast distances, thousands of kilometers, without any source of power, in perpetual flight. Why is it that we can’t beat them with modern technology?”

So, you know, with our incoming program directors, we basically say, “Listen, just have a starting point, and then we’ll go from there.” We don’t know if it’ll turn into a program, if it might pivot or not.

Eric: It’s also interesting because more recent people in aeronautical engineering who we think of as geniuses, like Kelly Johnson, [often] thought in wind tunnels. But when you go back to the Wright brothers, they were obsessed with birds. Like there were all these stories of them on the beach where they were apparently — I forget if it was Wilbur or Orville or both — they were impeccable at imitating the motion of the birds’ wings.

Ilan: Oh I love that.

Eric: [gesturing to imitate the Wright brothers imitating the birds] As they were trying to figure it out. So it’s fun that...to find a really ambitious program, he’s also kind of gone back to the source.

Ilan: So one of the interesting things about Rico is he also happens to be, like, a world-record holding long distance hang glider. So he’s been thinking about this a lot. We don’t know whether it’s a program, but one of the things we realized...we have an opportunity space called Scoping Our Planet. And the thesis there is, “With climate change, the thing we really care about is the climate changing. We obviously care about and know that fossil fuels and greenhouse gas emissions are causing the climate to change. But there’s very little we know and very little energy has been put in by the innovation community to understand how we monitor the climate.”

And you can think about why that’s important. Everything from measurement and verification of things like carbon removal, which is sort of a big and unsolved market failure problem. But you can also think about weather prediction. You can think about understanding climate tipping points, which is what our program is focused on. Rico came in and we realized, “Oh, is this like a new aeronautics opportunity space?” And then we realized, “Well, maybe this is just the capability that can be very valuable in scoping our planet.” So now he’s talking to Sarah [Bohndiek] and Gemma [Bale] [the Scoping Our Planet program directors] and trying to understand, “Is this something that can be game-changing around how we have new sensors for parameterizing the earth and communication protocols?” So that’s kind of an interesting, surprising one [new program idea] that connects to something we’ve done.

Eric: So, it seems like you really do hire people first, and find the specific programs later. A lot of your first cohort of program directors are working on programs that, like, it’s maybe not what their academic lab focuses on. Are there any people you’ve hired in this new cohort where you think, “Oh, what they might end up working on is exceptionally broad.” You’ve hired them with a guess, but it’s hard to tell where precisely it will go.

Ilan: I think that’s true of a number of them. I was just chatting with one of our incoming program directors, Alex [Obadia]. Alex is really interesting. He’s a mathematician by training, who then got involved in cryptography. He’s been doing essentially crypto and blockchain stuff, but he’s really passionate about thinking about how the kernels of innovation in real cryptography, that’s being applied to financial systems and otherwise, are going to be consequential to the future of society.

He turned me on to — folks should look this up if they haven’t — this paper on programmable cryptography. The idea there is, right now you think of cryptography as, “Okay, I have a certain piece of data in a discreet way. It’s encrypted or it’s not. I can see it or not.” There’s a whole community of people, now, thinking about, “Could I have a whole program that’s encrypted? Could I have a computer program that goes and does programming on your data without actually seeing your data, because of the encryption?” Or even a computer program that is doing computation and getting answers based on data where no one actually knows what the program is. And you think, “Like, what are we talking about here?” But clearly, as we think about AI, and AI agents playing a bigger role in our lives, there is a question of, “How do we optimize the efficiency of what we can learn and the value we can create with AI, where one of the big barriers is privacy of data?”

So, you can imagine — and I’m not an expert here — but the sense I get is, you can imagine programmable cryptography allowing me to, say, have a set of preferences, my values, things that are very sensitive to me, where I can tell an AI agent that is going and engaging in some democratic process on my behalf. Or I can have an election process that runs a program where it queries, “What are my really sensitive personal preferences?” and has it feed into an answer of a program without ever seeing my data.

So I got very excited about this! And I said, “Alex, this is massive, this is definitely an opportunity space!” And his view was, “Yeah, but I think enough people are already working on this, it might happen anyway.” And we got in this big argument. So now he’s thinking about, “Actually, what’s the next stage beyond that?” We have this massive convergence that’s coming, where we’re going to be fusing humans with technology in new ways. You can think of neurotech. You can think of, you know, just the way we’re much closer to devices. What does preserving privacy and trust look like when your devices are not purely digital, purely physical, or purely biological, but somehow sit at the intersection? I don’t know whether there’s a ‘there’ there, but that’s what he’s starting to explore.

Eric: You seem to have a number of life sciences-adjacent folks in the new cohort.

Ilan: We do.

Eric: What’s the thinking behind that?

Ilan: It’s a good question. Again, it’s another indicator that we really were people-first. When we got the first cohort, we realized there were really only two program directors working on things related to life sciences and biology — Angie, on Programmable Plants, and then Jacques [Carolan], who has the Precision Neurotech program within the Scalable Neurotechnology space. But if you look at the UK, it has incredible strengths in biology. If you look at technology vectors that are progressing extremely quickly — synthetic biology, right? You have all these pieces. So, shouldn’t we have more?

We do have more program directors thinking about biology in the new cohort. Brian [Wang] is an organic chemist, but then moved into thinking about pandemic preparedness. And he still cares about that very deeply, and one of the questions he’s asking is, “Can we utilize new learnings about the innate immune system?” The innate immune system is the part of your body that reacts to pathogens and toxic in broad spectrum ways, right? So it’s not creating antibodies to specifically attack a specific vector, it’s just that initial response. It turns out, actually, the way plants deal with pathogens is entirely innate immune system. And his view is, “Well, maybe we can be inspired by that to figure out how to create therapeutics that can be effective against new pathogens, new bugs, that we haven’t seen before.” And there could be a number of implications to that work where, through that same inspiration, basically the question is, “How do you get ahead of things that are evolving?” You know, you think about the Covid experience. But cancer is the same way, right? How do you get out ahead of something that is a foreign...

Eric: So, when you’re selecting program directors, you’re in a room and you have some exceptional electrical engineer who wants to work on some project nobody has ever done before. You might have a biophysicist interested in some medical area. You might have somebody from a field that doesn’t even have a name. How do you go about...? You have to come out of there with eight people. What does that prioritization even look like? Is it painful, because there truly is no apples-to-apples comparison? How does it feel? What do you do?

Ilan: No, this is actually where the approach...Historically, I think I always felt like I had a pretty good instinct on talent. You know, you hear about this in research and otherwise. I mean, you’ve written about this, which is just, like, there’s a taste and there’s ability to spot talent. And that’s something I always try to be attuned to, and refine.

I think over time I’ve come to realize that one of the best ways to find talent is actually to just create a product that really talented people want, that doesn’t exist. If you think about the ‘product’ we have at ARIA, you have, in this very open way, an ability to take £50 million and figure out how to create a wave that changes the world. That’s really compelling if people believe you. So that’s one thing that I think has changed where you say, “Okay, that’s a great way to attract talent.”

The other thing is, and the big lesson for me around our program director recruitment, by trying to strip away to first principles around, like, “What do we really care [about] in terms of the characteristics of these people?”...and this is very much like a Kahneman-based approach of recruiting and selecting people, which is just like, “We’re going to have the criteria. We’re going to probe the system a bunch of ways. We’re going to see how they spike against the criteria.” And ideally where that ends up, to your point...when we went through that process based on paper applications, phone call screens, technical interviews, we ended up with this set of people — this year I think it was 16 — where we said, “Based on everything we’ve seen, any one of these 16 people could be a program director.” Because who are we to know? Like again, the intellectual humility. I could guess, but I’d probably be wrong.

So then we said, “Forget it. Rather than just trying to pick the right ones out of the 16, let’s actually bring the 16 people together.” One thing that’ll happen is...these are awesome people who will go drive impact in the world, and they’ll probably all do more if they’re connected to each other. But it’ll [also] give us one more way to look at how they show up, not just as individuals, but also how they sort of connect to each other.

Basically, at the end of the process, we’re just thinking about the portfolio. We’re thinking about the portfolio of people, portfolio of ideas, based on what we think they might do, what they might bring to the cohort. Just curating based on that.

Eric: If I can probe you with a bit of an edge case from history, I’d love to get your...

Ilan: Oh boy, I might not...

Eric: So, speaking of intellectual humility, I’m a big Warren Weaver fan.

Ilan: Yes!

Eric: And something that comes out all throughout his writing...he is an exceptionally...he’s a Midwesterner with a lot of humility. It comes out in all sorts of his [writing]. So when he first came to the Rockefeller Foundation, even though he had humility, he had one of the more extreme bets in philanthropic history. The [Rockefeller] Natural Sciences Division, which gave out on the order of what ARIA gives out per year, maybe a little less. It’s hard to do a one-to-one, because science was cheaper back then. But, essentially, they [Weaver’s predecessors] said, “Oh we fund electrical engineering, zoology, etc. And we fund the very best people who come in.” And when he came in, he said, “That’s fantastic. We’re not going to do that anymore.”

When he’d gone around and talked to people, he said, “There’s something right there in that area between biology and physics. So, yes, our budget is really big, but it’s very finite. We’re going to press on that.” And he essentially put 80% of his budget towards — it was 1932 — five years later, he would name it ‘molecular biology.’

Ilan: Out of curiosity, do you know how long he was in the foundation before he made that bet? You probably do.

Eric: I think he came in with the bet.

Ilan: Oh okay!

Eric: He took the interview with them to say, “I don’t think I’m your guy. I’m just an applied mathematician. But there’s this thing..."

Ilan: “Here’s the thing we would do!”

Eric: Yeah. He read very broadly, for sure. And he kept at it [funding molecular biology] for 20 years before moving on to something else. But if you all felt you had an ARPAnet or a molecular biology on your hands, how much is too much of your funds focused on that one area? There’s maybe a limit. 80% is a lot, for example.

Ilan: Yeah. This is interesting. This is something we talk about a lot. First of all, the question of, “We have these seven opportunity spaces. Why are we recruiting more program directors? You know, aren’t we doing enough? Like the goal isn’t just to boil the ocean.” And actually, we tend to think really deliberately about, “We need to get to this one outcome that is so massive. Do we think we’re touching enough surface area right now to fast forward ten years, and realize we got lucky and we just noticed that thing that was going to be so valuable?”

And when we looked at our budget, we looked at the size of the UK ecosystem, and we looked at what we were doing, we said, “Actually, no. We’re probably not touching enough surface area.” So we now have more program directors coming in. There will be some new opportunity spaces. We’ve also said, “There’s a limit.”

It’s likely that every other year ARIA will recruit a set of program directors. It’s a three to five year term. So that gets you into a steady state of 15–20 program directors. And that’ll be it. So we’ll have a first set of opportunity spaces. That’ll be the surface area that we touch. And really what success needs to look like is...out of that, we’ve found something that we not just want to double down on, but 10X on. And so if a big portion of ARIA’s budget does not end up in one of those areas over the others, there’s a problem.

Eric: Oh wow.

Ilan: That’s my take! You know, I’m on a term limit, too! I probably won’t be the CEO when they have to figure this out. But is that 80%? I could imagine it. We talk a lot about…

Eric: And could the person in charge of ARIA...

Ilan: Decide?

Eric: ...if they were a brave person, if they thought it was right, is that that? Are there some structural or institutional barriers that would make bravery even tougher, beyond the pure pressure of people looking at you like you’re a little crazy or something?

Ilan: One of the nice things about ARIA, it’s worth mentioning, the UK — like a number of people who were in UK government, civil servants, others, Parliament — they got this right. In the sense of, ARIA really does have the right mandate, freedom, flexibilities. There’s not political intervention, there’s not a lot of BS processes...they’re the right processes to make sure we’re responsible stewards of taxpayer funds. But one of those things is the CEO of ARIA makes programmatic decisions; that’s very much modeled after DARPA. I think that’s 100% true. And that’s good to have in place from a governance perspective. The question is, “Will the organization have a culture that leans towards the bold bet?

We’ve been talking about this a lot. So far in ARIA’s history, the things we’re most proud of are when we’ve gotten exposed to something and then we just said, “Ok, let’s do the bolder thing here.” And it happens in big and small ways. Sometimes it’s about a program director we decided to bring in. Sometimes it’s about a program that we launched. Sometimes it’s like, “Do we use Slack?” Because there are potentially any number of issues from having your Creator community on a Slack system. We just had this conversation and said, “We should bias towards doing the bold thing.” [Laughter]

And my hope is that we build into the agency that a leader in the future will be celebrated by pushing that bold bet.

Eric: I had a question written down, and I think maybe you just answered it. I was going to ask, “How do you deal with a program that you’re exceptionally excited about, but maybe the follow-on funder doesn’t get it? Like the VCs aren’t sure what to think about it five years in advance. Which is relevant, because with early autonomous vehicles, it’s not that the Army desperately wanted...they didn’t really know what to do with it. But it sounds like your answer is, maybe, “If it’s a bold enough bet, we can be the follow-on funder. For a while at least.”

Ilan: Yeah, that’s definitely our mindset. And we’ve even tried to build that in. Some ARPA agencies require cost share in their projects: “Oh, you have to show us that someone else has skin in the game by funding 20%.” We basically said, “We’re not doing that, because that actually could as easily could be a counter-indicator for us.” In the future, we think we’ll probably leverage other funders as follow-on partners, and everything else. But let’s make sure there’s nothing that prevents us from doing the bold thing.

That said, it can’t be as easy as, you know, in your scenario, “What’s going to get us to one day have 80% of our funding in one of these spaces?” It can’t just be someone walked in, looked around, and said, “Okay, yeah, let’s just do that.” We should be evidence-based. [Laughter]

We have a pretty good set of principles. A lot of this is, “What are the principles? And then what are the processes that allow us to make sure we’re holding true to those principles?” We have that for creating opportunity spaces. We have that for approving budget for programs. The next thing is to have that for portfolio allocations. Meaning, “We’ve run these opportunity spaces. Where do we think we are seeing the bigger bets emerging versus not?” So that we can say, “This opportunity space is going to get bigger or more focused, and this other one might go into hibernation because we don’t think the elements are there to create the wave.”

Eric: Yeah, no, that’s very interesting. I didn’t know a lot of that, so that was great to hear! So a bit of a question very relevant to...

Ilan: How many questions do you have?! [Laughter] It’s like an endless amount. Are you just making them up and the papers [Eric’s notes] are just like a...

Eric: Well I’ve skipped some [gestures at discarded paper pile], I’ve come up with some new ones [waves papers]. They’re not all off the page! They [Asimov Press] brought me in for a reason.

Ilan: {Laughter] He’s just got like blank pieces of paper just to make it look official.

Eric: I could show blanks! [shows the blank side of his stack of papers]

Do you spend a significant amount of your time thinking about setting up a portfolio to take advantage of AI? Obviously a lot of big breakthroughs in science history come from using the big thing from last year or 5 years ago to apply to some new area?

Ilan: Yeah, it’s a really hard one. I mean, the simple answer is, “Yes.” We are spending a lot of time thinking about how AI — the fact that we’re starting an agency like ARIA at this moment in time, given what’s happening in AI development. One of the easy inspirations for new programs at places like DARPA or ARPA-E is, “What are the technology vectors that are making incredible progress? What are the learning curves you can ride?”

So, I remember at ARPA-E, we had a number of projects we funded based on the fact that fiber optic lasers were getting much cheaper and more powerful year after year. So we had a project we funded on basically using that for geothermal — like drilling hard rock — and other things. AI is like that, just massively.

I think we’re trying to be a little bit deliberate before we figure out exactly what that means for ARIA. For instance, we haven’t jumped in and built, for ARIA, a major compute cluster. We’re starting to think about, “What does an AI program director look like?” You know, how to actually do a lot of the soundboarding that we do. Our program directors are actually using AI. It turns out AI is actually, like, an incredible tool as a program director. Because often you’re looking at things outside of your area of expertise. Oftentimes you’re like, “Oh, if only I had an expert that I could bounce this off of!” Having a thousand experts in all the areas at your fingertips makes a big difference.

I think the much more powerful part is, when we think about these opportunity spaces, one of the questions becomes, “Relative to the impact we think AI could have on this space, how much activity is there to start to deploy that?” And I think that’s going to be one of the important...especially for this new cohort, we’ll end up with some programs that really have AI at the center.

But I think for all of them, there will be this question of, “In this space, where are we in terms of the max benefit that AI is going to build by integrating with this discipline or set of fields, versus now?” And I think that will help dictate how much of an AI flavor we have in the space, and what we’re doing to be differentiated. I actually don’t know if I believe that answer that I just gave, but it sounded good.

Eric: Before we move on, is there any other technology you spend a lot of time thinking about? The world of science and engineering is broad. There might be something else that you think people don’t pay enough attention to.

Ilan: I mean, there are little examples. Like expansion microscopy, that’s such a cool thing! Most people don’t know! It’s popping up in different places.

Eric: For people who don’t know, the TLDR on expansion microscopy is, “What if you made the sample bigger!” And...

Ilan: By literally just putting it in a gel and expanding it. And it works! It’s unbelievable.

I don’t know. I think synbio comes up a lot. I think data, beyond just AI, data comes up a lot as an area where, when you look at some of these disciplines...this is something we’ve found in some of our climate work. You look at the best repositories and the most valuable repositories of data around climate and weather, and realize that none of them are engineered to be accessed in a way that’s compatible with modern data techniques. So there are just infrastructure gaps where you think, “Okay, there’s something cross-cutting around data that we can do.” Or a barrier that we need to get over.

Eric: All right. So I have a suite of metascience questions for you.

Ilan: Okay.

Eric: So, you can go rapid-fire responses, or spend time on them!

Ilan: [Laughter] Okay!

Eric: Do you consider now a great time to work on metascience?

Ilan: Yes. It’s probably the best time to work on metascience that has existed in my life.

Eric: You’ve done this for a few decades, depending on how you want to look at it. Were some periods a lot more frustrating than others?

Ilan: When I left ARPA-E, I wanted to go start...actually, I may have sent you the paper, I had a paper...

Eric: ‘ARPA Lab.’

Ilan: Yes, ARPA Lab. I wanted to create an ARPA Lab. And the point was, like, “ARPA-E is great as a funding agency. The problem is I don’t have resonant institutions to fund. So let’s imagine a lab that is ARPA-minded, that is very entrepreneurial.” And I sort of mapped it out. It was probably like a lab that would have been a compilation of FROs.

And I thought maybe we’d get philanthropy to fund it. And, you know, who was I? So maybe someone else could have done it. But like, crickets, you know, nobody! You know, philanthropy funding science, it sounds like it was a long time ago; it wasn’t that long ago. But the state of affairs, the default for philanthropy and science historically, probably for all time — you can tell me if I’m wrong — has been like, “Oh, I have a family member who got this disease, and I’m going to fund research on this disease.” Or, “I went to this university, so I’m going to give them money to do scientific research.” The idea that you’d have philanthropists making big bets like Convergent Research or Arc [Institute], in terms of new modalities for R&D, that is a new phenomenon and a really exciting one.

Eric: Speaking of something like Arc...

Ilan: This is meant to be rapid fire! [Laughter] You let me go on!

Eric: A lot of people conceptualize the new science orgs as experiments, in and of themselves, in how to do science differently. ARIA is also some version of this. Is it difficult to see what’s going on at a place like Arc, and, if there’s a useful learning, fold it into your operation? Am I not thinking about that right?

Ilan: I think you are thinking about it right. I think the biggest thing — as someone who grew out of venture capital, startups, and Silicon Valley — the thing I’ve realized is, the power of a vibrant ecosystem of startups is you create evolutionary pressure for institutional change. Meaning, nobody talked about OKRs until Google came around. And then everyone needed to do OKRs! And then Stripe came around, and I don’t know whether Stripe does OKRs, but they do things differently. And then all of a sudden it’s like, “Oh, that’s the way to do things.”

It drives you to say, “Oh there are competitors, or peer organizations, that are figuring out better ways to do things institutionally.” And that’s keeping the system super fresh. We don’t have that in research. The institutions are so stagnant for so long that you don’t have the evolutionary pressure to change.

For me, what metascience means is: go run those experiments and find ways to start diversifying and creating a more vibrant institutional ecosystem. Partially so you can get that institutional pressure! You need a critical mass of those things to get that institutional pressure.

So you asked, “What is ARIA learning from Arc?” I love Arc as an experiment, and I think it’s going to change the world. I initially was skeptical that Arc was going to be so close to Stanford as a university. And that they were going to hire people that were still in the academic incentive structure. I think, actually, it’s probably proving me wrong on that. And that’s a great learning. But it’s a very high level, like from a distance, learning.

The beauty would be if we had enough of these things where I could tell you that, “With a 90% certainty, that some one in ARIA’s team, in three years, will come from Arc.” Like if you look at startups in the Valley, right, like, “What are the chances that someone who was at Tesla will end up at another one of these companies?” Absolutely, right? That’s one of the things that drives that learning. It would be awesome if, within the R&D ecosystem, we had that kind of vibrancy and mobility, because I think then you really get the learning.

Eric: And if you personally had the funds, for whatever reason, to fund additional orgs to complement ARIA, like completely different metascience experiments, does anything come up in your head?

Ilan: [Laughter] Yeah, what came to mind, just thinking about your audience, was a conversation with Michael Nielsen when we started ARIA, who was basically giving me a hard time: “What you’re talking about doesn’t feel that differentiated. Where’s the gap? What about a research organization that gives 100-year grants?” The idea of, “We don’t have long term modes.” If you think of time constants as one of the axes of the portfolio, there’s not a lot of diversity on time constants, to like 100 years. Which I loved! I basically said, “Yeah, but if I look at ARIA’s mandate, actually, ARIA’s mandate is not to create new fields.” That’s important. It’s not like [to create] molecular biology. We are built to get something to a new technology platform or industry base. And I don’t think the hundred year timescale is going to work for that.

But I think long duration...basically modes of getting talent focused on either creative research or creation research for long periods, I think there are far too few modes for that, and fewer than there used to be.

Eric: So, that’s obviously ambitious. It’s also expensive. ARIA is pretty expensive, Arc is expensive. Do you think you need a minimum amount of funds to do a good metascience experiment? Do you have low-cost ideas?

Ilan: Well, I don’t know if the long-term thing is expensive. I mean, look, what is expensive? It’s all relative to the potential impact. You have to normalize it, right? So, one of the things I’d love to see someone do is...I think a lot about, “How do we train scientists?” Translational scientists in particular...there aren’t great training environments anymore for translational scientists. One thing I’d love to see someone do is basically say, especially with AI, the training is going to be less and less about the knowledge, and more and more about the taste, the tacit knowledge. and the instincts. So apprenticeship is really important. Why don’t we take kids that are high potential — and when I say kids, I don’t know whether I mean age 15 or 22, who knows? But take high potential people and basically say, “We are going to engineer for you a program where the next 15 years of your life you are constantly being taught how to do research, translational research, through a series of apprenticeships with incredible people.”

And maybe that’s in conjunction with a university, maybe it’s not. To pay for that, you can basically say, “We’ll give you fellowships,” or “You’ll work part of it.” That sounds very expensive, but my instinct is that if you did something like that, you could create the super researchers of the next generation that change the world. So probably not that expensive. [Laughter]

Eric: I’d love to poke on that...

Ilan: Maybe that’s too expensive.

Eric: ...but I’m going rapid-fire.

Ilan: Let me give you one more!

Eric: Yeah, for sure!

Ilan: Very cheap metascience experiment, probably the cheapest one I can think of. I noticed in my PhD, and actually when I hired for my startup, I found myself hiring people...a mode that I would hire was someone, who went into their PhD, they totally butt heads with their advisor, and they walked out. They ended up having to change advisors, or even complete disciplines, and then they thrived. And they come out with this like, “Oh, that was a horrible experience. But it made me change and I’m in a good place.”

Anyway, I keep thinking, I found an amazing PhD advisor that was really resonant. I didn’t waste my time. I felt like I hit the ground running. And I had this incredible experience. So a simple idea: a matchmaking app for PhDs and their advisors, when they come in. If you could increase and improve the compatibility of a PhD student and their advisor, on whatever axes, I think that ends up being a big deal. And it’s basically free.

Eric: Okay, great. That’s fantastic. Thinking of ARIA as a kind of experiment in and of itself, what’s a current bottleneck you all have that you’re very eager to find a way around or work on? There’s always small things in working on an organization.

Ilan: I’ll tell you what comes to mind. It’s kind of a hard one, but I think we’ll probably have to solve for it. We want to move fast in our programs. The view is, “You’re doing something speculative, so time to the next learning cycle is really important and valuable.” And we want to be funding people with diverse skillsets in diverse institutional types. Some institutions...the kinetics of the institution are very slow. And yet, we still want to fund people in those institutions. So one of the questions becomes, “If the kinetics of the institution, just in terms of how fast they can get stuff done, are generally very slow, but you have a Creator in that institution who wants to move fast, how can we help them move faster?”

And I actually think the only answer is we need, like, a fixer. We need an organization whose job it is to de-bottleneck activities within other institutions.

So my imagination is, Eric, you start the Research Speed Fixing Company, and we contract you, and you know, Jenny has found this great performer, but things are moving too slow in this institution. And Jenny gives them a phone number, that’s like a magic phone number. And they call it and you say, “Eric’s Fixing Services! We’ll send someone over right away!” And someone shows up in that institution, and they're running around the admin, just like putting pressure on things and getting people to move faster!

Eric: There are actually things like this in DARPA history.

Ilan: Are there? Amazing!

Eric: I can’t think of one from a performer, but I can think of one from central office. So, for example, when you read through all the oral histories, there’s this guy from early IPTO — which is computing office — Al Blue. And his name just keeps coming up as the guy who makes any bid you want to put out legal, workable, straightforward. I don’t know if he was an engineer or scientist or anything like that. He might have been one of the military guys who finds a way in and has to make his way in the office.

But that kind of stuff makes a big difference! Like in the same breath where they’ll be like, “Oh Licklider was a god,” they’ll also be like, “Al Blue made X thing happen. You need to talk to him.”

Ilan: Yeah! Well, we’re trying to build it into the culture! Right? Like the program directors know...it’s my [Ilan’s] job to have some pressure around, like, learning cycles, it’s their job to do the same. Hopefully we’ll create a Creator community...picking people that are intrinsically motivated, so they want to be pushing the kinetics of what they’re doing. And, yet, it would be really nice if you had more help on that.

Videographer: Time for a battery swap.

[Camera Break]

Eric: Alright, next in the rapid-fire questions…

Ilan: Can we, can we just pause for a bit. I love that we’re doing this long form thing, but people have to realize that this camera right here — there’s a camera with like a bag of ice on top of it because we’re going so long that it’s overheating. Which is great!

Eric: We’re making it work, though. I think the camera looks great, you know? It’s like when you go in the locker room at the end of the game and everybody’s got ice on their knees.

Ilan: Yeah, totally.

Eric: Both teams played hard.

Ilan: We’ll feel like we played hard today. [Laughter]

Eric: Is there anything you personally think people who write about metascience spend too little or too much time on? Like is there some hypothetical Substack where, as the CEO of ARIA, you would hoover up if somebody wanted to spend the time on it? I already have homework where I have to write a Rad Lab piece.

Ilan: That’s right! You do have to write a Rad Lab piece. Multiple! There’s like a series there. Oh, you also need to write a piece about how, actually, DARPA ended up emerging in part because of the UK’s radar effort. Which is important for me because it brings things full-circle to ARIA.

Metascience...honestly, what I would love is a blog that just chronicles metascience experiments and their learnings, and just keeps tabs on them. And that that grows and grows.

Eric: And what are variables that you think should be...some of would be qualitative, you meet them where they are...but would there be any underlying variables you’d want if somebody was writing about Convergent or Arc or ARIA, to constantly revisit?

Ilan: Incentives.

Eric: Incentives?

Ilan: Incentives. I think that is the only theme that matters. [Laughter]

Eric: Yeah, I think that makes sense. Do you have any call to action for any groups involved in metascience, or who would want to be involved in metascience, that you think would be useful to put out there? That can be researchers, engineers, ops people, policy people, whatever it is I am.

Ilan: I’m going to say something controversial, which has been on my mind a lot, which is...I think metascience is, at the same time, one of the most important movements, in terms of driving more progress out of research, and also in some ways one of its most dangerous movements.

The reason I say that is, when I was at ARPA-E, I was really interested in the question of, “How does ARPA-E understand its impact, map its impact, and measure it?” And I tried to dig into how we could do that. And both from my explorations of the community of people thinking about that and from the many activities we had to go through at ARPA-E to try and show and prove our impact, how things work...the amount that was “useful” was probably 10%, and the amount that was, “Actually, you’re trying to map something that doesn’t make any sense onto what we’re doing. You don’t really understand the context of how we work, and you’re trying to measure it. And you have a number of theories, but even when I tell you they don’t map on here, you don’t believe me,” actually led to a lot of inefficiency.

So I think what I would say is, if you’re in metascience, recognize that first of all, if you can do something experimental which pushes that evolutionary pressure on the system, great! If you can show there are better ways to do things by doing them and then having results, amazing! If you’re working more on the theoretical side or the evaluation side, just recognize you have a big responsibility, which is to make sure that when you add up all the hours of people who are engaged with that — from agencies doing the work, startups doing the work, or whatever else — that you have high conviction it’s going to be net benefit as opposed to net cost. Is that...?

Eric: No, that’s perfect!

Ilan: I feel like I’ve said that to people, and I think I’ve offended them in some ways, but I think it’s a really important thing to be thinking through.

Eric: No, that's perfect. We’ll end the official questions there. That’s Ilan Gur! Thank you so much for doing this and how long you were willing to spend with me today.

In the episode, I discuss:

• FreakTakes as an applied research shop, and how I choose what to work on

• The role Bell Labs-style systems engineers can play at new science orgs

• How I might give away $100M, inspired by Warren Weaver and the Rockefeller Foundation

• BBNs at early ARPA, the upside of building more BBNs today, and why I joined RenPhil to work on this challenge

• The effect of events like Watergate and Vietnam on scientific bureaucracies

• What Gerald Holton and certain scientists’ oral histories have to say on the ‘burden of knowledge’ as a human systems problem

• Tons of R&D history anecdotes, including:

  • The complicated takeaways of Claude Shannon’s time at Bell Labs

  • Wilbur Wright’s taste

  • Irving Langmuir at GE Research

  • Praise for early MIT and the role of market signals in shaping their research agenda

As always, just email me (gilliam@renphil.org) to discuss any ideas discussed in the episode! I’m particularly eager to spend time with those who are curious about founding for funding BBNs.

Spotify

YouTube

Subscribe now

Share

Thanks for listening:)

For those who have not read or do not remember prior pieces on BBN-model orgs (BBNs), the biggest takeaway from my series of ARPA project histories is that, for many of ARPA’s all-time great projects, exceptional projects were the result of exceptional contractors. By “exceptional,” I mean contractor groups who were not just staffed with elite talent, but uniquely aligned with ARPA’s mission. Early ARPA success stories like stealth aircraft, early autonomous vehicles, and the ARPAnet had contractors that shared three distinct traits, rarely found together. Each:

1. Was novelty-seeking, with a strong preference for projects that pushed the technological frontier forward substantially.

2. Built useful technology for actual users. This entailed professional contract management and a willingness to focus on difficult systems engineering tasks.

3. Used more flexible team structures than academia. When compared to academia, they more effectively hired, organized, and incentivized researchers, engineers, and other experts to collaborate on applied projects in a common-sense fashion.

While recent pieces have explored how great BBNs from history mixed contracts, grants, and hybrid team structures to pursue an ambitious technical vision, today’s piece will kick off a series of pieces profiling BBNs and BBN-shaped opportunities that exist today.

The subject of today’s piece is the Topos Institute, a BBN-shaped organization staffed with PhDs from top math and computer science departments. The CEO, Brendan Fong, has a computer science PhD from Oxford, specializing in category theory and its applications. The unifying technical ambition that motivates the Topos team is the goal of building technologies to enable scientists and engineers from adjacent fields to more effectively combine and reason with overlapping models from distinct domains. Topos pursues this technical vision with a mix of carefully selected R&D contracts and traditional research grants. In today’s piece, I interview Brendan to learn more about:

• Topos’ technical vision.

• How Topos balances contracts and grants to fuel their work.

• Topos’ comparative advantages when compared to university departments, startups, etc.

• How Topos is/is not like an FRO.

• What differentiates Topos from other AI for science groups.

Enjoy!

Subscribe now

Share

For those interested, Topos is looking to hire for a crucial position in its Oxford, UK office — discussed later in the piece. It’s an ideal position for someone with a nerdy fascination with the operational history of R&D and excited to contribute to managing a real applied metascience experiment. (I have also been meeting other BBN founders looking to fill similar roles. If that’s exciting to you, please see the end of the piece to learn more.)

In putting together this piece, Brendan and I had over three hours of messy, wide-ranging conversations and interviews. We’ve distilled them into this final, more readable, interview format. From here onward, the italic + bold text denotes my words and the standard text Brendan’s.

Topos, In Short

Brendan, what is Topos from a very high level?

Topos is an experiment in a new structure for doing research for the public benefit.  In pursuit of this goal, we look to combine a focus on fundamental, long-term research similar to a university with a theory of change similar to a tech startup.

The technical area we work in is mathematics and computer science — particularly in collective modeling, collective inquiry, and cooperating in situations that are incredibly complex and multidisciplinary. We do this as a nonprofit whose work is supported by grants and contracts from groups like DARPA, ARIA, philanthropic foundations, technical firms, and gifts from high-net-worth individuals.

Topos’ North Star Technical Ambition

Can you talk more about collective modeling etc. and how that drives Topos’ grand technical vision?

Ultimately, the sort of technology we want to develop and produce is, in some sense, a new form of programming language. As we’ll talk about later in the interview, Topos works on projects in areas like climate modeling, the systems engineering of large airplanes, ensuring AI systems are resistant to catastrophic risks, etc. The communities contributing to each of these technical systems are made up of a variety of technical experts navigating different aspects of the system with their own overlapping models — which often have their own merits and their own limitations.

Topos works on projects with specific user communities. That keeps us grounded in the practicalities, ensuring we don’t get too caught up solving problems we find beautiful but are only modestly connected to practical needs. The grand goal the organization has when working on these specific problems with users is a General Theory of the Specific. We want to build a tool that, like the spreadsheet, helps a variety of users reason across situations with the richness and complexity that comes with combining and interpreting a mix of overlapping models of the world. The term I often use to describe the goal of this general programming language or spreadsheet to combine ideas from overlapping domains is “collective sense-making.”

One difference between this general modeling environment we hope to build and the spreadsheet is that we want to empower our users to represent the world in the conceptual language they use in their work, not force them to translate those representations into numbers and formal logic. A key difference between our modeling environment and even high-level programming languages is that we are more focused on declarative representations of the world, not procedural actions in the world.

As transformative as spreadsheets have been, I think we could make a similarly significant advance if this plays out right.

Do you intend for this to be a tool for mixing models in the worlds of science and engineering, in particular?

Scientists and engineers are the target user groups in the vast majority of our applied contracts. While we think that one day these tools can be helpful to someone running a coffee shop, planning a wedding, or simply just communicating carefully with friends, we see the ‘edge of the wedge’ as working in areas where people already have a rich tradition of articulating the conceptual foundation of their work. And that tends to be in the sciences.

Where Topos Fits in the R&D Ecosystem

Would you like to say anything about the distinct advantages Topos has in tackling certain technical problems — when compared to universities, startups, etc.?

We’ve chatted for a while outside this interview, and I find you very perceptive about these topics. Can you give me a quick rundown of what you think the strongest case is? Like what three things would you name? And I can see if I disagree or would like to add.

Yeah. When we had a discussion surrounding the question, “Do you consider Topos to be a ‘center of excellence outside the university?’” you pushed back. That wasn’t due to a lack of talent. You believe a huge proportion of your staff could have gone on to have successful academic careers. You pushed back because you felt the term ‘center of excellence’ was often associated with centers of excellence for ideas, not technology. You want to be a grounded org. University departments and places committed to ideas, first and foremost, tend to not respond to user needs very dynamically. So, in terms of differentiating yourself from an academic department, I’d say being staffed with similarly talented people dedicated to building technology for actual users with pressing needs is a huge differentiator.

In terms of what differentiates you from startups…VC-funded startups are often limited to areas with a market cap of ~$1 billion+. That’s a big constraint. And in most cases, they don’t really embrace hard technical risks. They often prefer engineering risks to scientific ones, etc. They embrace risks that they hope can be mostly mitigated within a couple of years…a lot of the time, at least. You recruit as ambitiously as a university department or a top startup. But you go where the hard problems are as long as you can find funding from a DARPA, ARIA, or some open-minded grant funder in Berkeley. So, to be clear, you also have an arbitrary constraint. But the arbitrary constraint is “What’s DARPA funding?” or “What’s ARIA funding?” But if you can find that funding, you don’t have to clip the wings of your ambition. It allows you to tackle all sorts of areas VC-funded firms can’t. You can let me know if you agree with all that.

Yeah, I think you pretty much got it. For groundbreaking, public good technologies, we need tight integration between research — the source of deep new ideas — and product — the source of deep new questions. Our core advantage over a university is that we can be user- and impact-driven. Our core advantage over a start-up is that we can tackle deep technical risk. Let me add a bit of color to both points though.

First, academia. One thing that cannot be done in a university setting is, I think, build and maintain technologies for the long term. And that is a core part of our theory of change and how we assess our success in basic research. We want to do basic inquiry around foundational questions — and we win enough grants that allow us to do that — but we don’t want to be measured by the publications we produce: the publications are instrumental. This contrasts with the known incentives against maintaining software and a user base in academia.

Relatedly, another way we differ from a university is being intensely collaborative and team-driven. To me, the incentive structures of academia are entirely individual. Postdocs, tenure, awards, all go to individuals for individual success. One of the reasons for starting Topos was that it was the only way we could find ourselves collaborating day-in-day-out with the top people in our field: university departments often want breadth, rather than aligned faculty well-suited to teaming up. With that system, it can be difficult to find and maintain generative collaborations that lead to core breakthroughs.

What separates us from start-ups? We’re driven by producing public goods — as long as we can find the funding. We don’t orient ourselves around what’s best for investors.

Since I know you want to move on, the last thing I’d like to add is that Topos is a lot of fun! We’re a collaborative, creative, and compassionate place with a great sense of collective purpose. I’m inspired by our team every day.

Specific Technical Projects

Topos’ technical agenda is driven, in no small part, by pursuing a General Theory of the Specific. Could you talk about some specific projects you’ve 1) already worked on or 2) would love to pursue?

One very illustrative example is a systems engineering example. Imagine you’re Boeing or Airbus and you’re constructing a passenger jet. There are thousands of people involved in the process of this construction. How do you ensure it all comes together into a vehicle that people can trust with their travel and lives for decades to come?

Some of that is scientific or engineering modeling. “What is the weight of the engine?” “What are the properties of the materials going into the airplane wing?” “How do the material properties relate to the aerodynamic properties?” Etc. But in bringing the parts together, very significant social processes are also going on. Initially, this group is responsible for this piece of the puzzle, specified in this way; that group is responsible for another piece specified in that way. There are also a lot of initial guesses that get refined over time, through new insights and changes in responsibility. This is the evolution of a process, a highly dynamic process of knowledge discovery.  It’s really important to keep track of that history and understand how different decisions have been made in response to different sorts of problems, insights, and constraints as the work of various groups converges into an interoperable design.

Passenger jets like the Boeing 737 have been flying for 50 years or more…

And, at this point, these firms often need to keep long-retired engineers on retainer, sometimes until they die, in case they’re needed to answer a question. This need often derives from this issue of organizational memory — not remembering why you did what all those years ago, how that learning should apply to the present situation, etc.

Right. Exactly. That sort of knowledge, that sort of understanding of the product, is very difficult to institutionalize with existing technologies. But if you do study the processes of collective sense-making carefully, there is reason to suspect — and it’s Topos’ ambition to build — knowledge management software, structures, techniques, and cultures so you don’t have to keep someone on retainer…unless they want to be on retainer and you want them to be on retainer.

The folks at ARIA connected us. I’d love to hear about your current AI Safety project with them.

What we’re doing with ARIA is a project for Davidad’s Safeguarded AI program. There is this idea that AI systems will increasingly drive actions in the world. This will sometimes extend to critical or complex situations. For example, it would be great if we could use AI systems to optimize the power grid of the UK — payload distribution, inform the construction of new power generation sites, and so on. But there are huge possibilities for catastrophic risks to a country. The power grid is literally the backbone of getting work, as in physical force, done in society. Power grid failures, whether accidental, due to natural disasters, or from cyber and physical attacks, could cause widespread economic and physical harm, and even deaths.

How do you negotiate over the constraints of what you want — in an accessible, high-level way — so your team can use AI tools to generate solutions that you know to be safe and respect those constraints? One thing you need in these situations, where you have all these social and physical factors, is an appropriately precise model of the world. This model should be transparent, interpretable, and verifiable in certain ways. By providing a way for people — in a collaborative and interdisciplinary way — to construct subtle, formal models of the world, you can enable them to run these AI systems inside sandboxes. In these sandboxes, you might have tools helping you to combine techniques that allow you to validate your model and constraints — e.g. automated reasoning, formal verification, theorem proving, etc. Doing this, a team can generate very strong pieces of evidence (almost guarantees) that the system will be safe enough for public deployment.

Do you have any examples of ARIA projects you’d jump at the opportunity to work on? This can either be from their existing program areas or areas you suspect they’ll pursue in the future. ARIA’s program areas are all scientifically-motivated and seem to have an AI for science component. And, as we discussed, there are difficulties that come with pursuing this sort of work entirely within academia. There’s a lot of room for exciting Topos collaborations.

What would we love to do with ARIA? Another program that has conceptual similarities to the Safeguarded AI program is Sarah and Gemma’s program on Forecasting Climate Tipping Points. Their program deals with an extremely complex system — understood only through integrating perspectives from multiple disciplines, including both the physical and social sciences — with deep impacts on society. Their goal is to produce an early warning system for climate catastrophes that is precise, trustworthy, and actionable.

As the program thesis outlines, “our best climate models are computationally expensive and do not capture all the physical processes we need.” Moreover, we want to avoid tipping points, so we don’t and hopefully won’t have too much data on them! And the complexity of the system requires the integration of information at a wide range of spatial and temporal scales. We’d love to undertake a project on mathematical and software development of climate modeling tools that respond to these challenges. Similarly to our ARIA project with Davidad, we could work with ARIA creators (contractors) to figure out things like, “What are the bounds of the system?” and “Under what circumstances will it go into a catastrophic region of behavior?” We’d try to develop tools that enabled domain scientists to construct a shared, interdisciplinary understanding of the climate system that is responsive to new information from sensing systems and research.1

Of course, what that hypothetical project would look like specifically depends on many things — the needs of the program and other creators, etc. I’d be interested in exploring what you’ve mentioned, which is the idea of Topos being a sort of consulting AI for science shop that comes in and assists with tooling to solve problems in and between particular domain areas. Academia is not set up to support cross-disciplinary tooling and workflows, and we’d love to complement the academic system in this way. But…I’d like to be very clear and say that domains are very important. When talking about providing AI tools for other scientific disciplines, I want to do it from a place of humility…

If I can interrupt, do you mind talking about why climate science is such an interesting area for Topos first? And then we can re-visit your point on bringing a level of humility to AI for science work.

Yes, of course. Climate science is a great example of where our work can add value. We build computational tools for cooperating across the different conceptual frameworks and disciplines required to navigate complex systems. One basic, almost too-obvious-to-state idea underlying our work is that a good way of making concrete progress on difficult topics is writing things down. You want to write down models; you want these written models to precisely reflect your understanding of the world.

To adequately describe complex systems, we must use models from many different disciplines simultaneously. The problem, however, is that disciplines often create silos and it becomes difficult to collaborate across these different perspectives. So, we need a way of allowing conceptual-specific language that also enables interoperability between the various languages. A major focus of our tools is this process of translation, mapping between perspectives, identifying overlap, and identifying difference.

In tackling this work, you really reveal the complexity of the situation. It’s a challenge. Things become very messy, very fast. For some arbitrary climate science problem, an oceanographer might be trying to construct maps of the Pacific Ocean, how it works, the concepts involved, etc. The oceanographer will have a particular idea about what water is and what the important features of it are. This is very useful, but when building a climate model you’ll also want to have insights from an atmospheric physicist (etc.). Depending on the situation, an atmospheric physicist will have their own model with different primitives. Factors like solar radiation or windspeed might be more or less important, or interpreted in a different way, than in the oceanographer’s model.

There’s an overlap between these concepts: both fields are talking about water. But an atmospheric physicist is talking much less about ice and much more about water vapor, clouds, etc. But they’re still talking about water. How do you translate between what water means to an oceanographer and what water means to an atmospheric physicist?

One approach we use to tackle problems like this — drawing on our mathematics and computer science backgrounds — is, as you collect data, setting up database ontologies and more sophisticated logical tools to help specify what water is from various points of view. From one point of view, water is some combination of precipitation, cloud cover, snow, etc. But the relational aspects are also very important in forming a more full worldview. From this POV, water doesn’t mean anything in and of itself; it’s something like a database column heading. What reveals the quality of water could be how it relates to what the plankton need — or other parts of the ecosystem.

Having to translate what water means in one field vs. another can be incredibly complex. A single concept here can turn into a web of relationships over there. When dealing with problems like how water is witnessed through its relationships to animal life in the oceans, you might even have to translate between webs of relationships. In trying to create interoperability between frameworks, there are a whole lot of interdependencies that are incredibly difficult to track. You want sophisticated, formal computational tools to help you track that. That’s what Topos tries to build.

With our approach, the scope for ambition in climate science is immense. Consider the current state-of-the-art in the field. There are these gigantic climatic models that groups like the IPCC coordinate. With these models you’re talking about millions of lines of code, often in low-level languages like FORTRAN, written over the past half-century by thousands of scientists across many disciplines. There are good reasons for using FORTRAN, but there’s a recognized need for better tools too. They’re trying to weave all these models together, but they’re presented in ways that are so procedural — not conceptual — that you cannot get at the emergent behavior as these disciplines interact. To tackle this particular application would require succeeding on many smaller, shorter-term problems first — like the systems engineering problem above. But it’s the exact kind of problem we hope to build towards.

Humility, Sourcing Contracts, and the Prospect of Being an FRO

Now, if we can, I’d love to get back to the idea of pursuing this interdisciplinary work with humility. It’s very important to us. Our work relies on deep, respectful relationships with domain experts, our collaborators. To establish such relationships, it’s critical we acknowledge that it’s very rare for an outside group to come in and do things that are groundbreaking in the minds of people who have been working in that area…

If I can interject, it sounds to me like the Topos approach comes in with more humility than is typical from many AI for science groups. As I understand them, your proposals for applied projects are akin to what Blackboard would have been in the early CMU Robotics Institute’s autonomous vehicle work. You’re not coming in and saying, “Let’s build up more vision data and we’re going to run it through an existing set of models. Scale is going to solve it.” Your pitch, to me, sounds something like: “This is a really complex system with a lot of disciplines that have something useful to say about it. We’d like to come in and build you tools that can be something like a synthesized reasoning engine.” Your proposal attempts to take in all the existing models and output something very interpretable, while very overtly attempting to preserve what specialists would consider useful about their fields’ models. You’re not looking to supplant them with any kind of neural net or random forest.

Yes, exactly.

Now, there probably should be some contractor that comes in and tries to get people to build up more data, train neural nets, and things like that. But it sounds like Topos has a pretty differentiated offering — one that has a lot of humility.

Right. We’re not the kind of AI for science shop that seeks to combine clever algorithms, a wealth of data, and compute to yield breakthrough insights. That certainly has its place, and we’ve seen some stunning successes from this approach, including the Nobel Prize-winning work on AlphaFold. But that’s not our comparative advantage. What we want to offer is a much more bespoke product. We hope that through our applied projects we can assist users in eliciting the structures of and clarify the thinking in their domain. Over time, we hope to build up a software stack that helps reify some of that knowledge that has been clarified. If all goes well, our tools will enable them to more easily build up or utilize that knowledge.

If you could raise the funds, would becoming an FRO be an ideal scenario? That’s not easy. But let’s say someone offered you $50 million over 7 years to pursue a big FRO-style project, what would you do with it?

That’s a very interesting question. I guess the answer to your question is: we’d love a modest amount of FRO-style funding — say $10 million — but even with this funding we’d continue to seek out contracts too. It’s a bit too early for us to need something like $50 million over seven years. With the more modest $10 million, we could do a lot of research and engineering groundwork to develop fundamental aspects of our modeling environment. (See CatColab if curious.)

But for us to do good quality work, we need to remain grounded. General research funds are great, but another key resource for us is pressing problems from domain partners invested in the challenges of collaborating, articulating questions carefully, and working together toward solutions. Being resource-constrained, as we are, incentivizes us to maintain close engagements with contract partners with concrete needs.

We’re still figuring out the resource balance for Topos. I’ve found your work on BBNs as an alternative to FROs a very helpful framework for thinking this through. For example, it’s helped me find some questions that explore whether or not we’re on the right path. On one end, I ask questions like, “Are we becoming unmoored and doing too much research for research/philosophy’s sake?” or “Are we responding to actual problems in the world?” On the other end, I ask questions like, “Do we have the freedom to do really ambitious thinking and reset paradigms where we need to reset them?” or “Are we so constrained by the needs of particular contracts that we cannot, where we feel we have an edge on a new foundational approach, pursue that edge as necessary?”

We speak of Topos as vertically integrating foundational research, technology development, and public service. Navigating these questions, and thinking about the balance between FROs, BBNs, and other organizational models, is part of the process of making this idea reality.

Can you say a bit about how you obtain applied contracts?

Our most productive applied projects are driven by shared visions and strong relationships with program managers, whether at DARPA, ARIA, or in private industry. They’ll have a particular concrete problem space they’re exploring. Over the years, through our publications and through conversations, they develop an understanding of what we do and how we can help their programs. That often turns into them saying something like, “Okay, we want a proof of concept. My program has a need in [simulated multiphysics, systems biology, epidemiology, or something like that]. Can you propose a way to demonstrate your ideas in this particular domain?” We then go do a bunch of research on the problem area, often collaborating with domain experts, and construct a bunch of software around this problem — built on top of some basic core libraries we build and maintain.

How about undirected funds?

Undirected funds and, just as important, a strong network of visionary supporters, have been critical in getting Topos off the ground. We are an organizational experiment. Experimenting, exploring, and learning what does and doesn’t work for us: this takes real resources — it’s been around a third of our total funding so far.

Our founding board — particularly our founding chair, Ilyas Khan — provided not just resources but also mentorship, courage, and connections. I’ll be forever grateful to early major donors like Jaan Tallinn and Jed McCaleb for their trust, even despite our naivety and inexperience. Berkeley and Oxford have both been great communities for finding collaboration with open-minded funders who both appreciate our technical ambition and that breakthroughs in some of our approaches can have exciting spillovers for the wider science and technology ecosystem.

A Possible UK Expansion

You’re looking to make a big hire. As I understand it, you’re looking for someone to be your key operational person as you expand your Oxford, UK team. Finding the right person might even make the difference in whether or not you expand. It sounds like they would be both an operational hire and a strategy hire, interfacing between ARIA and your technical team as well as helping the team come up with new project areas. Can you talk more about what this person would do and why it’s such an exciting position?

I’ll preface this by emphasizing that Topos is an ongoing experiment and we’re still seeing it through. So it’s hard to speak on precisely what this person would do. But maybe that’s the point: the right person will find space for innovation exciting!

What is my best first guess? I think we need someone who sees the importance of Topos’ vision — humanity cooperating around issues of complexity. They should believe this technology can be produced by our team, which is (in my opinion) one of the most impressive technology teams around at the moment. And this person should want it to be produced as a public good.

This person is meant to help us continually find answers to practical questions like, “How do we organize this team of technical talent, fund them, and motivate them to ensure we have a sustainable approach to building technologies that move humanity forward?” A mix of philanthropic grants, contracts, and maybe even investments (for certain products) might be involved. Beyond helping us make these decisions, they’re meant to take on a lot of the day-to-day responsibilities necessary to ensure our collaborations run smoothly. This first guess will be refined in concert with the applicant, the wider team, and ultimately practical experience.

As far as the type of person we’re looking for? They don’t need to be good at mathematics. It might even be good if they weren’t. But they should appreciate mathematical ideas, be open-minded and willing to learn how researchers think, and be able to get along well with our technical staff and staff at partner organizations like ARIA.

Many backgrounds could be well-suited to the role. These include:

• An operator with the ability to pitch and manage — who is also driven by public service. This person should also be excited by the experimental nature of our organizational model and the prospect of shaping it.

• A mathematician/computer scientist who gets people and business…and is willing to not do the mathematics. While this person wouldn’t be doing research, having an appreciation for mathematics might give them a deep understanding of what this org is about.

• A domain scientist who, beyond being good with people and business, sees the particular problems of their field as a lens onto a general problem. We’ve had some collaborations with domain-specific scientists flounder a bit because they wanted to start over-tuning for that particular field. That is not a terrible instinct when working on a specific applied contract, but a bad one for someone helping steer our UK operation. We are looking for a General Theory of the Specific, after all.

• A software engineer whose first instinct is not to respond to things by building systems. Software engineers often respond to things by building rules-based systems; that is the nature of the craft. We’ve had only ten contracts throughout the history of the organization. The wrong software engineer might be tempted to immediately begin building databases, search tools, etc. around those learnings. Those approaches can be powerful, but I think what is needed is a bit of patience. This person needs to embrace that this is an ongoing organizational experiment in how to do science better; we’re still learning. We only have an inkling of what’s best. Formalizing too soon is not respecting the problem.

Lastly, I’d like this person to have a really creative spirit around how to build a team that does ambitious basic research that can also build. As we’ve discussed outside this interview, our early DARPA contracts taught us a lot of lessons we hope to incorporate moving forward. I’d estimate that at least half of the funds from those contracts directly funded work we’d have pursued anyway with an undirected NSF grant. This new hire should be excited about helping us figure out a revenue strategy that gets us as close to 100% as possible, and can find exciting partners/contracts to make it happen.

Conclusion

Any final words?

We’re something like a BBN-style org, right now. We have a clear technical vision that we pursue with a mix of contracts with carefully selected partners and grants. These bespoke projects are key to what we do and create deep insights into the nature of what the general problem actually is. Balancing vision and contracts enables us to build technologies that enable cooperation across different perspectives. Parts of our fundraising and governance approaches may change over time — there are many upsides to the FRO governance model, raising a basic research endowment, consulting using internal tools, etc. But our general goals of enabling cooperation in areas of complexity through a balance of basic research and applied projects will not.

Finally, thank you, Eric, so much for this conversation, and for the many leading up to it. It’s been a real privilege and, as always, you’ve helped push my understanding and articulation of Topos deeper.

It’s been a pleasure. I learned a ton through our conversations. I really hope one of my readers (or someone they know!) finds this piece and reaches out. It’s the kind of job I’d have dreamed of before FreakTakes, when I spent my nights reading about scientific history with no clue how to get hired and make a difference at any of the new, exciting organizational experiments I’d heard were being founded in places like Berkeley.

Thanks for reading:) If you’re interested in Brendan’s position, please reach out to him (first name@topos.institute) or me via email (gilliam@renphil.org) or Twitter. And if you’d love to fill a similar role at a BBN in a different technical area, please reach out! I might be able to connect you with a researcher in need of a systems engineer/operator. Some technical areas in which I already know a BBN founder open to hiring (at least part-time) for similar work include:

• Non-Model Organisms

• Economic Complexity

• Ocean Water Sensors

• Bespoke EE and Biotech Hardware

• Computational Chemistry

Subscribe now

Share

Before starting FreakTakes, I did not know whether people like Tom Kalil, Adam Marblestone, or Patrick Collison would find anything I might write useful in their work. But I figured it was worth spending a few hundred hours of my time finding out. I already spent most of my spare time reading about 20th C. R&D history and the economics of innovation. So, in January 2022, I started FreakTakes. I’d write five pieces and see how things went. On a trial basis, I’d turn my obsession into a blog.

That was the best decision of my professional life. Over the past three years, the influence of the blog has steadily grown — at least with the small number of key individuals who I see as my target audience. The research quality has steadily improved as I’ve accrued experience. My “taste” regarding which questions are most useful to look into has drastically improved as I’ve gotten to meet with those who run new science orgs and fund ambitious R&D. The experiment has worked out far better than I hoped. The success of the experiment is, in no small part, a result of Stuart Buck making two separate bets on me — the first in April 2022 to bring me on as a part-time Fellow at the Good Science Project, and later that year bringing me on full-time. For that, I will forever be indebted to him.

This year marks a new chapter at FreakTakes. I’ll be joining Renaissance Philanthropy (RenPhil) full-time — while remaining a Fellow at the Good Science Project. Working with Tom Kalil, Kumar Garg, and the rest of the RenPhil team, I hope to translate the lessons of R&D history I’ve written about for three years into action.

My Role at RenPhil

At RenPhil, I’ll be ramping up the amount of time I spend as a do-er. While I’ll be spending time on all sorts of projects at RenPhil, the biggest one will be helping build and grow more BBNs — organizations like 1960s BBN or the early CMU autonomous vehicles group. R&D orgs like these — novelty-seeking, well-equipped to build useful technology for actual users, with more flexible team structures than academic labs — were the driving force behind many of early ARPA’s most exceptional projects. As the number of ARPA-inspired grant funders grows, the need for exceptional contractors like these will also grow.

Building a new BBN requires founders who are world-class researchers or engineers, have an ambitious technical vision, and have a strong sense of why the existing R&D ecosystem cannot pursue the vision. If that sounds like you or someone you know, please reach out. I’d like to help hone your idea and try to make it a reality. Building a new BBN also requires R&D funders with ambitious projects. So, I’m also eager to meet with R&D funders (philanthropies, ARPAs, etc.) to discuss projects they are eager to pursue, but for which they feel ideal contractors do not exist.

Operationally, I’ll also be helping RenPhil execute its ARIA work in the UK — and a variety of other activities. In terms of research, I will continue to produce my long operational histories of R&D. But some things will change.

What is changing?

As I step into this new role, I hope to expand the scope of the Substack a bit. FreakTakes will continue to primarily serve as the home for my operational histories of R&D orgs, but I hope to make FreakTakes into a Substack ambitious researchers and ARPA-like funders eagerly follow for information, talent, and opportunities related to their work. For pieces in this category, I’ll be creating a “RenPhil” tab to file the pieces under.

Some pieces that would fall into this category include:

• Interviews with the founders of existing BBNs — describing the vision for their organization, their founding story, and upcoming projects.

• Short pieces from ARPA-like philanthropists who would love to fund a certain kind of project, but for which the ideal contractor does not exist.

• Proposals for BBNs in particular areas that, with the right founder and team, could tackle ambitious areas of R&D and have a generational impact on a field.

As I join RenPhil, there is one notable tradeoff for FreakTakes. As I spend more time on operational work, I’ll only be able to produce about one-third as many long (7,000+ words) operational histories. But there are two upsides regarding classic FreakTakes content:

1. For the operational histories that I do produce, I’ll be ideally placed to translate the learnings into action.

2. I will also write more FreakTakes Shorts — often around 2,500 words — reflecting on events from the history of R&D, sharing interesting excerpts from researchers’ memoirs, etc.

In general, FreakTakes will remain mostly the same. I love it too much to change it too much. The new pieces I produce in the RenPhil section will be pieces that allow the blog to translate ideas I encounter into real-world projects.

The Surprising Effectiveness of Blogging

Before moving on to my pithy conclusion, I’d like to take a moment to express my appreciation for this community. The open-mindedness and curiosity of the researchers and funders who make up the “new science” community made it possible for a (more or less) random guy in Chicago — who reads about the history of R&D instead of going to parties — to throw his hat in the ring and make something useful of his obsession. Given I have no credentials, none of you had to give my pieces a chance. The fact that so many of you did is a testament to your open-mindedness.

When I first started this blog, I dreamed that maybe someone at a place like the Arc Institute might find it interesting and help me get a job there — not even as a full-time researcher, just as a budget guy who got to do research assignments as they came up. Hoping for more than that felt unreasonable. But sometimes, if you work at it, unreasonable things happen. I was more grateful than words could express when Stuart Buck gave me the opportunity to be a researcher at his think tank. And I’m similarly grateful to have the opportunity to turn so many FreakTakes’ ideas into action at RenPhil. This blog has gone weirdly well, so far.

My Pithy Conclusion: Embracing Restlessness

For a very excitable person, I’ve settled into the life of a history researcher quite comfortably. The decision to reduce the amount of time I spend peacefully finding answers in old books is bittersweet, but not difficult. The reason the decision is easy, despite the pleasure I get from doing research, is probably due to the Midwesterner in me. Even with my passions, I always try to make them practical. I never hoped to become a writer. I began FreakTakes because I had practical goals; I choose what details to include in my pieces based on what would be useful to the CEO of an R&D org, not because I think most people want a 14,000-word piece on Warren Weaver’s work at the Rockefeller Foundation.

I hope I’ll have the excuse to become a full-time researcher again, at some point. But for now, it’s time to translate many of FreakTakes’ learnings into action at RenPhil. An excerpt from a 1954 Warren Weaver essay provides me with a fair bit of comfort in this decision. The excerpt comes in a chapter in which Weaver — the greatest scientific grant funder of all time, a fellow Midwesterner, and a good man — rails against those in the 1950s who sought peace of mind, first and foremost, in their spiritual endeavors. Weaver writes:

Peace of mind is not what I ask from religion. To pray for peace of mind appeals to me as a rather unpleasant insult to the God of the restless cell, of the blazing novae, of the swirling nebulae. The god, in short, of progress, not of stagnation.

...God pity me on the day when I have lost my restlessness! God forgive me on the day when I am satisfied! God rouse me up if ever I am so dull, insensitive, lazy, complacent, phlegmatic, and apathetic as to be at peace!12

I look forward to embracing this new chapter of professional restlessness. Please reach out to me if you’d like to found, fund, work at, or somehow help R&D orgs like the ones I’ve described in this piece. And if you have any ideas for me as I take this next step, I’m all ears!

Thanks for reading:)

I’ll be visiting both Cambridge, MA and London (and maybe SF) soon, so please reach out to grab some time while I’m visiting!

Please comment, email or DM me on Twitter me any ideas you think I should consider for the next chapter of FreakTakes (egillia3@alumni.stanford.edu or gilliam@renphil.org).

Subscribe now

Share

Subscribe now

Share

DARPA doesn’t exist in a vacuum. It is enmeshed in both the military ecosystem and the D.C. ecosystem. DARPA also has no labs; it is dependent on the community of performers (contractors) available to it — academic labs, companies, and others. Major changes in the politics, economics, incentives, or preferences of any of the above can substantially change what a DARPA office is at any given time.

It is a testament to DARPA’s resilience that it has been able to withstand so many changes throughout the decades, continuing to serve its mission in a less bureaucratic and more entrepreneurial fashion than the vast majority of the government R&D ecosystem. However, it is still essential that those seeking to understand historic DARPA successes understand the “regime” — a DARPA office’s political environment, performer ecosystem, etc. at a given moment — in which particular projects existed.

Through my research on DARPA history thus far, I have come across many small changes that have impacted what DARPA is at a given moment. But three factors have jumped off the page and proven salient enough that I will not sit down to write up a DARPA project from history without understanding them. They are:

1. The level of organizational oversight in a given period

2. The extent to which project “visions” came from office directors vs. PMs

3. The timeline in which projects were expected to pay off and how exactly a given office interpreted DARPA’s mandate that projects serve the military in some way

Today’s piece will not cover any specific project or contractor from DARPA history. Instead, I will detail how changes in these three factors alter how DARPA operates. I will provide examples from oral histories of long-time DARPA employees and performers as well as DARPA historians describing the specific effects of changes in these factors. Many of the specific examples will draw on DARPA computing history and the Information Processing Technology Office (IPTO) —responsible for many or ARPA’s now-famous early computing projects.

I’ll begin by exploring what might be the most substantial shift DARPA has ever experienced: the increase in bureaucracy and procurement rules following the Vietnam/Watergate era. These shifts fundamentally changed how DARPA PMs were allowed to operate.

1. From a Small World to a Bureaucratic One

The post-Vietnam and Watergate shifts in Washington D.C. mark a clear “before and after” in DARPA history. Regulatory crackdowns and increased oversight became typical in this period. Much of this oversight impacted the entire government or the entire DoD at once. Regardless of whether or not DARPA had done anything wrong in the public’s eye, the government had. The sweeping reforms left DARPA a very different agency — from the perspective of both its PMs and its performers. While DARPA remained a more fast-acting and less bureaucratic place than many other agencies in D.C., it became noticeably more bureaucratic than it once was.

Procedural Changes (e.g. 1984 Competition in Contracting Act)

This general change in political attitudes had begun to steadily impact DARPA operations throughout the 1970s, but among the years of steady changes there were also abrupt shifts. One example was the implementation of the 1984 Competition in Contracting Act. To DARPA PMs and longtime performers who served both before and after the changes, this shift is often seen as demarcating where an old DARPA stopped and a new DARPA began. While it is reasonable for one to believe that the ramping up of oversight in how government funds were spent was, in principle, good. It’s more difficult to argue that the changes did not substantially impact the speed and freedom with which DARPA PMs could source, iterate upon, and fund ideas.

Even the procedural changes that seemed, on the surface, very reasonable had a noticeable impact on the speed and fluidity of DARPA’s operations. One example of this was the new requirement that agencies publish a notice for all procurements above $10,000 in the Commerce Business Daily. Once published, PMs would only have to wait thirty days for bids to be solicited and could then begin negotiations. Roland and Shiman — who wrote the book on DARPA’s Strategic Computing initiative and extensively interviewed many of the major staff and performers of 1980s DARPA IPTO — wrote the following on the impacts of this change:

Paired with the Competition in Contracting Act, this legislation transformed the old IPTO-ARPA style of funding research.

In the salad days of IPTO during the 1960s and 1970s, program managers solicited proposals in the research community, often by direct contact with the researchers. An office director such as J.C.R. Licklider, or a program manager such as Larry Roberts, for example, might call up a researcher to ask if he or she would be interested in doing a project for ARPA. If there was a meeting of the minds, the researcher would submit a proposal that was essentially approved before it came through the door. Program managers achieved results by knowing what was going on in the research community and who could deliver. They also accepted unsolicited proposals coming over the transom if they were good and suited ARPA’s purposes. And they put some contracts up for competition, as they did the ARPAnet contract that BBN won with Robert Kahn’s help. The style, however, was still less formal than at agencies such as NASA or even elsewhere within the DoD.

With the implementation of the Competition in Contracting Act, far more DARPA contracts were now formally competed. As a result, a wave of new federal procurement laws now applied to these contracts. This particularly impacted an organization like DARPA because DARPA staff not only fund work on the technological frontier, but come up with plans to shape it. DARPA PMs had previously relied on their ability to work out what a contract might look like with the performer community before a formal contract was ever written. Now, all the performers were legally competitors; how DARPA PMs could interact with them once a bid was out had changed. Roland and Shiman describe how this change in the legal relationship between PMs and the performer community impacted project creation, writing:

Particularly crippling to DARPA was the requirement in the new law limiting contact between procurement officers and potential contractors. Such contact was the taproot of the PM’s success. It was by staying in touch with the community, by letting the community know what was needed at DARPA, by finding out what the community was capable of doing that the PM was able to map out a reasonable research agenda in his field and sign up the best workers to make it happen. In the new dispensation, such networking could be viewed as collusion. PMs in the future would simply have to be much more careful, much more circumspect, much more correct when dealing with researchers who were potential contractors. This did not make their jobs impossible, but it did make them harder and less enjoyable.

“Harder and less enjoyable” is a succinct way to describe the differences in the lives of a DARPA PM of the 1980s vs. a DARPA PM of the late-1960s. If it were truly critical, there were procurement levers DARPA directors, office directors, and PMs could pull to make DARPA work the way they needed it to work. As one example, directors could use an OTA-like authority to get around particular pieces of red tape if it was truly important. But directors were also aware that this should only be done sparingly. Also, if a technicality got in the way of common-sense project management — like the above regulation which negatively impacted PMs’ ability to brainstorm with performers when putting together contracts — entrepreneurial individuals at DARPA often found ways to at least approximate their old common-sense workflows in legal ways. In fact, this is how DARPA’s Broad Agency Announcements (BAAs) came to exist. To get around the aforementioned issue, Stephen Squires came up with a technique for soliciting proposals — once described as “a cross between an RFP and a fishing expedition” — which took hold at DARPA.1 It eventually evolved into what is now the BAA. While the BAA was a wonderful organizational innovation, it was only necessary because of the aforementioned changes in procedure.

There was now an added step to the game of being a DARPA PM. PMs and office directors could still be proactive and ambitious, but they often had to devise strategies to make their projects work within the confines of the new legal procedures and red tape. By the late 1980s, many longtime DARPA affiliates felt the PMs could no longer do precisely what they wanted exactly when they wanted to do it. These shifts in procurement laws and contract management contributed to DARPA becoming a place in which funding could easily be given to promising ideas in days or weeks, to one where months was increasingly becoming the norm.

Of course, there were also games to be played in the prior DARPA regime. In the case of IPTO, for example, many researchers at computer science departments that did not receive IPTO’s Centers of Excellence grants — which Licklider’s IPTO office had used to foster top-tier computing departments when the field was in its nascent stages — complained that they were unfairly overlooked. To overcome being outside of the network was also a game, to be sure. A researcher might need to go further out of their way to put themselves on a PM’s radar. But the new procedures ushered in a game that was quite different in nature. The following subsections detail some of the performers’ and PMs’ perspectives on this new game and how it had to be played.

The Performer’s Perspective On These Changes

In their oral histories, often recorded once they had retired, long-time DARPA performers often listed this change in procurement bureaucracy as the single factor that most threatened DARPA’s future effectiveness.

In a series of interviews with those who played major roles in BBN’s work on the famous ARPAnet contract, two separate BBNers used the free air the interviewer provided them to put on the record how bad they felt these changes were for DARPA. In his 1990 oral history interview, Frank Heart, BBN’s project leader for the ARPAnet contract, described what he believed to be the increasing silliness of the government’s approach to technology contracting:

[Interviewer]: Okay, that covers the questions I had, unless you have any general comments you would like to add to the record.

HEART: As I say, the only thing I would emphasize is that if people try to look at all the projects that this country does in the computer world, and all the failures there have been, all the large military computer systems that have fallen on their face, or had a factor of ten overruns in time or money...If you look at all those things, and then you ask, "Why do these few make it and be successful, and why do all those not be successful?" That's an important thing, if one could somehow get that lesson across to Congress and others, it would be kind of nice. It's very hard apparently because they keep doing it the wrong way, mostly. What happens is that people who know nothing about the technology write long lists of requirements. Then they go out to industry and ask industry to bid on these silly requirements, and they get back silly bids. Then they pick the low bidder of the silly bids, which often is the one who understands the technology least; and then ten years later they have a disaster. Instead of people who understand the technology very, very well in the government who are able to drive it to the groups that are the right groups, even if it isn't the low-price bid, and who take close control of it, who don't have an enormous list of specs, and then other people who never wrote the specs, but who have a participatory prototyping kind of relationship with the thing where it grows. If one could get across that lesson, it would be kind of nice. But I don't hold much hope for that.

[Interviewer]: Can you give me any examples of what you consider big failures, using this kind of method?

HEART: Well, gee, there are probably... No, I probably can't do that, just because I wouldn't want to be quoted. But, I mean, they're legion. The number of defense systems where they've tried to build computer systems, and they've been horrible, there are probably lists twenty or so long. I probably wouldn't want to be quoted on naming them, but it's many, many, many subsystems that have been ten years late at ten times the cost — things like that. The ARPANET went from a contract award to an installed operating system in nine months. It went from a contract award to equipment being delivered, on site and running, in nine months.

Heart also believed that working with DARPA, in 1990, was still quite a positive experience and not nearly the headache it had become to work with other pieces of the government that supported research. But the following quote from Alexander McKenzie’s 1990 oral history — Mackenzie was BBN’s so-called “ARPAnet generalist” — makes it clear that these procurement changes had pronounced negative effects on DARPA performers, compared to how things used to be. McKenzie’s interview reads:

[Interviewer]: We have covered what I wanted to ask you about. I want to open it up and ask if there's anything you want to add, any general comments you would like to make about the ARPANET, or your involvement in it, or ARPA in general.

MCKENZIE: I think that the example of the way ARPA worked in those days, in the 1970s — not only in communication but in a lot of other programs — was pretty effective in terms of cost and return on investment. That the model of some really bright people in the ARPA office, with some particular goal in mind, something they thought would help the country, or the DoD, being given the authority to go out and find the smartest people they could find who were interested in working on that problem, and then giving them relatively free rein to do some research or some development, produced pretty remarkable results. And I know that there has been fraud and abuse in government and in contracting and so forth, but it seems to me that the kind of rules and regulations that there are now, that are attempting to prevent that, really make it very difficult for the government to get the same kind of power out of its research dollars these days as it was able to then. I know it's hard to find a balance between accountability and free rein, and these days the government approach seems to be more on the side of accountability and less on the side of free rein. But I think that a lot is being lost.

Mckenzie continues by sharing a quite specific example that he felt characterized the kind of thing DARPA eventually began to worry about that it hadn’t in the 1960s:

I remember a discussion that I had with Jon Postel back in, I don't know, the early 1970s. We were talking about travel to a Network Working Group [which helped come up with the early internet protocols] meeting. And he was saying that he really would like it — maybe it was a subgroup meeting — if it would be some place rather than some other place, because the government accountants were getting university accountants to not want them to spend so much on travel. And his comment was, "I know that in my contract there's only a fixed amount of money. I want to use that money the most effective way possible. I don't know why they think that, when my salary is paid out of this, that I'm going to waste money on unnecessary travel. I want the money for my salary; I want to keep having a job, but sometimes the travel is necessary. They don't seem to understand that that's the case. They think it's just a boondoggle." I think there's a lot to that attitude. That, yes, we spent a lot of time in Network Working Group meetings hammering out ideas, but something really good came out of it.

Mckenzie closes by explaining how, in his eyes, the previous DARPA regime was ideally set up to keep costs down in its own way, while staying ruthlessly focused on outcomes and not process:

BBN wasn't required to file monthly fund expenditures reports and justify every decision on the basis of cost effectiveness. We could pick the Honeywell 516 because it was OK — it was an OK processor for this. It would do the job, and it wasn't, maybe, the most cost effective and we didn't have a subcontract bidding plan in effect to give everybody the opportunity to bid, and thereby put another three months of delay into the schedule - we could just decide. I think that was pretty valuable. If you're talking about multi-million dollar projects, it's probably more important to pay attention to those things than it is in tens-of-thousand or hundreds-of-thousand dollar projects. But, even there, there's probably things where better results would come to the government if they could find a way to shift back a little bit more towards accountability for the results, rather than accountability for the pennies. I think that DARPA in the 1970s did a really good job for the country in that way. It was a joy to be associated with the ARPANET project. It was fun. It was challenging. And I think it was good for the country. It's not so easy to find that mix now, and I think regulation is a big part of it.2

The negative opinions on these bureaucratic changes at DARPA were not limited to its performers.

The PM’s Perspective

As these procedural changes began to take effect, the life of a PM became “harder and less enjoyable” in many ways. Opinions varied on how annoying the changes were. Many PMs thought the changes were horrendous for the pace and cost-efficiency of projects. Some, such as IPTO’s Ronald Ohlander, expressed less polarizingly negative opinions. Ohlander — whose budget more than tripled from $14 million to $55 million as DARPA’s computing applications budget grew — felt this increase in resources granted him many more opportunities than the increased paperwork and bureaucracy took away. But even after taking moderate views like Ohlander’s into account, it is clear that the DARPA of the mid-1980s operated with the increased computing budget much differently than the DARPA of 1970 would have.

PM Paul Losleben’s thoughts on the new mix of resources and bureaucracy were clear:

I…had twice the budget that I had had before…and I was accomplishing less because I had lost the freedom to build the program, to interact, to define direction, to work with the community.3

Ohlander, on the other hand, merely complained that he had to hire more staff than he otherwise would have to keep up with his budget increase. But, with that being said, some of the difficulties he experienced managing the ill-fated Autonomous Land Vehicle (ALV) project may have been mitigated if he could operate more like a ~1970 ARPA PM. The original ALV project had to be scrapped largely because of the mismanaged expectations and misaligned incentives of both Martin Marietta and the university vision researchers. As I covered in a prior FreakTakes piece, Martin Marietta turned out to be a prime systems integration contractor with little interest in implementing cutting-edge component technology. Martin pressed DARPA to specifically define intermediate benchmarks for the technology. Martin would then find ways to hit these benchmarks using minimal cutting-edge technology, not trying to satisfy the spirit of the contract. Meanwhile, many of the university-based computer vision researchers cared more about developing algorithms that were interesting in a basic research sense than they did about developing algorithms to make the vehicle work. Their algorithms often did not perform when plugged directly into machines like the NavLab at CMU.

Whether or not these failures would have happened absent the new procedures, at the minimum the new regulations exacerbated these issues. Roland and Shiman describe how these procurement changes negatively impacted Ohlander’s ALV project:

The new methods of procurement…made Ohlander’s task doubly difficult. In a previous era, he would have held informal discussions with the researchers at an early stage in the planning process, so that contractors understood what was expected of them and what they would contribute. Such a casual approach was no longer possible. The potential contractors were all competitors now. Ohlander could gather the researchers together to help write the [project] plan, but he could not discuss specifics with them about who would do what. The researchers themselves would have been very reluctant to talk, especially with their potential competitors. Ohlander could not be certain that the successful competitors would work well together. He had never worked at all with some of them, especially those in industry, such as GE and ADS. All he could do was to specify in their contracts that they were required to participate in open discussions and exchange information freely with the other contractors. They [DARPA] would have to trust in his [Ohlander’s] ability, as the source of funds, to twist arms if necessary. Running SCVision in two phases made this easier, as it allowed DARPA to cut off recalcitrant or underperforming contractors after only two years.

The multi-phase structure of the project proved to be essential. At the end of the first phase, DARPA would have to fire the prime contractor, Martin Marietta. While many of the academics continued to receive their basic research funding, they gradually became only indirect contributors to DARPA’s main autonomous vehicle project. Their research was often not as applied as DARPA needed it to be. The spirit of the ALV project primarily lived on in the autonomous vehicle teams Chuck Thorpe and others were building at the CMU Robotics Institute. In the CMU Robotics Institute, Raj Reddy had built up a research operation that valued what DARPA wanted it to value: cutting-edge research alongside practical systems engineering work. In CMU, DARPA’s autonomous vehicle staff eventually found a one-stop shop that could manage the practical aspects of the project and many of the research portions. That was great. But more time and money likely had to be spent on the wrong performers than may have been necessary under DARPA’s old procedures.

In 1984, as the Competition in Contracting Act was being implemented, a Science reporter named Mitchell Waldrop — now of Dream Machine fame — wrote an article that characterized the feeling of DARPA PMs and their performers as “confusion and heated tempers.” One side was frustrated as they figured out how to write bids for government contracts while the other tried to figure out how to evaluate them in a legal and auditable fashion. ARPA’s office staff and performers had begun to spend far more time checking compliance boxes than they ever had before. Bureaucratic obstacles stood between the PMs and their beloved performer communities. Things had changed.

To learn from any project from ARPA history, it is crucial to understand whether the project took place before or after these procedural shift and how that affected a PM’s project management.

2. Office Director’s Visions vs. PM’s Visions

DARPA has become quite famous for the freedom of its PMs. Many advocates of the ARPA model believe a PM’s freedom to pursue their own unique vision is an emphasis that DARPA has always had. However, in DARPA’s early decades, in particular, the freedom to decide which project areas to pursue often fell to its office directors, not individual PMs. But even in cases when specific goals were dictated to PMs by office directors, the PMs still generally had the freedom to decide exactly how to go after those goals.

There are pros and cons to leaving the job of “visionary” to office directors vs. the distributed cohort of PMs. The former may provide an office with fewer unique opinions but ensure that the whole office is working in a coordinated fashion towards a few primary goals. The latter ensures that a given cohort of PMs is free to pursue as many unique ideas as possible. Many offices from DARPA history tend to fall somewhere in between the two extreme ends of the spectrum.

A prominent example of a relatively director-driven office was IPTO in its early decades. Early IPTO office directors were picked because they could both provide a strong vision for the office as well as effective leadership of the PMs. Roland and Shiman describe the different directions in which early IPTO directors steered the office, writing:

J. C. R. Licklider revolutionized computing by aggressively supporting his twin hobby horses of time-sharing and man-machine symbiosis, and by proselytizing for his vision of an “intergalactic computer network.” He recruited Ivan Sutherland to do the same with graphics. Strong successors such as Taylor and Roberts ensured that artificial intelligence (AI) received the support it needed well into the 1970s. The payoff from AI was slower to come than in time-sharing, graphics, and networking, but IPTO kept faith with the field nonetheless.

These IPTO directors often fostered beneath them a few academically-decorated PMs with visions of their own which they were green-lit to pursue. But, oftentimes, there were even more PMs stewarding the visions of others than were encouraged/expected to have visions of their own.

Three examples of projects that were dreamt up by someone other than the PMs who ran them were the Pilot’s Associate, Battle Management, and Autonomous Land Vehicle projects — all covered in a prior FreakTakes piece. Several individuals across IPTO, in consultation with the armed forces, had a hand in dreaming up these projects. The most important of these individuals was probably Clinton Kelly. Despite this, Kelly became the PM of none of them. He helped guide them, in a way, but was not the official PM. Despite the fact PMs like John Retelle (Pilot’s Associate), John Flynn (ship-level Battle Management), Al Brandensteein (fleet-level Battle Management), and William Isler (ALV) did not come up with the ideas themselves, they still had relatively long leashes to execute their projects as they saw fit. Despite not having the freedom to come up with projects from scratch, their freedom was more similar to that of a SEAL team commander who might not get to choose which objective to take, but often has the freedom to decide how to take the objective. In fact, PMs from the armed forces found themselves in the position of stewarding others’ visions relatively frequently.

A major upside of office directors coming up with a vision and several PMs being expected to execute that one vision is that PMs could be assigned to projects that they might not 100% believe in. This can be useful. There exist cases that might draw appropriate skepticism from all individual PMs, but which it is optimal for the office portfolio to pursue nonetheless. I’ll provide an example of a program which was roughly this shape — one in which the right course of action required PMs to pursue all possibilities, however unlikely.

In the early 1980s, IPTO director and chief visionary Robert Kahn thought that computers needed about a three order of magnitude increase in performance before they could truly approach AI-type applications. There were three paths that he felt could make this happen. To be thorough, he felt it made sense to fund all of them — even though some seemed far more likely to work than others. The goal was important enough that he felt no stone should be left unturned. Option 1 was to improve chip performance, funded through the VLSI program. Option 2 was to speculatively design new software approaches that could be run on later generations of machines more suited to the approaches than present machines. Option 3 was considered the most promising option. Option 3 was to design and build parallel architectures. With the office largely following the vision of one or a few people, and not every single PM, Kahn was easily able to ensure each of these approaches was pursued thoroughly — even if options like speculative software design felt risky and unlikely to pay off.4

In IPTOs early decades, such as in the 1980s, it was also not uncommon for PMs to be called in to manage projects that were the vision of another PM. PM Ronald Ohlander’s portfolio of AI projects was one example of this strategy. In the following excerpt, Roland and Shiman describe what was expected of the non-visionary PMs brought in to help Ohlander manage his then-swelling AI portfolio:

All three men were typical of the active-duty military officers who did one or more tours at DARPA on assignment from their services. They served beside civilian program managers who came to DARPA from industry or academia…The civilians usually came with more powerful credentials in their technical fields; this often meant that they had technical agendas of their own and attempted to shape the field while at DARPA. The military officers came with a better understanding of the clientele for DARPA’s products…They could translate service needs for the computer science community and translate technical capabilities to serving officers. Sears played the latter role. He took over programs for which he had little formal training, but he educated himself on the job and relied on his common sense and management experience to keep the programs on track. Most importantly, he had to recognize good excursions from the plan, to know when a project was making better-than-expected progress and needed more resources to exploit the opportunity. Just as importantly, he had to know when a project was failing and had to be invigorated, redirected, or abandoned.

While many seem to intuitively understand the pros of PMs’ freedom to have visions of their own throughout DARPA history, it’s important to understand that this is not something that has been true of every office throughout DARPA history. For example, DARPA’s golden goose, early IPTO, did not always work this way. The individual freedom of PMs has huge upsides, but so does the entirety of an office’s PMs working in a coordinated fashion to accomplish a few goals. There is room for both approaches in any ARPA-like organization; the state of a field, an office’s goals, the personnel available, and the set of institutional circumstances an office finds itself in will determine which approach is optimal.

3. Payoff Timelines and DARPA’s Military-Focused Mandate

The final section details a variable that heavily dictates which projects are in scope for a PM and how a project is pursued: a DARPA office's attitudes towards payoff timelines and dual-use technology. This is often dictated by things like the preferences of DARPA directors and the political moment an office finds itself in.

Changes in acceptable payoff timelines for a project can come about for a variety of reasons. For example, wartime or a change in political winds can lead administrations to push for more “fiscal responsibility.” A common response to this is for DARPA directors to emphasize project plans that can produce applications quickly. On the flip side, certain directors and presidential administrations have been known to openly embrace dual-use technologies and provide generous payoff timelines. DARPA’s exceptional track record often earns the agency a longer leash than is typical. There is also a lot of middle ground between these two extremes. It is not uncommon for certain offices within DARPA to maintain quite short leashes and keep a clear focus on military practicality while, simultaneously, other offices exercise long leashes and focus on more dual-use technology with unclear time horizons for payoffs. This was the case in the 1970s in which the applied Engineering Applications Office (EAO) coexisted with the risk-embracing IPTO.

Early IPTO’s Dual-Use Ethos, the Mansfield Amendment, and George Heilmeier

Obvious shifts in timelines and attitudes towards dual-use technology might abruptly follow a change in the President and/or DARPA director. One example of an abrupt shift in how military-centric DARPA was expected to be came in the early 1970s. In 1970, with the Vietnam War waging and amid a generally troubled political environment, the Mansfield Amendment was passed as an addendum to the Defense Appropriation Act of 1970. The amendment proposed that the DoD should not be allowed to conduct any basic research. While the amendment was repealed within the year, the point Congress intended to make was clear. In response, ARPA soon changed its name to DARPA. The D in DARPA was no longer silent.5

Throughout the 1960s, Licklider and prominent IPTO PMs, such as Larry Roberts, had openly funded the most promising and useful research they could find, often without too much thought to how military-focused it was. In general, the IPTO office visionaries were true believers in the technology and its eventual widespread usefulness in all areas of society. So, they were generally happy to grant funds to any researchers who would push the computing field forward. The general, dual-use ethos that drove these visionaries was something along the lines of, “What’s good for the country will be good for the military.”6

As we now know, this attitude proved to be spot on in the case of computing. When the field of computing was in its nascent stages and much still needed to be worked out, IPTO gave massive block grants to computing “centers of excellence” like MIT, Stanford, CMU, University of Utah, and others. While many somewhat applied projects and direct asks were made of the centers of excellence which were often funded by the block grants, there was a lot of excess funding from the grants which the departments were free to use to pursue their own interests. Notably, the ARPAnet contract itself was pursued by Larry Roberts largely with general use cases in mind. The driving force behind the project was not exactly a military-specific need, but the idea that a broader communication breakthrough like the ARPAnet must improve military operations somehow. Despite the success IPTO had with this approach, this style of operating soon became more difficult.

While the Mansfield Amendment was repealed, it succeeded in forcing changes on DARPA beyond making the D in the name no longer silent. At the minimum, it was now clear to DARPA PMs and office directors that they were being watched. They were now careful to justify projects with specific military applications in mind and find more development projects. Larry Roberts described some of the changes as follows:

The Mansfield Amendment…forced us to generate considerable paperwork and to have to defend things on a different basis. It made us have more development work compared to the research work in order to get a mix such that we could defend it…The formal submissions to Congress for AI were written so that the possible impact was emphasized, not the theoretical considerations.7

However, Roberts did note in his oral history that while the mix of funds did shift to become more applied, for his personal budget DARPA found additional money for applications and that his basic research budget was not cut. While this applied shift felt abrupt to PMs at the time, a modern PM would probably classify this era of DARPA history as relatively exploratory.

This applied shift, which came in the wake of the Masnfield Amendment, ratcheted up a level in 1975. This is when DARPA appointed its most well-known director: George Heilmeier. Heilmeier would become famous for exercising a very heavy hand in deciding which projects would be pursued at DARPA along with his famous Heilmeier Catechism. Office directors and PMs now needed to have satisfying answers to Heilmeier’s six questions related to things like current technological limitations, applications, and intermediate measures of success if they wanted funding for a project. While none of the principles in the Heilmeier Catechism were new to DARPA, the expectation that each of the six questions had a thorough answer was much stricter. Roland and Shiman attempted to describe many PMs’ views on the change, writing:

Heilmeier’s catechism was not unreasonable, but he reportedly applied it with a draconian enthusiasm that reflected a chilling political climate settling over Washington in the first half of the 1970s.

While Heilmeir’s DARPA may not look too strict to modern eyes, it felt very strict compared to prior ARPA regimes. Heilmeier believed DARPA should look much more like mission-driven agencies, such as NASA, than a basic research funder like the NSF. He thought DARPA had unreasonably broadened its scope. He even mocked what had come of DARPA, derisively calling it “NSF West.” Under Heilmeier, PMs were pushed harder to get their exploratory research out into the field as applications. Funding exploratory research with only a hazy idea of what the applications could look like became a taller task than before. For those in D.C. who felt DARPA could be perceived as spending recklessly, Heilmeier was just the man to rein it in.

But Heilmeier’s tenure at DARPA should not be viewed as an event that marked some monotonic decrease in DARPA’s embrace of exploratory, dual-use technology. While the federal government norm is often for things to grow only more onerous over time, in DARPA history it is quite common for operating rules mandating a certain level of strictness to abruptly become less onerous.8 The period coinciding with the departure of Heilmeier as DARPA director and the prospective appointment of Robert Kahn as IPTO Director brought about one of these shifts on an office level.

Under Heilmeier, basic research funding from DARPA had become insufficient by the standards of Kahn and the computing community. New DARPA Director Robert Fossum was sympathetic to this viewpoint and was happy to bring Kahn on as IPTO director and honor his one condition to take the job: that DARPA increase basic research funding in computing. Roland and Shiman described how this shift in how applications-focused IPTO was occurred, writing:

In congressional testimony in 1981, Fossum reported that DARPA funding in constant dollars had fallen steadily from 1970 to 1976; it had not yet recovered its 1970 level when Kahn became IPTO director in 1979. The post-Vietnam heat of the mid-1970s was abating, Fossum did not share Heilmeier’s catechism or his pessimism about artificial intelligence, and Fossum wanted Kahn to take the job. By 1982 the IPTO budget had climbed from its 1979 low back up to a respectable $82 million.

These changes in an office’s focus, which are often the result of non-technological factors, not only impact which projects enter the DARPA funnel and continue to receive funding, but also the life cycle that a given project takes. In the case of DARPA’s 1980 Strategic Computing Initiative and the underlying pressure to have clear applications for the program, this pressure likely had both positive and negative impacts. When assessing DARPA’s impact on the development of autonomous vehicles, the pressure may have been for the best. While the Autonomous Land Vehicle project did come up short, the push to get vision applications into the real vehicles quickly allowed CMU to prove itself to be DARPA’s great systems integration contractor in the field of autonomous vehicles. On the flip side, in the case of the Pilot’s Associate program, the PM in charge of the program thought the premature push for a mature prototype took a program that was making solid technological leaps and forced it to abruptly implement a system that was not as cutting-edge or intelligent as it could have been otherwise.

In very obvious ways, variations in DARPA’s attitudes towards dual-use technology and the pursuit of projects with long-run or unclear payoffs impact which projects get chosen and how they are pursued. But one would be incorrect to view some eras of DARPA as simply better or worse places for the most ambitious research. For example, while some (too crudely) might view Tony Tether’s wartime DARPA agency in the 2000s as a more difficult place to pursue ambitious programs — given its use of procedures like budget wedges and emphasis on programs that might pay off in the current war — an entrepreneurial PM like Geoff Ling proved this was an ideal time to pursue one of DARPA’s most ambitious programs ever: the Revolutionizing Prosthetics Program. Ling’s program made major breakthroughs in neurally integrated upper limb prosthetics and brain computer interfaces (BCIs) that helped lay the base for some of the ambitious BCI projects of today.

While the impact of an office’s attitude on dual-use technology and payoff timelines is somewhat intuitive, one important wrinkle to consider is whether the customer (the military) must actually agree an application is desirable before a project is funded. Whether or not an office can pursue a project despite the military being lukewarm or cold on an idea has a massive impact on what sorts of projects get pursued.

A military-focused office can often say “no” to the military

Military-focused project selection and listening to the military are not always the same thing. Different DARPA regimes and offices can have varying abilities to say “no” to the armed services. In the case of the Pilot’s Associate, the Air Force did want it and indicated they‘d be willing to actively participate in its development. In the case of the ALV, the Army did not express much desire for an autonomous recon vehicle — at least not strong enough to warrant DARPA’s spending on the vehicle — but IPTO staff had a strong belief that the Army would want the ALV once they saw it in action. So, DARPA was comfortable making the ALV application the centerpiece of its SCVision portfolio. This ability to say no to the army in the 1980s DARPA computing portfolio stands in stark contrast to the accounts of some PMs who worked under directors like Tether. Tether often mandated that PMs obtain a budget wedge — a formal promise by an armed service to continue funding a particular project if the R&D benchmarks reached a certain point — before beginning to fund many projects.

In Tether’s case, the recent 9/11 attacks and the country transitioning into wartime allegedly had a strong impact on this decision. Ling’s prosthetics program was the right kind of ambitious for the moment; meanwhile, overriding the no of the military to build seemingly futurist technology the Army said it did not want likely would not have been. One cannot truly understand and apply the lessons of any project’s success or failure without understanding the office’s attitudes towards dual-use technology, payoff timelines, and saying no to the customer in a given era.

Concluding Thoughts

DARPA is no longer a young government agency. In a difficult environment, it has managed to remain a place where bright minds take risks, act (relatively) nimbly, and get things done. The agency continues to specialize in pursuing projects that are big-if-true, fall through the cracks of the standard R&D ecosystem, or might succumb to the valley of death without DARPA’s intervention. The agency’s history is one we would be foolish not to learn from.

To adequately learn from this history, it’s crucial we understand what changes and what stays the same about the agency over time. Certain key factors have remained constant throughout (D)ARPA’s life. The twin constants that, to me, define the ARPA model over time are the use of a clear customer relationship as the agency’s North Star and the use visionary PMs (or office directors) with agency instead of passive funding panels. But it is possible that there are even more key variables across DARPA history than there are constants. Today’s piece outlines what I believe to be three of the most important ones those. Those looking to learn applied lessons from the agency’s history should keep them in mind.

Thanks for reading:) As always, I’d be eager to discuss/assist PMs or PM-like individuals in any way I can. Just reach out to me on Twitter (eric_is_weird) or via email (egillia3@alumni.stanford.edu) to chat.

Announcements

An Announcement on RenPhil’s AI For Math Fund. Talented and curious readers should eagerly apply for grant funding from RenPhil and XTX Markets’ AI for Math Fund. The funders are particularly interested in those working on software, datasets, field building, and big-if-true projects to build AI technologies useful to mathematicians.

A Coming Piece Will Cover Constants Across DARPA History. While the focus of this piece was on how DARPA has changed over time, there are also important constants throughout DARPA history. I will say more about these in a brief follow-up piece on FreakTakes. For now, I’ll simply say that the two most notable constants are probably:

1. DARPA’s continued commitment to replacing the panels typical in government R&D grant-making with PMs with agency

2. Its use of a clear customer relationship as a strong North Star for its projects

Subscribe now

Share

Appendix: A Note on Other Headwinds In The 1980s

Regulations are not solely to blame for the initial negative results of the ALV project, and projects like it. In the mid-1980s, Ohlander was fighting against two other sweeping trends that surely made the life of a PM for a project like the ALV harder. Many groups that would once have been more aligned with DARPA’s incentives were gradually drifting in scope.

The first of these headwinds was the gradual drift of university labs away from applied research contracts. With post-war federal basic research funding continuing its explosion, universities were leaning more and more into basic research funding and further away from the applied research contracts that were a key source of funding in prior decades. University labs were gradually growing less well-suited to DARPA-style contracts that required a very applied technological component — like operating a working vehicle in which to plug academic vision algorithms.

The second headwind PMs like Ohlander were fighting against in the 1980s was an increasingly myopic sense of what projects corporate R&D should pursue. A 1970s recession, the 1980s “financialization” of management in engineering organizations, and likely other causes saw corporate research drift in its ambition. My prior FreakTakes piece on Lockheed’s Skunk Works provides an interesting case study of what these pressures looked like from within a defense prime. It was even difficult for Kelly Johnson and the famous Skunk Works to fight against them as the decades wore on.

Both of these trends happening simultaneously contributed to an organization like BBN’s ability to recruit top academic engineers from places like Lincoln Labs so convincingly. BBN’s recruits simultaneously felt that Lincoln Labs did not work on truly useful technology, but also that their minds would be wasted at standard corporate R&D departments like Honeywell’s. BBN was one of a small number of organizations that still existed to work on truly novel, truly useful applied research contracts. These two simultaneous trends likely had their own negative impacts on DARPA on top of the negative impacts brought on by the new procurement rules. But, at the minimum, the growing apart of industry and academic research made it even more vital that DARPA keep a strong hand in coordinating — and when needed, twisting the arms of — performers on an ongoing basis. The new procedures made this much more difficult.

General Links

Beyond the specific citations above, much of the general knowledge in the piece is indebted to the great books and oral histories on early IPTO conducted/written by Judy O’Neill, Arthur Norberg, William Aspray, Alex Roland, and Philip Shiman:

• University of Minnesota IPTO oral histories can be found here

• Transforming Computer Technology: Information Processing for the Pentagon, 1962-1986

• Strategic Computing: DARPA and the Quest for Machine Intelligence, 1983-1993

By "exceptional," I mean contractor groups who were not just staffed with elite talent, but uniquely aligned with ARPA’s mission. All three aforementioned ARPA success stories had contractors — which DARPA calls performers — that shared three distinct traits, rarely found together. Each performer:

1. Was novelty-seeking, with a strong preference for projects that pushed the technological frontier forward substantially.

2. Built useful technology for actual users. This entailed professional contract management and a willingness to focus on difficult systems engineering tasks.

3. Used more flexible team structures than academia. When compared to academia, they more effectively hired, organized, and incentivized researchers, engineers, and other experts to collaborate on applied projects in a common-sense fashion.

I call orgs that check all three boxes BBN-model orgs — named after Bolt, Beranek & Newman (the ARPAnet contractor). In this piece, I make the case for the BBN Model and why it can be a fantastic complement to the FRO Model for the R&D community.

In the first section, I outline how BBNs compare to FROs and the bottleneck they address. In the second and third sections, I'll summarize how historically-great DARPA performers like BBN and CMU's early autonomous vehicle teams used the approach to set themselves apart from other academic departments and firms. In the fourth section, I'll list examples of problems this model is optimized to pursue and briefly discuss what is needed to get BBNs off the ground.

Subscribe now

Share

I've limited the scope of this piece to simply introducing the concept of BBN-model orgs. While this approach does not apply to every area of R&D, the idea is already finding traction with a small number of founders and funders. Just last week, a bootstrapped BBN, Amodo Design, was made one of nine ARIA activation partners — alongside groups like DeepMind and Convergent Research. I’ll share more details about Amodo and other aspiring BBNs in coming pieces. In the meantime, if you are an entrepreneurial scientist or philanthropist who would like to discuss this topic, please reach out (egillia3@alumni.stanford.edu).

Thanks to Thomas Milton, Corin Wagen, and Adam Marblestone for their comments and ideas on the draft.

BBNs: A Complement to FROs

In 2022, Adam Marblestone et al. published their now well-known Nature piece detailing the vision for Focused Research Organizations (FROs). The team made clear the problem they planned to address:

Our goal is to create a model to support an ecosystem of small-to-mid scale projects that fall between the cracks of what start-ups, academia and other organizations do.

The authors paint a picture of time-bound, nonprofit R&D startups wholly funded to pursue a well-defined goal — usually to build a specific thing like a dataset or tool. They point to "grand projects" like the Human Genome Project, Large Hadron Collider, Hubble Telescope, and Human Cell Atlas as having similar goals. Less than three years on from that piece, Convergent Research has brought six FROs into existence (and counting). Adam and the team have proven that you can recruit teams of the very best researchers to join these orgs, make them high-status, and get the operational scaffolding in place to begin doing great work with minimal delay. Convergent’s experiment in scientific structures is off to a great start.

But fundraising is a clear bottleneck to the creation of new FROs — often costing $20 to $100 million. There is, of course, an upside to this tradeoff. FROs raise so much money upfront to ensure they will have all the money they need to pursue their vision from day one. But until groups like universities or the NIH are convinced to set aside substantial war chests to fund and run FROs, fundraising will continue to be a bottleneck to FRO creation. In the meantime, many with the talent and technical vision to found and run FROs will not be able to do so if they are not one of the lucky few to complete a successful FRO fundraise.

BBNs offer an alternative path for these talented individuals. BBNs raise less money upfront than FROs. Instead, they heavily incorporate a contract research approach to build towards the ambitious technical vision of their founding field strategist. They can operate as firms or nonprofits, drawing on a mix of R&D grants and contracts to fund their technical agenda. Project by project, BBNs can make strides in key areas of R&D — as BBN did with real-time computing and the CMU teams did with autonomous vehicles.1 The approach limits BBN founding areas to those in which funders are eager to pay for contracts aligned with some ambitious technical vision. But where those markets do exist, BBNs can be spun up for an order of magnitude less upfront capital than FROs in many cases — some might even be bootstrapped.

The downside is that this introduces a level of ongoing financial risk that is not present in FROs. Founding a BBN is not advisable if you can successfully fundraise for an FRO or have an offer to run a team at the Arc Institute. Those are more stable options. BBN founders may need to be a bit scrappy or creative in their revenue strategies to survive and expand. They might have grand ambitions on the scale of FROs or the Arc Institute, but they must begin with plans that, in the beginning, operate on the scale of an NIH grant, philanthropic grant, or DARPA contract. But if done well, as with FROs, the BBN model can enable a “critical mass of scientists, researchers, and managers” to assemble and pursue ambitious R&D agendas outside of the university.

BBNs offer ambitious field strategists a chance. With BBNs, the chance offered by some contracts and a bright team can be leveraged to world-changing effect. While BBNs might even compete for the same contracts as groups like Raytheon, what differentiates the two is that BBNs are steered in ways that prioritize an ambitious technical agenda over revenue-maximization. This can be tough, but it can be done. The early career of the original BBN’s first computing field strategist, J.C.R. Licklider, demonstrates this. His steering of the BBN computing portfolio provides a clear example of how the BBN approach — with the right field strategist, team, and contracts — can drive an exceptionally ambitious R&D agenda.

BBN’s Computing Agenda: A Mix of Grants, Contracts, Talent, and Vision

BBN was founded in the 1940s. It began as a psychoacoustics R&D firm spun out of MIT by two professors. The firm began as a way for the professors to take on large contracts that were cutting-edge, but too involved to pursue in their own labs — concert hall design for the UN, cockpit design R&D for the Navy, etc. Over the next decade, the firm opportunistically expanded. Hiring for their firm as if they were staffing an academic department, the rule of thumb was to only make hires that would “raise the average level of competence of the firm.” In the late 1950s, the BBN partners looked to expand into computing. They believed J.C.R. Licklider was the best person to lead this effort. Licklider had a clear vision for the near future of computing. It was a vision that might have been too engineering-heavy to be optimally pursued at MIT.

BBN offered an interesting alternative. The firm had carved out a useful niche for itself, with a comparative advantage in projects that required both:

1. Cutting-edge academic knowledge

2. Professional engineering work or contract management

Licklider, a man of impressive vision, saw the administrative scaffolding BBN provided and believed it could be used to pursue his technical vision of real-time personal computing. This technical agenda — quite different from the batch-processing paradigm then in vogue — would require substantial engineering improvements in computing interfaces and real-time computing technology. Licklider left his tenured position at MIT to join BBN. Under his leadership, BBN’s computing efforts would go on to earn BBN praise from many former MIT researchers, later being described as “the third great university of Cambridge,” “the cognac of the research business,” and the true “middle ground between academia and the commercial world.”

A field strategist with an infectious sales pitch, Lick would gradually convince many MIT, Harvard, and Lincoln Labs researchers to defect and join BBN. This included a core of rare real-time computing experts, many of whom came from Lincoln Labs. While MIT and Lincoln Labs were great research environments, many who defected to BBN felt the nature of academia forced them to leave their best ideas as under-developed prototypes or “applied” ideas in papers. Many of these individuals — more engineers than pure researchers — would have felt their minds were wasted at a large firm like Honeywell. But BBN offered an interesting alternative to them as well. They could simultaneously work on the hardest problems and build useful technology.

Licklider convinced BBN to provide funds for computers that were relatively generous by the firm’s standards, but quite small given the scope of his ambition. The work began with a $30,000 Royal McBee ($300,000 today). Licklider soon convinced the BBN partners to buy another, more powerful machine that he felt would be more useful. After this second purchase, Licklider immediately proved the partners’ confidence in him was well-placed. One BBN founder described how seamlessly contracts fell into place following the second purchase, writing:

Lick and I took off for Washington D.C. to seek research contracts that would make use of this machine, which carried a price tag of $150,000 (~$1.5 million today). Our visits to the Department of Education, National Institutes of Health, National Science Foundation, NASA, and the Department of Defense proved Lick’s convictions correct, and we soon secured several important contracts.

Licklider’s computing group routinely found creative ways to offset the costs of investigations aligned with their technical agenda. Some of the group’s revenue came from relatively typical academic funding sources — federal research grants, a NASA contract to use the group’s computer to produce a textbook, etc. But BBN also took on much more applied contracts, well-suited to the group’s embrace of engineering and project management tasks. I’ll briefly describe two of these applied projects that took advantage of BBN’s comparative advantages. Both helped lay the groundwork for now-famous technical breakthroughs.

The first of these projects was BBN's Libraries of the Future Project. The project, indirectly funded by the Ford Foundation, sought to understand how computing might impact libraries moving forward. It was a contract for a philanthropy that, today, might go to a consulting firm like the Bridgespan Group. But in this unassuming contract, Lick saw an opportunity. The BBN team won the contract and used it as an excuse to explore seemingly futurist technology and make practical engineering improvements. As one example, the second half of the group’s book-length report for this unassuming contract contains technical explorations that attempt to emulate a future in which users interact with the library as a store of knowledge with a question-answering front-end, not as a way to find books. The final report also contained early work related to PC file systems, improved information retrieval methods, mathematical representations of information, and associated hardware improvements — all paid for by this unassuming contract. The opportunities to further explore and translate academic ideas via this contract proved attractive to other researchers in Cambridge. On this project alone, Marvin Minsky, John McCarthy, and Fischer Black all spent time working with the BBN group.

The second of these projects was built on a contract to develop a usable real-time computing system in a presciently chosen application area: hospital administration. The contract came about because a BBN partner made a compelling pitch to the NIH Clinical Center’s Director that hospital administration would be done using computers at some point. The director was convinced and had the partner write up a proposal for BBN to begin developing and building a computing system to pilot with a research hospital. The BBN hospital time-sharing computer project provided the firm with three years of funding, worth ~$10 million in total today. The contracting relationship would continue for several additional years, growing into an actual system deployed in the operations of Massachusetts General Hospital. With the contract, BBN was able to fund years of work a bit too far along for academia but not close to private-market-readiness. The contract required far more contract management and engineering work than would have been reasonable for a university team — they had to continually reduce machine error rates, install and maintain terminals at the hospital, coordinate with hospital staff, train doctors, provide customer service, make UX changes, etc. But with this applied work came a sizable budget. BBN’s small team of real-time computing experts used it to push the engineering frontier forward.

By the time Larry Roberts began putting out feelers for the ARPAnet contract in the late-1960s, BBN had been solving all sorts of related problems for years — similar to the ones that had stumped Roberts while he was at MIT. BBN won the contract. The first four nodes of the ARPAnet would be delivered within a year, on time and on budget. But it was BBN’s years of work on related contracts that paved the way for this overnight success.

Nonprofit BBNs: The Case of CMU's Early Autonomous Vehicle Teams

The BBN model can also be operated with a nonprofit structure. The early CMU autonomous vehicle teams — which helped pave the way for the autonomous vehicle revolution — demonstrate this. In another world, these teams probably could have operated with an FRO structure. But in practice, they operated using a model more similar to BBN’s contract research model.

The CMU Robotics Institute was primarily established to build ambitious, practical computing systems. While housed at a university, these CMU teams are more aptly compared to a group like BBN than a group like Marvin Minsky's computer science department at MIT. Contracts were a large part of the Institute’s plans from its inception. The Institute’s early autonomous vehicle group successfully deployed a BBN-style approach on DARPA’s 1980s autonomous vehicle contracts, achieving clear counterfactual impact and developing systems DARPA’s primes and academic labs under contract would not.2

The CMU group had an operational structure and incentives that enabled it to excel on these ambitious, applied contracts. As one example, the Robotics Institute created suitable tracks through which engineering and project management-focused individuals could be hired and tenured just as CMU's best professors would be. Instead of researching and teaching classes, these individuals would research and operate projects. A CMU Ph.D. and one-time project manager for the group’s early autonomous vehicle projects, Chuck Thorpe described the incentives of his role, recounting:

Raj [Reddy] took me aside. He said, “I don't care how many papers you write. I don't care how many awards you win. That vehicle has to move down the road. If you do that, you're good. I'll defend you. I'll support you, I'll promote you.”

The CMU teams also found ways to effectively use the grad students as both project staff and researchers. Requiring dissertation problems of their own they could “sole author,” grad students were assigned components of the system for their dissertation to improve upon. Each project had to easily plug into the larger system. High-risk, high-reward projects still had a place in this structure. If projects were high-risk, project managers like Chuck Thorpe might simply assign more grad students to improve a component to ensure at least one succeeded. A good project might reduce the processing time for data from some sensor 10X. The best grad student project would prove to be world-changing.

In 1988, a sharp grad student named Dean Pommerleau combined his knowledge of CMU’s systolic array machine and neural nets to build a neural net-based steering system for the vehicle — ALVINN. In the following years, the team built this neural net steering system into a new vehicle system. In 1995, with this new system, the CMU vehicle would drive cross-country 98.5% autonomously on its “No Hands Across America” tour.

Getting BBNs Off The Ground

Attempting to found and operate a BBN is not easy. You might fail to raise a required seed grant or win contracts to get off the ground. You might sell a contract funder on your idea but need a partnership with an existing org or philanthropy to get the deal over the finish line. You might win an initial batch of revenue and still have to shut down after a year due to a lack of funds. You might even succeed in raising funds, but raise them from misaligned funders and become a myopic R&D contractor. This is a failure mode all the same; becoming Battelle or Raytheon is not the goal of a BBN. But great field strategists can use the BBN model to turn revenue, people, and vision into what Patrick Collison recently called a “great scene of discovery.” This is what BBN made itself into for real-time computing and CMU for autonomous vehicles.

Building a BBN is a viable, alternative path for those field strategists and scientific entrepreneurs eager to do work in the cracks of the R&D ecosystem. Done well, BBNs can be a force multiplier to investments made by scientific funders, enabling them to undertake projects that are cheaper, better, or different than they would otherwise. This “create the performer” approach still has its uses in the context of modern philanthropy.3 As Jacob Trefethen recently discussed on his blog, the Gates Foundation funds “half or more” of many global health product development partnerships (PDP) budgets because they usefully fill cracks vital to Gate’s global health mission.

But founding BBNs can require overcoming a somewhat delicate coordination problem. While some BBNs can be bootstrapped, many BBNs will require some combination of a low-seven-figure seed grant to get started, contracts, and (maybe) NSF/NIH grants to optimally pursue their research agenda. It takes a bit of talent to overcome the associated chicken-and-egg problems that might arise. Philanthropies might want to provide seed funds to spin up a BBN, but only if they can be assured contracts for R&D projects will materialize. A contract funder like DARPA might require the org actually exist before engaging in some negotiations. OpenPhil might happily provide 60% of the budget for ~2 years for some BBN useful to its mission but require the group to source additional funds and contracts to cover the rest. The list goes on. It’s not an easy path. But if the crack is big enough and a field strategist is proactive enough, these challenges can be overcome.4

But none of this can happen without an ambitious, open-minded contract funder. Funders with the level of vision and ambition of ARIA are ideal. I would encourage funders reading this to ask themselves the question, “Could I take my portfolio to another level if I was able to create a single performer to work in a targeted area?” If the answer is “yes” or “maybe,” you might benefit from a BBN.

BBNs might apply to many areas. Some areas in which BBNs might have natural comparative advantages include:

• Development of new instruments

• Producing bespoke hardware

• Engineering new pipelines to serve a group of researchers

• Building and maintaining research software or datasets

• Pilot-plant style work with the intention to:

  • Scale promising lab bench ideas in manufacturable processes

  • Produce a key research material for labs

• Translation of academic ideas to non-venture markets.

  • One example could be a CivTech BBN that wins city government contracts to both translate the best social science insights into practice and conducts research projects on the org’s internal data.

It’s hard to predict how many areas might benefit from a BBN, but the model offers opportunities for the ideas of special individuals to surprise you. Progress studies funder and Stripe CEO Patrick Collison recently spoke of a similar phenomenon regarding a new Stripe product, tweeting:

We launched Stripe Issuing a few years ago. I wasn't sure how many use-cases there would be — like, issuing a card is surely a pretty niche thing to do. As usual, developer platforms surprise, and customers have now issued a cumulative 200 million cards.

BBN’s idea to bid on the Libraries of the Future project is one example of this cleverness at work. The BBN model has the potential to not just plug obvious cracks in the R&D ecosystem, but create great scenes of discovery in their place. In which cracks these scenes will take hold is hard to predict.

The fact that FROs now exist and are a fixture of the new science lexicon makes the task of coming up with ideas for BBNs easier for field strategists. A good FRO problem is, in many cases, rather similar to a problem for a BBN. Adam Marblestone recently shared one example, explaining that an FRO might arise from a problem like neuroscientists requiring a different kind of microchip. One complex chip solving many life sciences problems is a great example of an FRO problem. But what if many neuroscience applications required many, simpler microchips? That’s a problem well-suited to a BBN.5

Tyler Cowen points to the history of Hollywood as proof that exciting young people can find ways to create exceptional things if given an opportunity to take charge and figure things out. FROs, with their time-bound nature, remind him of Hollywood movies. BBNs, with the scrappy nature of their origins, have the chance to become a vehicle for unproven R&D entrepreneurs to prove they are masters of their craft — just as Tyler believes Paul McCartney did with the Beatles.

It is an exciting proposition. But founders and funders with vision are required to make it a reality.

Thanks for reading.

Please reach out if…

I am already working with a few field strategists who believe they might have ideas for BBNs in areas like automated material discovery, bespoke semiconductor production as test bed hardware for certain areas of semiconductor design, and modifying instruments to enable a wider range of life science experiments. I will release those pieces in the coming weeks. Please reach out if you are a funder or interested field strategist to discuss ideas (egillia3@alumni.stanford.edu or on Twitter). I’d love to help!

I am setting aside as much time as necessary to assist would-be BBN founders and funders. I believe we have J.C.R. Lickliders sitting on the shelf with vision, philanthropists with exciting ideas that are best served by a new contractor, and outside funders who might happily provide the seed funding to catalyze this change. BBNs can be a vehicle to turn this potential energy into action.

Share

Various ideas and details included in this piece have been explored in FreakTakes pieces on BBN, DARPA’s Autonomous Land Vehicle and NavLab projects, an interview with CMU’s Chuck Thorpe, and how BBNs might apply to new era semiconductor research.

I’d also like to mention that Adam Marblestone listed OtherLab as an org that does great work with a BBN-type model.

Introduction

Many experts believe we are living in a post-Moore’s law world. NVIDIA’s Jensen Huang and computing legends like John Hennessy and David Patterson count themselves among them. But that does not mean these predictions spell doom for the computing field. Their predictions simply acknowledge that we are approaching the end of the “there’s plenty of room at the bottom” era of general-purpose computing — and moving on to another.

As Hennessy and Patterson point out, this shift hearkens the semiconductor ecosystem back to a style of work that was common before the 1980s — an era FreakTakes has covered extensively. In this piece, I’ll detail how the best teams from that period operated and why it’s necessary to bring those operating models back to thrive in this new era of semiconductor design.

The piece is broken up into three sections.

• In the first section, I briefly cover the history of the “there’s plenty of room at the bottom” era.

• In the next section, I explain what experts believe a “there’s plenty of room at the top” era could look like. Current researchers and veterans like Hennessy, Patterson, and Leiserson et al. believe the shift to looking for progress at the top of the computing stack will involve the vertical integration of research teams and a focus on domain-specialized hardware.

• In many ways, this is a shift back to how computing research groups used to operate before the 1980s. So, in the third section, I will summarize key management decisions that characterized some of the best old school computing organizations — such as BBN and the CMU autonomous vehicle groups.

With that, let’s get into it.

Subscribe now

Share

This piece heavily draws on two prior FreakTakes pieces on the BBN and CMU teams:

• “The Third University of Cambridge”: BBN and the Development of the ARPAnet

• An Interview with Chuck Thorpe on CMU: Operating an autonomous vehicle research powerhouse

Readers familiar with the basics of semiconductor history may skip the first section. Readers familiar with Leiserson et al.’s “There’s plenty of room at the Top” paper may skip all but the final three paragraphs of the second section.

There was plenty of room at the bottom

The optimism that drove the early Moore’s Law era of general-purpose computing is well-characterized by Richard Feynman’s “There’s Plenty of Room at the Bottom,” speech to the American Physical Society in 1959. In his speech, Feynman implored scientists and engineers to focus their efforts on the exciting opportunity of engineering biological, electrical, and physical systems on exceedingly small scales. He believed we were nowhere near the limitations of physics when it came to engineering systems on these scales. So, he asserted that the area was ripe for talented engineers and scientists to pile in and build a world of systems whose possibilities were orders of magnitude beyond the status quo in 1959.

For decades, the semiconductor field did just that, doubling performance every 18-24 months until the early 2000s. It still continues to excel at a remarkable rate, simply with more tradeoffs than we have become accustomed to seeing. The practical foresight of individuals like Gordon Moore in his 1965 Moore’s Law paper, Robert Dennard et al. in the more technically-detailed 1974 Dennard Scaling paper, and Gordon Moore again in his 1975 paper — summarizing of the field’s progress and providing updated predictions for the coming decades — largely held true until the early 2000s.

As the early 2000s approached, it was becoming clear to many in the industry that the general approaches outlined in the Dennard Scaling paper were not going to continue to provide the same improvements. The aptly named 1999 Paul Packan paper, Pushing the Limits, outlined several physical limits the field was approaching with the current approaches — including thermal limit challenges. Many of the specific challenges Packan outlined in the paper had no known path of attack that did not entail major trade-offs. Industry attempted to take alternative paths of attack to continue to scale computing power — such as the increased emphasis on multi-core processors — but the limitations researchers like Packan outlined have proven to be quite difficult. Since the mid-2000s progress in the field has continued, but it has not been Moore’s Law-level progress.

So, if there is not much cost-efficient room to be found at the bottom, where should the field look? Well, many researchers believe the natural place to look is: the top.

There’s plenty of room at the top, but…

In Leiserson et al.’s 2020 paper, There’s plenty of room at the Top, a group of researchers from MIT outlines a vision for the potential gains in computing power by focusing on the top of the computing stack — software, algorithms, and hardware architecture. They are quick to point out that this approach will require divorcing one's thinking from the “rising tide lifts all boats” approach which many became accustomed to under Moore’s Law. The authors write:

Unlike the historical gains at the Bottom…gains at the Top will be opportunistic, uneven, and sporadic…they will be subject to diminishing returns as specific computations become better explored.

The authors paint a broad picture of what this integrated approach will look like, writing:

Architectures are likely to become increasingly heterogeneous, incorporating both general-purpose and special-purpose circuitry. To improve performance, programs will need to expose more parallelism and locality for the hardware to exploit. In addition, software performance engineers will need to collaborate with hardware architects so that new processors present simple and compelling abstractions that make it as easy as possible to exploit the hardware.

The authors then go on to demonstrate a toy example in the area of software performance engineering. They use a simple baseline problem as an example, multiplying two 4096-by-4096 matrices. To start, they implement this in the Python code that minimizes programmer time spent. They then proceed to do performance engineering until the same function eventually runs 60,000 times faster — first by changing to more efficient coding languages, then introducing parallelization, and beyond.

At the end of the performance engineering section, they reflect on what it will take for teams of researchers and engineers to effectively take advantage of opportunities analogous to this one in the future:

Because of the accumulated bloat created by years of reductionist design during the Moore era, there are great opportunities to make programs run faster. Unfortunately, directly solving [some arbitrary, domain-specific] problem A using specialized software requires expertise both in the domain of A and in performance engineering, which makes the process more costly and risky than simply using reductions. The resulting specialized software to solve A is often more complex than the software that reduces A to B. For example, the fully optimized code in Table 1 (version 7) is more than 20 times longer than the source code for the original Python version (version 1).

The authors’s next section explores a few major successes in the history of algorithmically-driven computing performance. They use the history of algorithms for matrix multiplication as a key example to note that while the history of algorithm improvements has been a bit uneven in terms of rate of discovery, in certain cases it has rivaled Moore’s Law-level progress.

The authors highlight algorithms as another area that, while only semi-reliable, holds extreme promise for continuing to push computing progress forward. Among other reasons, they think there might be promise in this approach because many commonly used algorithms were originally designed to utilize random access memory on the serial/sequential processors of the early 1960s and 1970s.1 They believe many algorithms that take this approach use modern hardware inefficiently because they under-utilize the parallel cores and vector units in modern machines.

But, at the end of the algorithms section, the authors once again note that there is a clear difficulty in pursuing problems of this form in the modern, rather modularized research ecosystem. They write:

These approaches actually make algorithm design harder, because the designer cannot easily understand the ramifications of a design choice. In the post-Moore era, it will be essential for algorithm designers and hardware architects to work together to find simple abstractions that designers can understand and that architects can implement efficiently.

In the final section, on hardware, the authors outline the general characteristics of computing areas which they believe can continue to experience extreme performance improvements — and which will curtail. Areas that benefit from parallelism and streamlining are notable examples with a lot of potential for progress to continue. The authors use the following graphic to highlight the continued progress in SPECint rate performance — a processing performance metric which increasingly benefits from parallelism as more cores are used — as an archetypal example of a metric that they expect to continue improving at a high rate.

As designers increasingly embrace parallelism, Leiserson et al. believe “the main question will be how to streamline processors to exploit application parallelism.” In tackling this challenge, they expect two strategies to dominate. The first is that they expect designers to increasingly use simpler cores that require fewer transistors in chip designs. These simpler cores will be specifically chosen to efficiently accomplish the task at hand. For this to be worthwhile, designers must find ways to ensure two simplified cores incorporated in a specialized design can accomplish more than a single complex core.

On the second approach to tackling the challenge of streamlining processors, the authors write:

Domain specialization may be even more important than simplification. Hardware that is customized for an application domain can be much more streamlined and use many fewer transistors, enabling applications to run tens to hundreds of times faster. Perhaps the best example today is the graphics-processing unit (GPU), which contains many parallel “lanes” with streamlined processors specialized to computer graphics. GPUs deliver much more performance on graphics computations, even though their clocks are slower, because they can exploit much more parallelism. GPU logic integrated into laptop microprocessors grew from 15 to 25% of chip area in 2010 to more than 40% by 2017…Hennessy and Patterson foresee a move from general-purpose toward domain-specific architectures that run small compute-intensive kernels of larger systems for tasks such as object recognition or speech understanding.

While many interpret the belief that we have seen the end of Moore’s law as lacking in technological optimism, that does not have to be so. Hennessy and Patterson’s 2017 Turing Award Lecture on this topic was titled, “A New Golden Age for Computer Architecture.” There’s no reason this shift in approach must result in disappointment.

The following excerpt from a version of the Turing Lecture Hennessy gave at Stanford in 2019 briefly characterizes their thoughts on what the near future of semiconductor design will look like.

I think Dave [Patterson] and I both believe that the only path forward is to go to domain-specific architectures that dramatically improve the environment and the application for that particular environment in the performance that’s achievable as well as the energy consumption. Just do a few tasks, but do them extremely well. And then, of course, combination — putting together languages which help specify applications for these particular architectures and combining it with the particular architectures.

Hennessy concludes the talk by specifically emphasizing the implications of the new domain-specific design approaches on scientific team structures, saying:

This creates a new Renaissance-like opportunity — because now unlike what’s happened for many years in the computer industry. Now people who write applications need to work with people who write compiler technology and languages; they need to work with people who design hardware. So, we’re re-verticalizing the computer industry as it was before the 1980s. And it provides an opportunity for a kind of interdisciplinary work in the industry that I find really exciting. It’s sort of like this great quote, “Everything old is new again.” It’s back to the future. And I think this will provide interesting opportunities for new researchers as well as deliver incredible advances for people who use our technology.

What did the old models look like?

Hennessy has as much of a right as anybody to be excited about the re-verticalization of hardware design and the re-introduction of the old models. This “old” era is one that Hennessy understands well; it is the era in which he came up. In the following excerpt from his oral history, Hennessy describes the integrated nature of his PhD thesis — which he called “a combination of language design and compiler technology” — on a domain-specific problem at Stony Brook:

My thesis work was on real time programming…[I worked with] a guy who was a researcher at Brookhaven National Lab [that] came along with an interesting problem. He was trying to build a device that would do bone density measurements to check for long term, low level radiation exposure among people who were working at Brookhaven. What he wanted to do was build a very finely controlled x-ray scanning device that would scan an x-ray across an arm or so and get a bone density measurement. The problem is you had to control it pretty accurately because, of course, you don’t want to increase the radiation exposure to people. And micro processors at that time were extremely primitive so the real question is could we build a…programming language that could do this kind of real time control in which you could express the real time constraints. You could then use compiler technology to ensure that they were satisfied.

After joining the Stanford department in the late 1970s, Hennessy continued to partake in projects that were structured something like the one above — such as Jim Clark’s Geometry Engine project. As Hennessy mentioned, it was normal in that era. Several historically-great computing research groups that utilized this general operating structure have already been covered on FreakTakes. Early BBN and the early CMU autonomous vehicle teams are two great examples of the dynamism of this approach at work.

While BBN was a private research firm and the CMU Robotics Institute was technically an academic group, when it came to operations and incentives, the two had more in common with each other than they did with many fellow companies or academic researchers. Both were:

• Novelty-seeking

• Committed to building useful prototypes and technology for users, not doing mere “paper studies”

• Utilized firm-like management structures

These traits led each to great success in their respective fields. The following two subsections will briefly summarize the relevant operational strategies each used in their work that the new-era semiconductor ecosystem can learn from.

Early BBN

BBN had an exceptional talent for finding ways to generate revenue through contract research and research grants that allowed the firm to build academic ideas into real technologies. In many areas — such as real-time computing in the late 1960s — BBN was the go to group for projects that required both academic research toolkits and the kind of cutting-edge engineering which academic departments are rarely structured to pursue. J.C.R. Licklider’s Libraries of the Future project, BBN’s work as the prime systems integrator and IMP designer on the ARPAnet contract, and BBN building a time-sharing computer for the Massachusetts General Hospital in the early 1960s are just a few examples. BBN’s projects often simultaneously required bleeding-edge knowledge of computing, performed a practical service for a paying customer, and could result in technology that pushed the technical frontier forward.

BBN often did all of this with relatively small, vertically integrated teams of talented people. The firm was able to recruit many of MIT’s best researchers because of its unique model of both seeking novel technical problems and commitment to building practical technology. At the time, researchers who did “paper studies” and built basic prototypes at places like Lincoln Labs often grew disenchanted; they felt their work might not be built into industry products for a decade or more. But these disenchanted individuals still felt they would be wasted and bored if they joined a company like Honeywell. But BBN presented an exciting third option. With BBN, there were no such fears. The firm earned nicknames from MIT affiliates like “the third great university of Cambridge,” “the cognac of the research business,” and the true “middle ground between academia and the commercial world.”

With this reputation and the ability to recruit top talent that came with it, small teams at BBN could do big things. The BBN team primarily responsible for building the first ARPAnet IMPs and managing the first year of the project was only about eight people. Small is how ARPAnet team leader Frank Heart preferred it. As he saw it, that small, vertically integrated team of elite talent in the relevant areas was all BBN needed. In his oral history, he described the project’s management style, saying:

I mostly I tend to believe important things get done by small groups of people who all know all about the whole project. That is, in those days all the software people knew something about hardware, and all the hardware people programmed. It wasn't a group of unconnected people. It was a set of people who all knew a lot about the whole project. I consider that pretty important in anything very big. So I suppose if you call it a management style, that would be something I'd state. I think also that they were a very, very unusually talented group. I think things tend to get done best by small groups of very, very good people — if you can possibly manage that. You can't always manage it. So if you again want to call it a management style, it is to get the very, very best people and in small numbers, so they can all know what they're all doing.

The approach Frank Heart describes is not complex, but it’s also not how most universities handle similar projects. The more vertically integrated semiconductor design becomes, the more common it will become for BBN-model groups to be the ideal team structure for research projects. If veterans like John Hennessy are right and the computing industry is re-integrating and re-verticalizing, semiconductor research funders should ensure that groups with this uncommon structure can win funding to work on the research areas to which they are ideally suited.

CMU’s Early Autonomous Vehicle Teams

There are successful historical examples of universities operating in a very BBN-like fashion, as well. For example, much of MIT’s early history as an institute of technology dedicated to serving industry above all else demonstrates this point. In the early 1900s, the Institute placed such a heavy emphasis on contract research that, at one point, its applied chemistry laboratory was over 80% industry-funded. However, a much more recent example of this mode of operating is CMU’s 1980s autonomous vehicle research groups. CMU succeeded in maintaining a vertically integrated, applications-focused ethos in a period when most universities were taking a step back from this approach.

CMU’s autonomous vehicle vision research teams — and many other computing teams at the university — had a talent for operating in a way that could only be described as “firm-like.” Many grad students who worked on projects like the 1980s NavLab project were as much project employees as they were graduate students. They generated theses they could be sole authors on, but their work often had to be a part of a team effort on a single piece of integrated technology. Also, many staff were solely incentivized to build great technology, not rack up citations and build useful technology when they could find the spare time. In his FreakTakes interview, Chuck Thorpe recalled the following conversation he had with Raj Reddy about his incentive structure as he was being hired to manage the autonomous vehicle project:

Raj took me aside. He said, “I don't care how many papers you write. I don't care how many awards you win. That vehicle has to move down the road. If you do that, you're good. I'll defend you. I'll support you, I'll promote you.”

Aligning the incentives of key staff and organizing grad students in a way that enables the group to build the most ambitious, useful system it can makes sense. But this is also not how university’s tend to manage applied projects. Few universities succeed in doing things like creating tenured research tracks that are truly equal to traditional professorships. The best staff scientists and engineers I meet from places like MIT feel there is an obvious ceiling on how high they can rise. By getting many of these key incentive alignment, strategy, and team structure questions right, CMU built up a powerhouse research team that, in the 1980s, laid the base for the autonomous vehicle revolution.

The CMU team’s status as a sort of middle ground between academia and the commercial world proved to be a clear comparative advantage for them in the early days of DARPA’s first autonomous vehicle project, the Autonomous Land Vehicle (ALV) — detailed in a prior FreakTakes piece. At the end of the project’s first stage, DARPA had to fire the prime contractor, Martin Marietta — the “Martin” in Lockheed Martin. Martin had continually pressured DARPA to give them concrete, intermediate benchmarks to hit. When DARPA did just that, Martin found ways to hit the benchmarks without incorporating the best tech from DARPA’s basic vision researchers. The basic researchers also fell short of DARPA’s expectations. Many of DARPA’s SCVision researchers seemed more interested in collecting data from the machines’ cameras and sensors to plug into their academic projects. Oftentimes, their models would not work well when plugged into real-world machines. The academics knew this not because Martin was diligently testing each of their ideas, but because the CMU research team had pushed DARPA for modest funds to build their own vehicles. Using these vehicles, they tested and iterated upon cutting-edge vision models in a practical setting. Unlike most academic departments that did not want the massive systems integration and logistical headaches that came with this engineering-heavy task, CMU wanted to do it. DARPA was impressed by the productivity of the scrappy CMU team and continued to fund them even after the ALV project ended.

To the modern eye, the most striking early success from the CMU group might be the group’s neural net-based wheel-turning system. In my introduction to the Chuck Thorpe interview, I summarized the breakthrough:

In 1988, then-CMU grad student Dean Pomerleau successfully integrated the first neural net-based steering system into a vehicle — his ALVINN system. In 1995, Dean and fellow grad student Todd Jochem drove the successor to Dean’s ALVINN — the more complex RALPH — across the country on their “No Hands Across America” tour. The RALPH system employed neural nets but also put much more effort into model building and sensor processing. Holding Dean and Todd, the upgraded NavLab was able to autonomously drive 98.5% of the way on its successful cross-country journey.

According to Chuck Thorpe, there was something specific about Dean that enabled him to make this breakthrough with such limited compute. The trait Thorpe pointed out is one very relevant to this piece. According to Thorpe, what made Dean special was that he closely worked with the hardware developers at CMU and was one of the only people who understood how to get the most out of the group’s systolic array machine. Thorpe described Pomerleau’s extreme effectiveness in getting the most out of the Warp machine:

One of the supercomputers [DARPA] built was at Carnegie, and it was a thing called the Warp machine. The Warp machine had ten pipelined units, each of which could do ten million adds and multiplies at once.

H. T. Kung was working on systolic computing, where things just sort of chunk through the pipeline. And we said, “Here's this big box. Because we're sponsored by Strategic Computing, we'll put it on the NavLab. Can anybody figure out what to do with it?” “Well, it's a really complicated thing and really hard to process.” But Dean could figure out what to do with it. Partly because Dean is a really, really smart guy.

Partly because the inner structure of a neural net is, “Multiply, add, multiply, add.” You take these inputs, you multiply them by a weight, you add them together, you sum it up, run it through your sigmoid function. So, Dean was, as far as I know, the only person…well, that's maybe a bit of an exaggeration. Some of the speech people were able to use the Warp also, because they were doing similar kinds of things…But Dean got the full hundred megaflops of processing out of the Warp machine! And then he put it onto the NavLab and we got the whole thing to work.

It was not random that CMU was working on a systolic array computer. DARPA had contracted CMU to build the systolic array machines with computer vision as an early use case in mind. DARPA hoped that the two CMU projects would prove to be complementary. While the Warp team and the autonomous vehicle teams were not working together as one at this point, Pomerleau’s hard work ensured that they complemented each other to world-changing effect.

As Hennessy pointed out in his Turing Lecture at Stanford, Google’s now-famous TPUs with systolic array processors — optimized for machine learning applications — are a key example of the coming trend in domain-specialized hardware. Research groups that operate like BBNs ARPAnet team or CMU’s early autonomous vehicle teams seem ideally suited to help build this future.2

Everything old is new again

I am not qualified to make predictions on the future of Moore’s Law or what percentage of R&D budgets should be allocated to domain-specific semiconductor design teams. But if what people like Hennessy, Patterson, and Leiserson et al. are saying is even partially true, more research organizations set up like the best ones from the past are needed. The early operating structures of BBN and the CMU Robotics Institute, two underrated giants of the old computing community, are ideal groups to use as models.

As I understand it, Groq’s approach is rather similar to what I’ve described. And it’s phenomenal to see the progress they’ve made! But the vast majority of potential use cases for this org structure might be in market areas that are too small to raise VC rounds or for ideas considered too speculative. These are ideal areas in which to deploy CHIPS Act funding to seed BBN-like organizations and research groups. Whether it be seeding groups do practical work on the most speculative ideas coming out of academic labs or building in areas that are more important to the US than they are to VCs, BBN-model orgs have to chance to connect many pieces of the CHIPS contractor ecosystem that currently do not fit together well — just as DARPA’s contractors on the Autonomous Land Vehicle project did not fit together without CMU.

With the ramp-up in government CHIPS Act funding — endowing groups like the NSTC with up to $11 billion that has not been spent yet — now is the time to ensure groups are built up that can utilize this money as effectively as possible. One straightforward way to do this is for:

• Groups like the NSTC and the DOD Microelectronics Commons to decide and make it known that they are willing to wholly fund Day 0 orgs of 5-20 talented engineers, researchers, and systems engineers organized in BBN-model project teams as contractors.

• Alternative approaches that are similar in spirit but with key logistical differences are also possible. For example:

  • Attaching similar groups to pre-existing government contractors or universities.

  • Having NSTC program managers scout a domain area and assemble these teams to best fill acute needs in their portfolios.

  • Etc.

If any relevant parties are reading this, please reach out and I’d be happy to discuss alternative approaches and bureaucratic details. What is important is that research groups like the ones I’ve described find a way to get funding somehow.

On the university end, it would be great if institutions could quickly find ways to establish homes for research groups like this, with incentivizes and team structures that align with the task at hand. But given the difficulty of making decisions within universities that upset the natural order of things, this can often be slow and requires tough “consensus building.” So it’s maybe not worth holding our breath on that.

In the post-war period, it was common for government R&D funders to be the first — and sometimes only — major funding into new organizations of talented individuals. If they felt the team could do work that was good for the country, they took the risk. This was a common path to starting technical firms for grad students and professors. The market sizes were often modest and the teams often didn’t raise VC money. But the best of MIT, CMU, or the University of Illinois could simply set up shop as a firm and solve whatever problems the government needed them to at a fair market rate. With this recent, strong push to make things happen in the American semiconductor ecosystem, talented groups might have the chance to do something like that again.

Thanks for reading:)

Also, while doing background research for this piece and trying to make sense of the technical and management trends in different eras, I found the writing of Hassan Khan exceptionally helpful. I encourage any FreakTakes readers to check out his blog here.

Subscribe now

Share

If you would like to read more, check out some of the following FreakTakes pieces on related topics:

• “The Third University of Cambridge”: BBN and the Development of the ARPAnet

• An Interview with Chuck Thorpe on CMU: Operating an autonomous vehicle research powerhouse

• ILLIAC IV and the Connection Machine

• MOSIS: The 1980s DARPA 'Silicon Broker'

• Managing Lockheed’s Skunk Works

One Last Announcement: Interested readers should check out this policy opportunity FAS sent over. “The Federation of American Scientists is looking for federal policy ideas to tackle critical science and technology issues. Areas of interest include R&D and Innovation, Emerging Technologies, Artificial Intelligence, Energy and Environment, Federal Capacity, Global Security, and more. Learn more and submit an idea here by July 15.”

Subscribe now

What the Skunk Works does is secret. How it does it is not. I have been trying to convince others to use our principles and practices for years. The basic concept as well as specific rules have been provided many times. Very seldom has the formula been followed. — Clarence “Kelly” Johnson

Introduction

This piece will round out a trio of pieces covering exceptional contractors from ARPA’s early decades. The first two pieces covered BBN and CMU’s autonomous vehicle teams. The former was a research firm unusually focused on novelty; the latter was a group of university researchers unusually focused on shipping real technology and incorporating firm-like management structures. Unlike those organizations, Skunk Works appears at first glance more similar to a traditional defense contractor than a research lab. But the novelty of its outputs demonstrates that Skunk Works is an all-time great R&D lab in its own right. Skunk Works got much more funding from the armed forces and CIA than it did ARPA. But it still deserves to be covered in the ARPA playbook because, to many, the name “Skunk Works” has become synonymous with the topic of defense innovation.

Today, many view a big bureaucracy like a defense prime as a terrible place to build ambitiously. This is more true today than it was in the 1950s, but it was still often true back then. Yet, Kelly Johnson was able to build an applied research powerhouse within Lockheed that designed arguably the three most impressive spy planes and stealth bombers in American history. To top it off, many of Skunk Works’ novel planes were built on budget and in only 180 days. This was all made possible by Johnson’s clear vision of what was important when running a lab and the trust Lockheed management and DC project funders had in him.

But before I detail how Kelly built Skunk Works, I must first dive into how Kelly Johnson earned his stripes as a young aircraft engineer. This is key to understanding his management of the Skunk Works. In many ways, his management practices sought to maintain an environment similar to that of the young aviation industry he came up in.

The Education of Kelly Johnson

To many decades-long Skunk Works employees, the essence of the lab was inseparable from its founder and boss: Kelly Johnson. So, to begin to understand the lab and how it operated, we first need to understand Johnson and his background.

As a boy, Kelly came to the decision that he wanted to learn how to build airplanes. He read every book on the only two-decade-old subject he could get his hands on at the local Carnegie Library. When he got to the University of Michigan, he quickly found a job helping a professor operate the school’s wind tunnel. The professors at the University of Michigan, like many engineering schools at the time, did much of their research on industry contracts. This gave Kelly the opportunity to help improve real vehicles for paying customers. In his 1985 autobiography, More Than My Share of It All, Kelly describes several contracts he was working on with a Professor Pawlowski, writing:

He, like some of the other professors, had contracts outside the college. In the wind tunnel, I worked for him on design of the Union Pacific streamlined train, on a smoke-removal project for the city of Chicago, and on one of the very early proposals for generating energy with a wind machine.

In addition to assisting professors with their projects, Kelly asked for permission to take on his own contracts. They agreed and said that he could use the wind tunnel when it was idle. He simply had to pay the university an hourly fee to rent it. The first customer he sought out was the Studebaker Motor Company. Kelly was confident he could help the company. The wind tunnel was a relatively young scientific instrument. Many companies did not have one of their own. He described his attitude going into the contract, writing:

It was obvious that the wind tunnel could be very useful in designing streamlined automobiles. We got an assignment to test the Pierce Silver Arrow, which was to become one of the early “totally-streamlined” cars. We knew all the tricks on how to reduce drag caused by air resistance. We found, for instance, that the big ugly headlamps on Studebaker cars were eating up 16 percent of the power the engine developed at 65 miles an hour. We managed to get them shaped into the fenders. We worked on a lot of other problems and ran many, many tests.

In 1932, Johnson graduated with his bachelor’s degree. It was a horrible job market, but he and a friend borrowed a professor’s car and drove to the West Coast to introduce themselves to aircraft companies to see if they had any work available. One of the companies was a small outfit named Lockheed which had recently been purchased for $40,000 (~$850,000 today). They told Kelly that they did not have a job for him, but that if he came back next year they might have something for him. So, he returned to Michigan and enrolled in a master's program. In this period, his consulting and research assistant projects became even more impressive. The outputs of one of his personal contracting endeavors even made a splash in the local newspaper:

Five of the qualifying cars which will race at Indianapolis Memorial Day have bodies designed by two university graduate students, C.L. Johnson and E.D. Palmer. All of the cars are semi-stock Studebakers and all qualified for the race at speeds ranging between 110 and 116 miles an hour.

Kelly referred to the experience earned on these contracts as “a liberal education for me in the practical application of aerodynamic theory.”

During this year, one of the airplane models sent to Kelly’s professor for wind tunnel testing came from Lockheed. The model, which would become the Lockheed Electra, was in a new business area for the small firm. Board Chairman Robert Gross had decided that the recently re-organized company’s future was no longer in single-engine wooden aircraft, but in twin-engine all-metal designs. These designs could carry more passengers. At the end of testing, Kelly and his professor markedly disagreed on the readiness of Lockheed’s model. Johnson noted:

It developed some very serious problems, I thought, from what I then knew of aerodynamics. It had very bad longitudinal stability and directional-control problems. But most aircraft of that day had similar failings. Professor Stalker, in consultation with Lloyd Stearman, already a recognized top-notch designer at age 33 and first president of the company, decided the figures were acceptable.

The report was sent to Lockheed without Kelly’s opinions included.

After completing his additional year at Michigan, Lockheed hired Johnson to work under Chief Engineer Hall Hibbard — a young man himself, only seven years older than Johnson. Kelly was going to be making far less than he was running his consulting projects at Michigan. Johnson recounted his initial hiring arrangement, writing, “I was to receive $83 a month (~$2,000 today) to start in tool design until they could assign me as an engineer. There were five engineers at the time.” He may have received a small pay bump when he got his title changed to engineer. But this was not about the money for Johnson. This was about doing what he always wanted to do: designing new planes. He reflected on why leaving his lucrative wind tunnel practice and joining Lockheed was an easy decision, writing:

At the university we certainly weren’t going to design aircraft — and that was my goal. But I didn’t make that much money again until ten years later.

Johnson was outspoken from his first moment at the company. He was not one to sit quietly if he thought mistakes were being made. Johnson recalled his first days at the company, writing:

Practically the first thing I told Chappellet and Hibbard was that their plane was unstable and that I did not agree with the university’s wind-tunnel report.

Hibbard was not insulted. He was, instead, quite open-minded and would not forget that Johnson had said this — even if he did not scramble to fix the problem at that moment. Hibbard recounted years later in an interview that he had wanted to get some “new young blood…fresh out of school with newer ideas” in his engineering department. Kelly Johnson proved to be that and more.

But, initially, the young Johnson was assigned to work in the tooling department. There, he received a kind of hands-on education that he felt most airplane designers couldn’t find anywhere in the post-war years. He took quite well to the hands-on nature of the work. He had, after all, spent much of his youth working in construction. In the tooling department, he worked with an old hand named Bill Mylan and learned the airplane production business from the workman’s end of things. While in the tooling department, Kelly learned to actually build planes, designed a furnace to heat-treat a metal that was new to Lockheed’s production, and made beginner’s mistakes passing design instructions to workmen. In no small part, Kelly credited these early experiences with helping him understand how to design airplanes that were buildable.

Johnson’s Growing Reputation

After a few months of tooling work, Hibbard called Kelly into his office to give him a new assignment. Hibbard was going to put the young would-be engineer’s ideas to the test. The assignment would prove that Johnson’s wind tunnel days had left him with a remarkably deep intuition for aerodynamics. Johnson recalls the meeting with Hibbard and getting sent back to his old wind tunnel, writing:

“Kelly, you’ve criticized this wind-tunnel report on the Electra signed by two very knowledgeable people. Why don’t you go back and see if you can do any better with the airplane?”

Hibbard sent me back to the University of Michigan wind tunnel with the Electra model in the back of my car. It took 72 tunnel runs before I found the answer to the problem…It was a process of evolution. On the seventy-second test, I came up with the idea of putting controllable plates on the horizontal tail to increase its effectiveness and get more directional stability. That worked very well, particularly when we removed the wing fillets, or fairings onto the fuselage — put on apparently because they were coming into style and being used successfully on such airplanes as the Douglas DC-1. And we avoided the trouble others had with them when not used properly…We then added a double vertical tail because the single rudder did not provide enough control if one engine went out. That was so effective we removed the main center tail. And there you had the final design of the Electra. The distinctive twin tails on all of the early Lockheed metal airplanes, and the triple tail of the familiar Constellation airliner of the mid-’40s and ’50s, were the result of these tunnel tests.

Hibbard was impressed and sent Johnson a note telling him how great of a job he had done. Johnson very proudly noted that:

When I returned to the plant. I was a full-fledged member of the engineering department. I was number six.

As a member of the small engineering department, he did crucial work on all parts of the Electra. In the process of learning to wear so many different hats for the Electra and other projects, Kelly felt that he became an “honest-to-god aircraft engineer.”

I worked not only on the aerodynamics of the airplane, but on stress analysis, weight and balance, anything and everything they threw at me. And, of course, more wind-tunnel testing. From that, I became the logical choice to be flight test engineer on the airplane when it was ready to fly.

Kelly noted that the Electra, with its iconic twin-tail, became the fastest multi-engine transport in the world at the time, with cruising speeds above 200 miles per hour. Lockheed delivered the first aircraft in mid-1934. At the time, the small company had six engineers and about 200 factory crew.

Over the next few years, Johnson would do all sorts of work on several derivatives of the Electra. For one of these planes, the Model 14, Kelly ran extensive wind tunnel tests to help design a new airplane part called wing flaps. These flaps allowed the wing to remain large during takeoff to aid in control and then become smaller in flight and increase speed. This Lockheed-Fowler Flap earned Johnson the Lawrence Sperry Award — the first of countless aviation awards he would win in his storied career.

But this achievement was nothing compared to Kelly’s 1938 efforts. With World War II ramping up, the British knew they needed an anti-submarine patrol plane. Getting word of this, Lockheed quickly threw together a proposal for an anti-submarine version of their new Model 14 to show the British. The British representatives were so impressed at how quickly the small company incorporated the recommendations from the group’s first meeting — in a single weekend — that they invited Lockheed to meet with their technical staff in England. Lockheed sent a four-person team on the trip. Kelly was the sole representative for the engineering team.

In the first meeting in England, the British requested several adjustments that required an almost complete re-design by the Lockheed team. Upon leaving the meeting, the team acquired a drawing board and drafting equipment. Kelly then proceeded to lock himself in a hotel room over the three-day weekend. Johnson recalled the event, writing:

I had to fit in all this new equipment, re-arrange copilot and radio operator positions, make weight and structural analysis, figure contract pricing, and guarantee that the design would meet certain performance requirements.

It was a three-day holiday weekend — Whit Sunday, Whit Monday. I worked a solid 72 hours on this redesign — not taking time for sleep, just catnapping briefly when absolutely necessary. I was a rumpled figure…When finally I fell into bed for some very sound sleep — in the room I shared with Courtlandt to save on expenses — it was the first time I had removed my clothes in 72 hours.

Johnson’s efforts were well worth it. The team at the Air Ministry was extremely impressed. Amid an ongoing war, Lockheed’s ability to work quickly was reassuring. Lockheed closed the deal in England that week. Before giving Lockheed a contract to build at least 200 these Hudson airplanes, the British only had one point of concern for Lockheed. The impressive designer who re-modeled the plane over the weekend was extremely young — Johnson was 28. In fact, the entire Lockheed team was young. The leader of the attaché, Courtlandt Gross, was only 36. Gross recounted Air Marshal Sir Arthur Varnay expressing his one concern as follows:

Mr. Gross, we like your proposal very much, and we very much like to deal with Lockheed. On the other hand, you must understand that we’re very unused in this country to dealing — particularly on transactions of such magnitude — on the technical say-so of a man as young as Mr. Johnson. And, therefore, I’ll have to have your assurance, and guarantee, in fact, that if we do go forward, the aircraft resulting from the purchase will in every way live up to Mr. Johnson’s specifications.”

Gross assured Varnay that he and his brother, Lockheed’s Chairman, had every confidence in the young Johnson. With this assurance, a deal was struck. It was, according to Kelly, “the largest aircraft production order placed up to that time in the United States.”

Not long after to returning from the trip, Johnson was named Lockheed’s Chief Research Engineer. The young man now had a big title and a fast-growing reputation.

Skunk Works is Born

As the war progressed and Lockheed’s production lines rolled off Hudson’s vital to the war effort, the firm found its way into more and more contracts. Before the US entered the war, Lockheed had proposed building a prototype jet engine-powered plane for the Army Air Corps. The Air Corps was initially cool on the idea, but eventually agreed as they saw the British working on a similar aircraft. Kelly agreed to produce an experimental model of the proposed aircraft in a timeline that would become quite standard for his Skunk Works crew: 180 days.

The Army was quick to get Kelly his contract. Kelly recalled how quickly General Frank Carrol went from deciding this was a good idea to producing a contract, writing:

“We’ll give you a contract for the airplane, Kelly, and for Nate’s engine as well,” said Gen. Frank Carrol, commanding officer of Wright Field. “But you’ll have to use the British engine in the first airplane because we need it — and all the jet fighters you can build — as soon as possible to use against the Me-262. Your new engine couldn’t possibly be ready for service in time.”

Since I had promised to build a jet airplane within 180 days, I asked, “When will we get a contract? When will the time start?” “You will have a Letter of Intent this afternoon by 1:30 p.m.” he replied. “There is a plane leaving Dayton for Burbank at two o’clock. Your time starts then.”

And it did. The date was June 8, 1943. Gen. H. H. “Hap” Arnold, himself, had approved the contract.

Upon returning to Lockheed’s Burbank headquarters with the good news, Robert Gross was quick to tell Kelly that Lockheed had few people, few machines, and almost no room for such a project. They were already up to their eyeballs producing around 28 airplanes per day — P-38s, B-17s, and Hudsons, and others. Gross was also quick to tell Kelly that he thought nothing would come of the contract. But none of that meant Johnson wouldn’t be allowed to pursue the project. Kelly wrote:

But he [Gross] and Hibbard always were open to new ideas and backed me in many critical times…“You brought this on yourself, Kelly,” Gross stated. “Go ahead and do it. But you’ve got to rake up your own engineering department and your own production people and figure out where to put this project.”

This was an opportunity much bigger than just this one contract for Johnson. The Lockheed company had been rapidly expanding throughout his tenure there, and he felt this was making the design of new, interesting planes more difficult. He reflected on these difficulties, writing:

For some time I had been pestering Gross and Hibbard to let me set up an experimental department where the designers and shop artisans could work together closely in development of airplanes without the delays and complications of intermediate departments to handle administration, purchasing, and all the other support functions. I wanted a direct relationship between design engineer and mechanic and manufacturing. I decided to handle this new project just that way.

Upon receiving Gross’ green light, Kelly set up some crates adjacent to the new Lockheed wind tunnel and found a circus tent to put over the top of them. With that, the Skunk Works was born.

To acquire more tools, Kelly got permission to buy out a small local machine shop. He scrounged together 22 other engineers from around the company who he had worked with before and trusted. Johnson had high standards, and he believed they were all very good. He also set the group up with its own purchasing department. To Kelly, it was vital to Skunk Works’ success that “every function…needed to operate independently of the main plant.”

The Air Corps moved almost as fast as Johnson did. Nine days after the project was green-lit, Skunk Works hosted colonel and a major at Skunk Works for a mockup conference that went smoothly.

Many complications arose in completing the project. Kelly emphasized that “it was a complete new world of flying and testing” with these jet-engine aircraft. The plane ran into the same compressibility problem when it approached Mach 1 that Lockheed had encountered with its P-38 development. The team was able to overcome the airplane’s aileron buzzing by installing shock absorbers in just the right fashion. In addition, common problems present in new aircraft designs popped up. Certain parts of the plane proved precarious to weld together. Extensive re-designing and vacuum testing were required to overcome these issues. But whatever problems came up, the small team of particularly talented engineers and workmen worked through them quickly. The small operation seemed to be as effective at producing experimental aircraft as Kelly had hoped.

And the Air Corps was also an excellent project partner. This was vital, in Kelly’s estimation. He noted that:

There were only six people from the military involved and two or three of us from Lockheed. We had approval to proceed that night. Six days later we had our government furnished equipment — guns, radio, wheels and tires, etc. At every stage of the work, we had excellent cooperation from Wright Field and the officers involved with the project. The job could not have been completed on such a tight schedule without it.

Skunk Works delivered the XP-80 plane to the Air Corps on Day 143 of the contract — more than a month ahead of schedule. The XP-80 took its first flight on January 8, 1944. Almost immediately, the Air Corps gave Skunk Works another contract to produce a modified version of the plane with an airframe almost twice as large — to make use of GE’s new engines. The two copies of the aircraft the Air Corps requested were delivered 132 days later. This design, the YP-80, was the precursor to the famous P-80 and two-seat T-33 training vehicle. According to Kelly, there went on to be more than 6,000 built in total.

The plane would not go on to see major use in World War II given the ending of the conflict. But it would later prove itself — first in the North Korean vs. South Korean conflict in 1950 — and also set several records. The most notable of which is probably the F-80 flying the route from Long Beach to LaGuardia in a record 4 hours and 13 minutes, averaging 584 miles per hour.

These first Skunk Works projects were a smashing success. Producing a prototype of a truly novel aircraft will never be completely smooth. But with these projects the team at Skunk Works was beginning to make it clear that they could routinely overcome these obstacles on tight schedules, even if not easily.

The First Iconic Skunk Works Plane: the U-2 “Spy Plane”

In 1953 the group began work on their most ambitious airplane yet. Around this time, defense contractors were made aware of the country’s desperate need for aircraft that could fly recon missions over the USSR to acquire information on Soviet military capabilities. They needed equipment that could do this without being shot down by the Soviet’s advanced anti-aircraft weapons. Always game for a challenge, the Skunk Works put together a proposal for a plane that could fly approximately 70,000 feet, have a range of 4,000 miles, and would be a steady platform for advanced photographic equipment. Without a plane that could reach that height, the CIA and Air Force would have to resort to low-flying, high-casualty tactics to acquire information1

The Air Force initially did not think Skunk Works’ proposal was realistic. Other proposals to achieve this altitude were somewhat primitive technology like weather balloons.2 But the folks in DC decided to let the Skunk Works go ahead with the project anyway. Many in DC had exceptional faith in Kelly Johnson, just as the Gross brothers and Hall Hibbard did back in Burbank. Johnson describes the events that led to Skunk Works winning the contract as follows:

Our first presentation was to the Air Force, where it was turned down as too optimistic. They questioned that any engine even would operate at the altitude we were proposing. They were correct in that there was not proof at that time that this was possible. And the Air Force already had an airplane in development with the Martin company — an airplane with two engines which was preferred to our single-engine design.

But our proposal reached Trevor Gardner, then Assistant Secretary of the Air Force for Research and Development, and a brilliant engineer in his own right. Late in 1953, he invited me to come to Washington to discuss it. He had assembled a committee of scientists and engineers, and for three days they put me through a grilling as I had not had since college exams. They covered every phase of the aircraft design and performance — stability, control, power plants, fuels — everything.

Later I met and lunched with Air Force Secretary Harold Talbott, CIA Director Allen Dulles, and his right-hand man, Larry Houston, among a distinguished group. When I was asked why I thought Lockheed could do what I proposed — build 20 airplanes with spares for roughly $22 million and have the first one flying within eight months, Gen. Donald Putt graciously volunteered, “He has proven it three times already—on the F-80, F-80A, and F-104.”

The CIA took the lead in funding the project and Richard Bissel — Special Assistant to CIA Director Dulles and a fixture of post-war CIA leadership — became Skunk Works’ project liaison.

Kelly selected 24 other engineers to work on the project with him. The total project staff would eventually grow to a headcount of 81. Given the secrecy of the project, Skunk Works was not allowed to fly the plane at its Burbank headquarters. So, Kelly sent a test pilot around to find a secure location on which to build a runway and basic testing facilities. He named the remote strip of land that the pilot found in the Nevada Dessert “Paradise Ranch.” Today it is known as Area 51. Once selected, Lockheed hired contractors to begin constructing some roads, hangars, offices, living accommodations, etc. on the land.3 All of this was done while Skunk Works was working on the novel aircraft.

Skunk Works had to keep an extreme focus on ruthlessly minimizing weight throughout the design and construction of the aircraft. Many staple building materials had to be made thinner or replaced altogether. The entire team was constantly scouring for lighter materials and designs for each part of the aircraft. Each would have to be extensively tested and modified until the engineers were comfortable using it. All of this led to a quite interesting aircraft. Ben Rich explains how strange the aircraft seemed to other pilots and engineers, writing:

We designed and built that airplane for lightness. The wings, for example, weighed only four pounds per square foot, one-third the weight of conventional jet aircraft wings. For taxiing and takeoffs, jettisonable twin-wheeled “pogos” were fitted beneath the enormous fuel-loaded wings and kept them from sagging onto the runway while taking off. The pogos dropped away as the U-2 became airborne.

The fuselage was fifty feet long, built of wafer-thin aluminum. One day on the assembly floor, I saw a worker accidentally bang his toolbox against the airplane and cause a four-inch dent! We looked at each other and shared the same unspoken thought: was this airplane too damned fragile to fly?…Pilots were scared to death flying those big flapping wings into bad weather situations — afraid the wings would snap off…Adding to the sense of the airplane’s fragility was that the razor-thin tail would be attached to the fuselage by just three five-eighth-inch bolts.

(The plane and its enormous, drooping wings that almost sagged onto the runway, proved more resilient than the pilots or designers ever imagined.)

The engine was also a massive obstacle. This was partially a problem for the Skunk Works team but primarily fell to a long-time Skunk Works contractor. Rich described the difficulty in building an engine to operate at these high altitudes, writing:

My principal work was on the engine’s air intake, which had to be designed and constructed with absolute precision to maximize delivery of the thin-altitude air into the compressor face. Up where the U-2 aimed to cruise, just south of the Pearly Gates, the air was so thin that an oxygen molecule was about as precious as a raindrop on the Mojave desert. So the intakes had to be extremely efficient to suck in the maximum amount of oxygen-starved air for compression and burning.

In the end, the engine problem was overcome by the work of Skunk Works and two very impressive contractors. The most important contributor was probably Pratt & Whitney, a long-time Skunk Works partner. The P&W team modified an existing high-altitude engine they made that was considered top-of-the-line. The head of Pratt & Whitney himself put his best people on the job and they found a way to modify the engine so it could function in the extreme conditions. A major contribution was also made by Shell’s research team. Kelly Johnson got Shell’s R&D team to develop “a special low-vapor kerosene for high altitudes” that would not boil or freeze in the extreme conditions in which the U-2 operated. In many Skunk Works projects, key contributions from trusted contractors were vital.

The U-2 was completed and shipped to Paradise Ranch in late July — less than nine months after the contract was first issued. In early August, it accidentally took flight for the first time while conducting a routine taxi test. As test pilot Tony LeVier recalled, “Who’d have guessed an airplane could take off only going 70 knots? That’s how light it was.” The plane took its first official test flight a week later. The high-flying plane was a marvel, “able to glide 250 miles from 70,000 feet.” Within nine months of the first test flight, the CIA U-2s became operational.

On top of the U-2s achieving Kelly’s lofty technical goals and obtaining high-priority intelligence on the USSR, many consider this contract one of the greatest bargains in defense procurement history. Johnson and the Skunk Works came in far under budget. They returned $2 million of the original $22 million budget and were able to construct six extra airplanes from the spare parts the team didn’t need. Rich said they were able to build with these spares simply “because the U-2 functioned so well.”

What did Kelly Johnson think was Skunk Works’ special sauce?

Skunk Works’ success on the U-2 was not an isolated event. The Skunk Works would also go on to produce two more planes considered all-time greats — the Blackbird models and the F-117 stealth bomber — in addition to dozens of other successful planes. This, of course, raises the question: how did Skunk Works staff build so many novel planes on such a routine basis? Did they have a kind of special sauce?

Kelly Johnson believed they did. Kelly famously had a list of 14 rules he used to run the Skunk Works. However, upon reading his biography, it becomes clear that the beliefs and experiences that led to those rules could be distilled down to something like three key principles. The three principles are, roughly:.

1. Reduce the bureaucracy to almost zero. Ideally, one person should have almost complete authority over day-to-day decision-making.

2. Keep the team ruthlessly small.

3. Whenever possible, only take on contracts where there is enough mutual trust with funders and subcontractors to work with them with a minimum amount of bureaucracy. If funder decisions cannot be made swiftly, the project is probably not worth pursuing.

The three beliefs above can be seen at work all over the Skunk Works operation. The following subsections will give the reader a clear idea of how these beliefs manifested themselves in the practical management of the Skunk Works.

Luckily, in More than My Share of It All, Kelly Johnson dedicates an entire chapter to explaining the simplicity of his special sauce. In the chapter, he makes it clear that he can’t believe how few true imitators Skunk Works had — given that so many companies claimed to have their own Skunk Works. Kelly opens the chapter, a bit incredulously, writing:

What the Skunk Works does is secret. How it does it is not. I have been trying to convince others to use our principles and practices for years. The basic concept as well as specific rules have been provided many times. Very seldom has the formula been followed.

As this section will detail, the lack of true Skunk Works imitators will not be shocking to any who are familiar with bureaucracies. The key management principles that helped Skunk Works thrive are simply non-starters in most bureaucracies. This is precisely why so many of Kelly’s rules-of-thumb are obsessed with warding off bureaucratic processes; this is also why Ben Rich lived in such fear of bureaucracies smothering Kelly’s great band of engineers from all sides when he took over the Skunk Works.

Principle #1: Minimize Bureaucracy, Centralize Power

For a variety of reasons, Kelly believed bureaucracies killed progress in designing and building new aircraft. Kelly believed one person should have complete control over day-to-day decision-making. An engineer should be able to come to him with a problem, and he needed to have the authority to tell them “yes” or “no” on the spot.

This type of approach may not have been as rare in his youth, but as his career wore on it became almost extinct. He lived to see the onset of slow decision-making processes begin to expand in the industry. Lamenting this, he writes:

I fear that the way I like to design and build airplanes one day may no longer be possible. It may be impossible even for the Skunk Works to operate according to its proven rules at some point in the future. I see the strong authority that is absolutely essential to this kind of operation slowly being eroded by committee and conference control from within and without. The ability to make immediate decisions and put them into rapid effect is basic to our successful operation.

In the 1950s, even the comparatively small Lockheed had grown bureaucratic enough to stifle ambitious work. Before being recruited to the Skunk Works in the late 1950s, Ben Rich worked in the main portion of the Lockheed company. He said the following about his time there and how the company operated:

Lockheed was very regimented and bureaucratic, and by my fourth year on the payroll I felt stymied and creatively frustrated. I had a wife and a new baby son to support, and my father-in-law, who admired my moxie, was pushing me to take over his bakery-delicatessen, which earned the family a very comfortable living.

The two decades between Kelly Johnson and Ben Rich joining the company had seen Lockheed cease to be the kind of place where an exceptionally driven individual or a genius had the chance to spread their wings. Hall Hibbard used to say that Kelly Johnson could “see air.” The then-Chief Engineer kept assigning the young Johnson more and more responsibility until, by the time he was made Chief Research Engineer at 28, he’d been largely responsible for many key aspects of Lockheed’s designs anyway. Rich — who was eventually Kelly’s hand-picked protegé — on the other hand, was considering leaving Lockheed to run a deli!

Johnson wrote the management chapter in his book hoping that individuals would find it who were brave enough to run a truly ambitious R&D operation. While the aircraft industry did not take his advice, readers of this Substack would be wise to. The same reasons that large organizations resisted properly copying Skunk Works back then are similar to the excuses they would use today — many aspects of bureaucracies and their emergent behavior are evergreen. Johnson outlined the primary reason truly copying Skunk Works was a non-starter in most companies, writing:

Most companies, while desiring the benefits, will not pay the price in revised methods and procedures for setting up a Skunk Works-type of operation. They will not delegate the authority to one individual, as Lockheed did in my case from the very first Skunk Works. It requires management confidence and considerable courage. Without the authority assigned to the Skunk Works by our military customers and the Lockheed corporation, we would not have been able to accomplish many of the things we have done, things about which I felt we could take a risk — and did.

Skunk Works may never have been green-lit in the first place had Hibbard and Gross not had an exceptional level of trust in Kelly Johnson. As Rich recalled, there was a general sentiment at Skunk Works when he joined in the late 1950s: “The open secret in our company was that the chief engineer [Kelly] walked on water in the adoring eyes of CEO Robert Gross.” And Johnson took full advantage of his well-earned long leash.

Kelly set up Skunk Works with a belief in the supreme power of individual visions. He did not believe that committees would design broken planes. But he did believe that they would produce uninspired and unoriginal planes. And there was absolutely no room for processes like that when the goal of his team was to design truly novel, experimental aircraft. He reflected on the impacts of bureaucracy on airplanes themselves, writing:

There is a tendency today, which I hate to see, toward design by committee — reviews and recommendations, conferences and consultations, by those not directly doing the job. Nothing very stupid will result, but nothing brilliant either. And it’s in the brilliant concept that a major advance is achieved.

Development of some of this country’s most spectacular projects — the atom bomb, the Sidewinder missile, the nuclear-powered submarine — all were accomplished by methods other than the conventional way of doing business outside the system.

Johnson’s efforts to minimize bureaucracy and belief in individual visions allowed Skunk Works to both produce ambitious plane designs and build them on short timelines. Kelly recruited some of the best Lockheed had to offer. He implemented a system where a small number of simple drawings were used and things could be changed on the fly. He put all the engineers, designers, and workmen in close quarters. And he did everything he could to enable them to test ideas as fast as they had them.

A genius would feel challenged by the environment and Johnson’s no-bullshit technical grillings, not stifled by bureaucracy. In his book, Ben Rich referred to a 24-year-old aerodynamicist working on a project with him, the late Dave Campbell, as “destined for greatness.” Skunk Works was an environment where those qualities could both show and be taken advantage of. The primary portion of Lockheed was no longer like that. But in Skunk Works, Kelly kept a culture alive similar to the company he first arrived at in the 1930s. In that young Lockheed company, with only five engineers, Kelly was not just an aerodynamicist, but an aerodynamicist, stress analyst, weight/balance engineer, and flight test engineer all in one. He was challenged and largely unencumbered by rules and processes. In that role, his genius could truly be put to work.

When Kelly Johnson was at the helm of the Skunk Works, the individual visions overriding the projects were often his own. Of course, during his reign there was plenty of room for Skunk Works staff to demonstrate their genius on their own components of the plane, helping others problem solve, or via influencing Johnson with their ideas. Johnson even kept quarters on him to dole out if anyone ever proved him wrong. However, the fact of the matter is that in his decades-long run as the lab’s leader, he lost very few quarters. Rich said the following on Johnson’s wide-ranging, freakish understanding of aircraft design:

Nothing got by the boss. Nothing. And that was my sharpest impression of him, one that never changed over the years: I had never known anyone so expert at every aspect of airplane design and building. He was a great structures man, a great designer, a great aerodynamicist, a great weights man. He was so sharp and instinctive that he often took my breath away. I’d say to him, “Kelly, the shock wave coming off this spike will hit the tail.” He would nod. “Yeah, the temperature there will be six hundred degrees.” I’d go back to my desk and spend two hours with a calculator and come up with a figure of 614 degrees. Truly amazing. Or, I’d remark, “Kelly, the structure load here will be…” And he would interrupt and say, “About six point two p.s.i.” And I’d go back and do some complicated drudge work and half an hour later reach a figure of 6.3.

With someone like Kelly around, it makes sense that his vision was the individual vision elevated to the fore. But the lab’s success was not dependent on his vision. Arguably the lab’s greatest military contribution, the F-117 stealth bomber, came in its first project after Johnson retired.

Kelly picked Ben Rich as his successor because he thought Ben was innovative, practical, and an absolute terrier in working through some of Skunk Works’ most difficult problems. Those traits were what was essential. But Rich knew he was no visionary like Kelly Johnson. Rich’s stealth project was extremely unique and gutsy. This was the case because he elevated the ambitious vision of a single individual to the fore of the project. But it was not his vision. He describes the project’s inception, writing:

I wish I could claim to have had a sudden two a.m. revelation that made me bolt upright in bed and shout “Eureka!”…The truth is that an exceptional thirty-six-year-old Skunk Works mathematician and radar specialist named Denys Overholser decided to drop by my office one April afternoon and presented me with the Rosetta Stone breakthrough for stealth technology…

Denys had discovered this nugget deep inside a long, dense technical paper on radar written by one of Russia’s leading experts and published in Moscow nine years earlier…As Denys admitted, the paper was so obtuse and impenetrable that only a nerd’s nerd would have waded through it all…As he explained it, [Pyotr] Ufimtsev had revisited a century-old set of formulas derived by Scottish physicist James Clerk Maxwell and later refined by the German electromagnetics expert Arnold Johannes Sommerfeld. These calculations predicted the manner in which a given geometric configuration would reflect electromagnetic radiation. Ufimtsev had taken this early work a step further. “Ben, this guy has shown us how to accurately calculate radar cross sections across the surface of the wing and at the edge of the wing and put together these two calculations for an accurate total.”

Rich then explains how radar technology works and how existing planes often show up on radar as large as houses. He then continues:

We desperately needed new answers, and Ufimtsev had provided us with an “industrial-strength” theory that now made it possible to accurately calculate the lowest possible radar cross section and achieve levels of stealthiness never before imagined. “Ufimtsev has shown us how to create computer software to accurately calculate the radar cross section of a given configuration, as long as it’s in two dimensions,” Denys told me. “We can break down an airplane into thousands of flat triangular shapes, add up their individual radar signatures, and get a precise total of the radar cross section.”

So, Rich had Overholser and a labmate put together a program to take advantage of this insight and design a model plane out of 2D surfaces that had a minuscule radar cross section — computing power was not strong enough to build the model out of 3D surfaces. The final design’s radar cross section appeared to be smaller than existing planes by several orders of magnitude. Instead of the plane being the size of a home or a barn, the question was now, “Which of these ball bearings in Ben Rich’s pocket would show up bigger than the plane, and which wouldn’t?”

Many Skunk Works veterans thought the design was horrible. It looked like an ugly, crude, origami rendition of a plane. The more a lab member knew about aerodynamics, the uglier the model looked. Many Skunk Works old-timers pushed back. Ben Rich wanted his team — which specialized in building planes that flew exceptionally — to focus on building the least aerodynamically sound plane they’d ever seen. Additionally, many in DC thought Rich must be mistaken about what the true radar cross section would actually be. Kelly Johnson held both of these gripes against Rich. He referred to the radar cross section computations as “theoretical claptrap.” Rich won a rare quarter off Johnson when the wooden model did register a vanishingly small radar cross section. Johnson, still skeptical, flipped him the quarter and said, “Don’t spend it until you see the damned thing fly.”

Luckily, it did. It was a tall task. But with the help of recent developments in computing power, the plane was able to fly using constant computer adjustments to help counteract issues with its chaotic aerodynamics. Kelly was wrong to not believe in the plane. But he was visionary in building a lab that elevated individual design visions and a single, strong boss above the gripes of a retired consultant like him. And he was right to entrust the lab to someone who was a terrier in assessing a technical idea’s merits, even if the results looked very unorthodox.

The plane broke no speed records, but that didn’t matter much since it showed up no bigger than a bird on enemy radar screens. The stealth bomber project is a strong data point in favor of the idea that minimizing bureaucracy and entrusting a bright individual with power was more essential to Skunk Works’s success than any stroke of genius from Kelly Johnson. Kelly Johnson was a genius. But, more importantly, Skunk Works was a place built to facilitate genius ideas into built reality.

Kelly built an operation that left enough leeway for the common sense and practical wisdom of his team to take over. From one project to the next, many of the operational specifics changed. Johnson noted:

The theory of the Skunk Works is to learn how to do things quickly and cheaply and to tailor the systems to the degree of risk. There is no one good way to build all airplanes.

At the start of each project, the Skunk Works leader was free to sit down with his best people and plan out exactly how to build the new plane in the way that made the most sense. From one plane to the next, the process might look extremely different. To make this work, the team needed to be kept small enough that new operating procedures could be implemented without chaos ensuing. And they needed to do this without countless pre-arranged meetings. That wasn’t the Skunk Works way.

Principle #2: Keep the Team Viciously Small

Skunk Works’ small team emphasis allowed team members to seamlessly stay updated on the entirety of the project. It kept things small enough that even the machinists could contribute knowledge to the early design stages of the project, not just have ideas dictated down to them.

Johnson emphasized the high-level importance of maintaining a small team, writing:

Working with a limited number of especially capable and responsible people is another requirement. Reducing reports and other paperwork to a minimum, and including the entire force in the project, stage by stage, for an overall high morale are other basics. With small groups of good people you can work quickly and keep close control over every aspect of the project.

Ben Rich’s book describes precisely how this small team emphasis made a difference in Skunk Works’ processes. He writes:

We had our own unique method for building an airplane. Our organizational chart consisted of an engineering branch, a manufacturing branch, an inspection and quality assurance branch, and a flight testing branch. Engineering designed and developed the Have Blue [stealth prototype] aircraft and turned it over to the shop to build. Our engineers were expected on the shop floor the moment their blueprints were approved. Designers lived with their designs through fabrication, assembly, and testing. Engineers couldn’t just throw their drawings at the shop people on a take-it-or-leave-it basis and walk away.

Our senior shop people were tough, experienced SOBs and not shy about confronting a designer on a particular drawing and letting him know why it wouldn’t work. Our designers spent at least a third of their day right on the shop floor; at the same time, there were usually two or three shop workers up in the design room conferring on a particular problem. That was how we kept everybody involved and integrated on a project. My weights man talked to my structures man, and my structures man talked to my designer, and my designer conferred with my flight test guy, and they all sat two feet apart, conferring and kibitzing every step of the way. We trusted our people and gave them the kind of authority that was unique in aerospace manufacturing. Above all, I didn’t second-guess them.

Even the procurement staff were a part of this. Johnson described why it was vital to keep them in the loop in a similar fashion, writing:

We maintain a very close liaison from me to the designer, to the purchasing agent — so that he understands the urgency in acquiring materials; to the tooling people; and to the people who actually will build any part of the plane.

For those who read the previous FreakTakes piece on ARPAnet contractor BBN, this approach should sound remarkably familiar. In it, I shared the following quote from ARPAnet project leader Frank Heart, reflecting on how he managed his small team of bright engineers:

I think mostly I tend to believe important things get done by small groups of people who all know all about the whole project. That is, in those days all the software people knew something about hardware, and all the hardware people programmed. It wasn't a group of unconnected people. It was a set of people who all knew a lot about the whole project. I consider that pretty important in anything very big. So I suppose if you call it a management style, that would be something I'd state. I think also that they were a very, very unusually talented group. I think things tend to get done best by small groups of very, very good people — if you can possibly manage that. You can't always manage it. So if you again want to call it a management style, it is to get the very, very best people and in small numbers, so they can all know what they're all doing.

In that piece, I then expanded upon Heart’s point, writing:

Heart was extremely averse to the team growing so large that communication had to be done on paper. He preferred their style of “very, very frequent interaction on problems.” As Walden remembered, “I don’t remember such a thing as we had a weekly progress meeting. We probably were more in tune with the progress than that; we probably did it hourly.” Heart and the entire, small team did their best to stay technically involved with each other’s work. Heart, as a rule, insisted on technically understanding every bit of the project. The individual team members attempted to do the same, which they felt was quite easy to do. The team all had offices and work stations immediately in the vicinity of each other. As things came up, they’d simply go talk with the relevant individual or call an impromptu meeting.

BBN’s Frank Heart and Kelly Johnson shared extremely similar beliefs on how to manage engineering teams building extremely novel equipment.

The Skunk Works project teams were about ten times larger than BBN’s project teams, given the more industrial nature of the operation. But these were still very small teams by aviation standards. Kelly estimated that Skunk Works’ teams were only 10%-25% of the usual size when “compared to the so-called normal systems” of his day. Ben Rich, after some time at the Skunk Works, helped lead a staff of three on the Blackbird project. He was quick to note that “by Skunk Works standards that was almost an empire.”

Even an exceptionally difficult project like the SR-71 Blackbird did not balloon in size. That’s saying something. Nearly every material used and part designed for the airplane needed to be rethought due to the extreme cruising speeds (~2,000 mph) and altitude (~85,000 feet) the plane would reach. Kelly made it clear at the very beginning of the project that the team would need to “start from scratch as if we are building the first airplane, just like the Wright brothers.”

In the face of this overwhelming project, Johnson doubled down on his own people and systems. He did not resort to an inordinate increase in headcount. The following excerpt from Ben Rich confirms the still-modest scale of things on the massive project. Rich writes:

Developing this air-inlet control system was the most exhausting, difficult, and nerve-racking work of my professional life. The design phase took more than a year. I borrowed a few people from the main plant, but my little team and I did most of the work. In fact the entire Skunk Works design group for the Blackbird totaled seventy-five, which was amazing.

Writing his book in 1994, Ben Rich was acutely aware of how much the industry had changed. He closes the above quote with the following quip:

Nowadays, there would be more than twice that number just pushing papers around on any typical aerospace project.

Principle #3: Maintain Similar Expectations of Project Partners

Kelly’s belief in minimizing bureaucracy and keeping team size ruthlessly small did not stop with his own Skunk Works team. He was adamant that, if at all possible, projects should only be undertaken when a project’s funder could promise similar, strong project leadership. This was the second of his famous fourteen rules:

Strong but small project offices must be provided both by the military and industry.

Johnson likely felt there was little use in putting his team into overdrive just for a customer to hold up progress and changes every step of the way. So, he wanted quick “yes” or “no” decisions from funders as well.

Throughout his book, Johnson was quick to lavish praise on those individuals and areas of the armed services that, once they decided to move forward, were excellent project partners. When he had it his way, Johnson’s projects often had something like one project liaison and ~6 people whose opinions mattered in key decisions. Kelly provided many specific cases of what ideal relationships with funders looked like. Sharing one example, he writes:

Operating at its best on our Air Force programs, the Skunk Works could get an almost immediate decision on any problem. I could telephone Wright Field, Dayton, for example, talk to my counterpart who headed the small project office there for the military — and who was allowed to stay with it to conclusion — and get a decision that same morning. Now, that’s just not possible in standard operating procedure. It’s a difficult concept to sell for the first time, though, since it means abandoning the system.

This is what an ideal, standard case looks like. But it should also be noted that Rich and Johnson mention several cases of project liaisons going above and beyond in almost unbelievable ways. In one case, Rich and Johnson noted that an exceptional CIA project partner set up shell companies and a kind of false supply chain to to acquire a large quantity of titanium from the Soviet Union for an aircraft. But, of course, that level of proactivity from the funder was not the usual.

But working with a customer that knew how to make decisions quickly was essential. It was so essential that Ben Rich noted that Kelly had an unwritten fifteenth rule:

Starve before doing business with the damned Navy. They don’t know what in the hell they want and will drive you up a wall before they break either your heart or a more exposed part of your anatomy.

On one occasion, Ben Rich did not heed this warning and sought to build a stealth boat for the Navy. He planned to do it using the same design software that they were using to build the F-117 bomber. Pursuing this project was a decision Rich came to sorely regret.

The project actually broke multiple Kelly Johnson rules. But the first misstep was forced onto Ben Rich due to internal Lockheed politics. Lockheed’s Ocean Division was in financial trouble. Meanwhile business was booming at the Skunk Works even without the stealth ship contract. So, Rich handed off the development contract for the ship to Lockheed’s Ocean Division. But Rich also sent several core team members over to the division to help lead the project. The Skunk Works group group was led by Ugo Coty. Before sending Coty and co., Ben had told the Ocean Division executives that it would be essential to let the mathematically inclined team members have the final say over everything in the ship’s design.

To the executives, this was a seemingly off-the-wall way to build an aircraft or ship. But this is exactly what Rich had decided to do with the F-117 project. It was not at all the normal procedure, but it’s what made the most sense. The radar cross section of the ship could only be as low as the most visible of its parts. Even one periscope being slightly misdesigned could make the ship appear bigger than a barn. And the mathematics of radar cross sections was unintuitive to almost all of the ship engineers.

The Oceans Division proved incapable of following Ben’s advice. And the Navy brought so many cooks into the kitchen that it probably wouldn’t have mattered even if that had not happened. Rich describes the mess caused by the Lockheed and Navy bureaucracies on the project, writing:

“We need Ugo to keep those damned shipbuilders from going off on a tangent,” I told Roy Anderson. “This is one project where the method of shipbuilding is much less important than the stealth technology,” I told Roy. “They’ll want to sacrifice the stealth if it gets in the way of the ship’s performance, but Ugo will force them to stay focused. All Dr. Perry wants to prove out is the stealth. That’s key to this test. If the ship merely floats that’s good enough.” As it happened, my fears about the conflicting agendas between professional shipbuilders and experts on stealth technology, like Dr. Perry, were realized almost from the first day that the Ocean Division took over the project.

Ugo Coty did his best, but he ran into heavy weather. His original six-man operation quickly was shunted aside by eighty-five bureaucrats and paper-pushers running the program for the division. Then the Navy marched in, adding its supervision and bureaucracy into the mix with a fifty-man team of overseers, who stood around or sat around creating reams of unread paperwork. No ship ever went to sea — not even a top-secret prototype — without intensive naval supervision to ensure that all ironclad naval rules and regulations were strictly enforced before the keel was ever laid.

“Where is the paint locker?” a Navy commander demanded of Ugo, rattling the blueprint plans. Since the days of John Paul Jones, every naval ship afloat has a damned paint locker on board. Sea Shadow would definitely not be the only exception since the Revolutionary War.

To hammer home the point about the “Navy way,” Rich goes on to tell a story of Skunk Works modifying its Jetstar for the Navy under Kelly Johnson. The original Jetstar project itself took 55 engineers eight months. The Navy project, which was a little more complicated, took 27 months. Rich noted that “one hint as to the reason why” is that Jetstar’s mockup conference had six visitors on hand. For a similar conference, the Navy sent 300. To him, that was the “Navy way.”

Before moving on, it is important to note that Johnson also had a strong preference for only working with contractors who didn’t require increased bureaucracy to be added to the project. In an ideal case, this would look like the U-2 project in which key technical project problems were offloaded to Pratt & Whitney and Shell researchers to work their way through. In another case, Skunk Works took a crucial tire problem to B.F. Goodrich “which developed a special rubber mixed with aluminum particles that gave our wheels a distinctive silver color and provided radiant cooling.” This was just the solution Skunk Works needed. However, in more standard cases what this means is simply sticking with trustworthy suppliers and extending them the same respect Skunk Works liked to have itself. As Kelly Johnson put it:

Suppliers and others associated with a project must be extended the same kind of rules and permissions that are given us for the entire program. This cuts red tape and costs and allows all participants to concentrate on the product instead of a system. It is so simple.

This was more feasible in some cases than in others. But these were the relationships Johnson strove to maintain.

While all of this sounds quite simple, replicating Skunk Works has proven anything but simple for most would-be copycats.

Why So Many Skunk Works Copycats Failed

In the early 1970s, shortly before Kelly Johnson’s retirement, Northrop made an offer to poach Ben Rich — then one of Johnson’s top lieutenants. Ben Rich recalls Kelly’s frank warning when he confronted Johnson with his offer, writing:

I laid out Northrop’s offer, and he closed his eyes and solemnly shook his head. “Goddam it, Ben, I don’t believe a word that guy said to you. I’ll bet my ranch against Northrop starting its own Skunk Works. Companies give it lip service because we’ve been so successful running ours. The bottom line is that most managements don’t trust the idea of an independent operation, where they hardly know what in hell is going on and are kept in the dark because of security. Don’t kid yourself, a few among our own people resent the hell out of me and our independence. And even those in aerospace who respect our work know damned well that the fewer people working on a project, the less profit from big government contracts and cost overruns. And keeping things small cuts down on raises and promotions. Hell, in the main plant they give raises on the basis of the more people being supervised; I give raises to the guy who supervises least. That means he’s doing more and taking more responsibility. But most executives don’t think like that at all. Northrop’s senior guys are no different from all of the rest in this business: they’re all empire builders, because that’s how they’ve been trained and conditioned. Those guys are all experts at covering their asses by taking votes on what to do next. They’ll never sit still for a secret operation that cuts them out entirely. Control is the name of the game and if a Skunk Works really operates right, control is exactly what they won’t get…Mark my words, you’ll be reporting to a dozen management types and they won’t let you out of their sight for one minute.”

Ben Rich didn’t leave. And the Northrop operation went on to operate essentially how Kelly said it would. This prediction was not Kelly Johnson demonstrating any kind of remarkable prediction abilities. He was simply recounting what he continually saw happen when companies said they wanted a Skunk Works.

The sorts of problems Kelly figured would happen to Northrop and those that had happened in the Naval stealth project arose repeatedly. The following excerpt describes similar problems that afflicted the Army’s Cheyenne rigid-rotor helicopter program. Kelly himself attempted to train Jack Real and six of his top supervisors who would be running the program. Kelly had them shadow his Skunk Works operation for six months. But his efforts were in vain. Kelly writes:

Real and his team began with great enthusiasm to apply our operating methods to meet the Army’s design specifications. But within six months, the satellite Skunk Works had a purchasing department larger than my entire engineering department working on seven projects. They had become buried in the usual paperwork already.

Despite the best of intentions, the Army had at the time ten different test centers and bases involved in the procurement of new weapon systems. And when you have that many representatives involved in design and development, with no single person in charge to represent the customer, the Skunk Works concept cannot work.

It is absolutely imperative that the customer have a small, highly-concentrated project office as a counterpart to the Skunk Works manager and his team. It is not a concept easily adopted after years of working within the system. There has to be an all-out commitment, or the method will not work.

When there was not an “all-out commitment” to work within the basic system, the differences in results were startling.

At the time the Cheyenne contract was cancelled, 145 Army personnel were involved in the program. In contrast, the total at the Skunk Works for both CIA and Air Force representatives in our U-2 and SR-71 programs did not exceed six people.

It should also be noted that Kelly himself was able to come into at least one operation outside the Skunk Works and implement his system successfully. At one point, he was called into the Lockheed Missiles and Space Company to help turn around an ailing project. Kelly came in and rolled out the same playbook I’ve described to you in this piece. He describes his work on the project, writing:

When Skunk Works principles really are applied, they work. An example of their successful application was development of the Agena-D launch vehicle…The satellite that was to become this country’s workhorse in space was in trouble in terms of design and cost but especially in reliability, which stood at an incredibly low 13.6 percent. I was drafted, in effect, to go up to the Lockheed Missiles and Space Company and fix it. We set up a Skunk Works operation with the company’s design project engineer, Fred O’Green, as head…

It proved again our axiom: If you have a good man and let him go, he’ll really perform. In terms of today’s world, that axiom should apply to women as well. When I first reviewed the Agena project, I discovered that 1,206 people were employed in quality control alone, achieving only that 13 percent reliability factor! It should have been the world’s most reliable vehicle just using the inspection department. That was enough people to design and build the thing.

At the Baird Atomic Company, which made the vehicle’s horizon sensor, Lockheed had 40 people inspecting, coordinating, and reporting. Yet Baird had only 35 people building the instrument. We resolved that situation by returning responsibility for the product to the vendor. For example, I telephoned Walter Baird personally since he and I had worked together on a number of other Skunk Works projects. He immediately agreed to pick up his end of the log.

Kelly estimated that the changes saved the government at least $50 million in costs and led to completing in nine months what had been scheduled for 18. 350 drawings had been made instead of the projected 3,900. And quality control personnel had been “slashed” to 69. The operation was now able to output design drawings in a day rather than a month.

Ben Rich had this to say about Kelly’s turnaround of the Agena operation:

The president was already spending a billion dollars in covert funds on the Agena rocket that would boost our first spy satellite into orbit. Bissell was in charge of that program, too, and the first twelve test firings had all been failures. Lockheed’s Missiles and Space Company in Sunnyvale, California, had that contract, and Bissell asked Kelly to evaluate and reorganize their operation. Kelly set up a mini Skunk Works and, coincidentally or not, the thirteenth test shot was a success.

Many claim to have copied Skunk Works in its heyday. But few ever truly did.

A Different World

In many ways, times have changed. Generals can no longer have an aircraft development contract written up and sent out in a few hours. Lowest bidder subcontracts might be required before an outfit like Skunk Works has a new kind of fuel made. Many think the management of public companies is much more short-sighted than it was in the 1960s. The list goes on.

The writing of Ben Rich himself emphatically demonstrates how much these changes haunted him. Throughout his book, he routinely mentions things like nightmares regarding corporate bean counters. In one breath he tells us of the best idea his Skunk Works would ever produce. In the next, he mentions how the Lockheed bureaucracy might threaten the idea being realized. He writes:

I wish I could claim to have had a sudden two a.m. revelation that made me bolt upright in bed and shout “Eureka!” But most of my dreams involved being chased through a maze of blind alleys by a horde of hostile accountants wielding axes and pitchforks.

And these were not small, nagging fears. Ben Rich’s tenure as head of Skunk Works saw the Lockheed and DC bureaucracies creeping in from both sides. At one point, he lobbied Lockheed management to take on a relatively risky production contract for the F-117 sooner rather than later. The idea was only green-lit with much skepticism. Rich writes:

“They’ll want at least one hundred bombers, and we’ll be looking at tens of billions in business. So what’s this risk compared to what we can gain later on? Peanuts.”

It was not a very happy meeting, and the conclusion reached was reluctant and not unanimous. The corporate bean counters insisted we install a fail-safe monitoring and review procedure that would sound the alarm the moment we fell behind or hit any snags. “Above all, no nasty surprises, Ben,” Larry Kitchen warned me.”

The Gross brothers and Hall Hibbard were gone. Kelly Johnson had been a living legend who routinely turned down Lockheed’s CEO position. His freedom and insistence on the same for his Skunk Works was tolerated by subsequent generations of executives throughout his career. But by the end of his career, many in DC and Burbank had little patience for him. They would not be so lenient with his successor.

Ben Rich’s Skunk Works — despite proving itself with its inspired F-117 project design — was being watched. He recounted:

An independent engineering review team, composed entirely of civil servants from Wright Field in Ohio, flew to Burbank to inspect and evaluate our entire program. They had nothing but praise for our effort and progress, but I was extremely put out by their visit. Never before in the entire history of the Skunk Works had we been so closely supervised and directed by the customer. “Why in hell do we have to prove to a government team that we knew what we were doing?” I argued in vain to Jack Twigg, our assigned Air Force program manager. This was an insult to our cherished way of doing things. But all of us sensed that the old Skunk Works valued independence was doomed to become a nostalgic memory of yesteryear, like a dime cup of coffee.”

He was largely right.

Johnson shared an anecdote in his 1985 book, noting just how much the defense procurement world had changed in a few short decades. He noted:

Our Air Corps project officer for the XP-38 was a pilot, a young lieutenant, Benjamin S. Kelsey. He was excellent. In those days (1938) a project officer with that rank had more authority than many four-star generals do today (1985). If we asked Ben for a decision, we got it — on the spot.

Conclusion

Times have surely changed in defense procurement. But the reader’s takeaway should not be that Skunk Works’s management playbook was great, but no longer feasible. It is true that many pools of government capital that used to be risk-tolerant and trusting of individual visions are the opposite nowadays. However, there still exist fantastic funders with a deep belief in bright people with ambitious visions. And many of these funders require only common-sense oversight. With capital from phenomenal funders like these, the Skunk Works management playbook can still prove exceptionally effective. Kelly Johnson’s rules regarding limiting bureaucracy, keeping his team ruthlessly small, and doing whatever possible to allow individual genius to flourish helped create a historically great lab.

The heart of what made the lab exceptional is also strikingly similar to what made BBN and CMU’s autonomous vehicle units so exceptional. All three maintained operating structures that were operated like small companies and were aggressively focused on novelty. The CMU group found a way to make a novelty-focused university group operate in a more firm-like fashion. BBN set up a private firm staffed with MIT professors that could only be rivaled in its novelty focus by the most daring of university labs. In Skunk Works, Kelly Johnson found a way to run a research outfit within a corporate bureaucracy like a small firm. He also insisted that the lab spend close to all of its time building planes that were novel and ambitious. When he stepped down he chose Ben Rich as his successor largely because he knew Ben wouldn’t play it safe with the lab. Without that ambition, the lab would be effective but not ambitious. It wouldn’t be Skunk Works.

Ex-Lockheed CEO Roy Anderson recalls Kelly Johnson saying the following when recommending Rich as successor:

Roy, I raised Ben in my own image. He loves the cutting edge as much as I do, but he knows the value of a buck and he’s as practical as a goddam screwdriver.

To Kelly Johnson, you likely couldn’t run a true Skunk Works without both.

Thanks for reading:)

Subscribe now

Share

Pattern Language Tags:

• Utilizing a contractor made up of individuals with research-style goals and training working within a ‘firm’ structure.

• Facilitating tool/hardware improvements in a key technology area far from its suspected theoretical frontier.

• Selecting multiple industry contractors for a trial period before choosing the final contractor. (F-117)

This piece is a part of a FreakTakes series. The goal is to put together a series of administrative histories on specific DARPA projects just as I have done for many industrial R&D labs and other research orgs on FreakTakes. The goal — once I have covered ~20-30 projects — is to put together a larger ‘ARPA Playbook’ which helps individuals such as PMs in ARPA-like orgs navigate the growing catalog of pieces in a way that helps them find what they need to make the best decisions possible. In service of that, I am including in each post a bulleted list of ‘pattern language tags’ that encompass some categories of DARPA project strategies that describe the approaches contained in the piece — which will later be used to organize the ARPA Playbook document. These tags and the piece itself should all be considered in draft form until around the Spring of 2024. In the meantime, please feel free to reach out to me on Twitter or email (egillia3 | at | alumni | dot | stanford | dot | edu) to recommend additions/changes to the tags or the pieces. Also, if you have any ideas for projects from ARPA history — good, bad, or complicated — that would be interesting for me to dive into, please feel free to share them!

General Links:

• More Than My Share of It All by Kelly Johnson

• Skunk Works: A Personal Memoir of My Years of Lockheed by Ben Rich and Leo Janos

Jeremy Howard (former President and Chief Scientist of Kaggle) and Eric Ries (creator of The Lean Startup movement and Long Term Stock Exchange) have teamed up to found a new applied R&D lab: Answer.AI.

When speaking with Jeremy, he made it clear that many details of Answer.AI's structure are still being worked out. Only announced a month ago, the org is still in its early development stages. But the founders have conviction on certain principles. The most prominent of them is one extremely relevant to this Substack: The founders seem to be particularly inspired by Edison’s Menlo Park Lab and the early days of commercial electric research.

Previously, I’ve covered Edison’s Menlo Park Lab, the early GE Research Laboratory, and many other industrial research shops. Since FreakTakes is explicitly written for individuals founding new science orgs, has a particular focus on applied R&D, and has covered the lab that partially inspired Answer.AI, I thought it would be worthwhile to put together a piece specifically for Jeremy and Eric. The goal is to distill into a single piece the takeaways from prior FreakTakes pieces that are most useful to their nascent company. If they take even one nugget of information and use it to run their organization slightly differently, I’ll consider that a win.

In the piece, I’ll briefly examine the (working) plans for the lab and do some historical analysis, detailing:

1. What the earliest electrical R&D labs can teach Answer.AI

2. Useful rules-of-thumb from other historically great applied R&D labs

3. Potential pitfalls to keep in mind as they move forward

You can find more thorough historical evidence in my prior pieces for any of the lab details I mention, listed below:

• Thomas Edison, tinkerer published in Works in Progress

• Tales of Edison's Lab (podcast)

• Irving Langmuir, the General Electric Research Laboratory, and when applications lead to theory

• How did places like Bell Labs know how to ask the right questions?

• “The Third University of Cambridge”: BBN and the Development of the ARPAnet

• An Interview with Chuck Thorpe on CMU: Operating an autonomous vehicle research powerhouse

• ILLIAC IV and the Connection Machine

• A Progress Studies History of Early MIT — Part 2: An Industrial Research Powerhouse

• How Karl Compton believed a research department should be run

Each of the orgs listed has lessons to teach Answer.AI. But none are a perfect analog. So, as the piece progresses, I’ll explain which lessons I think most strongly apply to Answer.AI. With that, let’s get into it!

Subscribe now

Share

Answer.AI in a Nutshell

Jeremy’s blog post announcing Answer.AI makes it clear that the org is, to a large degree, inspired by the field of electricity’s path of progress in the 1800s. He believes the current state of the AI field is similar to the state of the electricity field between the work of Michael Faraday and Edison’s lighting projects. This was an era in which new electrical findings were being pieced together, but few had made any progress in turning the potential of electricity into great applications.

I don’t find this comparison crazy. So far, I don’t believe AI has come close to the level of breakthrough that electricity proved to be. Electricity brought the sunlight indoors for a negligible cost and powers so many of our modern conveniences— refrigeration, TVs, central heating, etc. That’s a high bar. However, given that human ingenuity created the breakthrough that was electricity and each of those applications, it is surely worth considering that AI could grow to be the most impactful field of them all. Whether AI does reach that level of promise, to me, is a question of human ingenuity. So, I have no issue with Jeremy comparing the AI field to the electrical field c. 1830 to 1910.

With that elephant out of the way, let’s briefly examine what sets Answer.AI apart from AI labs like OpenAI and Anthropic. From a funding perspective, Answer.AI seems much, much cheaper. The founders have initially raised $10 million. This stands in stark contrast to the gargantuan initial rounds of OpenAI and Anthropic. Also, Answer.AI's research agenda is more application-centric. The following excerpt from Jeremy’s blog post highlights what he thinks differentiates the lab’s approach:

At Answer.AI we are not working on building AGI. Instead, our interest is in effectively using the models that already exist. Figuring out what practically useful applications can be built on top of the foundation models that already exist is a huge undertaking, and I believe it is receiving insufficient attention.

My view is that the right way to build Answer.AI’s R&D capabilities is by bringing together a very small number of curious, enthusiastic, technically brilliant generalists. Having huge teams of specialists creates an enormous amount of organizational friction and complexity. But with the help of modern AI tools I’ve seen that it’s possible for a single generalist with a strong understanding of the foundations to create effective solutions to challenging problems, using unfamiliar languages, tools, and libraries (indeed I’ve done this myself many times!) I think people will be very surprised to discover what a small team of nimble, creative, open-minded people can accomplish.

At Answer.AI we will be doing genuinely original research into questions such as how to best fine-tune smaller models to make them as practical as possible, and how to reduce the constraints that currently hold back people from using AI more widely. We’re interested in solving things that may be too small for the big labs to care about — but our view is that it’s the collection of these small things matter a great deal in practice.

It would be unfair to say that an application-centric research agenda is necessarily less ambitious than AGI. Those biased toward basic research might say so, but I don’t think that opinion is very historically-informed. Edison himself was application-centric above all else. His deep belief in market signals is fascinating when juxtaposed with the market indifference of many great academic physicists. In the book From Know-How to Nowhere, a history of American learning-by-doing, Elting Morison described the interesting nature of Edison’s motivations:

If the means by which he [Edison] brought off his extraordinary efforts are not wholly clear, neither is the cause for his obsessive labors. No diver into nature's deepest mysteries carrying next to nothing for the advancement of knowledge and even less for the world's goods, he would become absorbed in making something work well enough to make money. The test in the marketplace was for him, apparently, the moment of truth for his experiments.

Edison built his god-like reputation by dreaming in specific applications. He kept market, resource, and manufacturing constraints in mind from the earliest stages of his projects. Edison dreamed practical, realizable dreams. And when the limitations of component technologies stood in the way of his dreams, he often had the talent to invent new components or improve existing materials. Edison’s biggest dream, the light bulb, mandated that  Edison solve a much broader set of problems. The following excerpts from my Works in Progress piece on Edison paint a clear picture of his ambitious but practical dreams:

After Edison’s bulb patent was approved in January 1880, he immediately filed another for a ‘System of Electrical Distribution’. Filing for these so close together was no coincidence. To Edison, it was never just a bulb project. It was a technical business venture on a possibly unprecedented scale. Edison wanted to light up homes all over the world, starting with lower Manhattan.

Bringing the project from dream to mass-market reality would require solving over a hundred technical problems. His was a new bulb that needed to be powered by a generator that did not yet exist at the start of the project, strung up in houses that had no electricity, connected via underground street wiring that was only hypothetical, and hooked up to a power station that had never existed before.

Yet, at the end of two years’ time, Edison would do it. And, just as importantly, the entire venture was profitable by the end of the project’s sixth year.

Edison was clearly doing a different kind of dreaming than those who do basic research. His lighting work embodies what extreme ambition looks like in application-centric research. Answer.AI making this kind of ambitious, applied work their North Star is an extremely interesting goal.

This goal has the potential to give Answer.AI a comparative advantage in the growing space of for-profit AI labs. For example, the most ambitious aspects of OpenAI are considered to be in its research, not its work on applications. Answer.AI’s particular setup can also set it apart from AI startups and academic labs. New AI startups do some research on how to commercialize new AI models in new ways, but they generally have short runways. In this kind of environment, only specific types of research projects can be pursued. Academic labs — for many reasons covered elsewhere on this Substack (such as in the ARPA series) — don't have the right combination of incentives, experience, and staffing to build new technologies in most problem areas. The main incentive of the profession, in a simplified form, is producing many paper studies that get cited many times. Answer.AI has the chance to let its alternative focus lead it to areas under-explored by academics, companies with brief timelines to hit revenue benchmarks, and more AGI-focused R&D labs.

Legally, Answer.AI is a company. But in practice, it might hover somewhere between a lab and a normal “profit-maximizing firm” — as was the case with Edison's lab. The founders seem perfectly content to pursue high-risk projects that might lead to failures or lack of revenue for quite a while. In saying this, I do not mean to imply they are content to light money on fire doing research with no chance of a return. Rather, they hope to fund a body of research projects that ideally have positive ROI in the long term. They are just not overly concerned with short-term revenue creation.

(Making the pursuit of research agendas like this easier is actually one of the founding goals of Ries’ Long Term Stock Exchange — which I address later.)

There is apparently no pressure to produce a product that can hit software VC-style revenue goals within 12-24 months, or anything similar. This is good. Seeking to satisfy these types of metrics does not traditionally permit a company to act like a truly ambitious R&D lab. I’m not saying it can’t happen — DeepMind seems to have made it work in its early years — but it does require pushing against investor pressure quite strongly. The VC money raised for Answer.AI has left the founders with enough voting shares that investors can’t veto founders’ decisions. Additionally, Howard says the company’s investors understand what they are trying to build is, first and foremost, a lab. This is a great step towards building an organization focused on building very useful, very new things rather than the most profitable thing possible — which often comes with bounded technical novelty.

Interestingly, Answer.AI will also keep a small headcount. Jeremy built Fastmail up to one million accounts with only three full-time employees. He hopes to keep the Answer.AI team exceptionally talented and “ruthlessly small” in a similar way; he believes keeping teams small is important to building new, technically complex things.

Now that I've outlined some important pieces of Answer.AI’s vision, I'll dive into the historical analysis. In the first section, I detail lessons that Answer.AI can draw from both Edison’s Menlo Park laboratory and the Early GE Research Laboratory. In the following section, I'll share useful lessons from other historically great industrial R&D labs. Lastly, I’ll highlight the bureaucratic details that explain why the operational models of the great industrial R&D labs have not been replicated often.

Learning from the First Electrical R&D Labs

I find it exciting that Edison’s Menlo Park lab is a North Star for Answer.AI. I covered Edison’s work in several pieces because I think evergreen lessons can be drawn from his work. But I think a more complete way to incorporate lessons from the 1870-1920 electrical space is to draw on the work of both Edison’s Menlo Park Lab and the young GE Research Lab. The latter operated as a more traditional industrial R&D lab. GE Research’s history holds many lessons to help steer Answer.AI’s problem selection and work on its standard projects. However, exceptionally ambitious projects may draw more heavily on the lessons of Edison’s lab.

(As a note, while Edison General Electric was one of the two companies that merged to become GE — along with Thomson-Houston Electric — Edison had essentially nothing to do with the formation of the iconic GE Research Laboratory.)

Different types of projects characterized the work of the two electrical labs. When it came to electrical work, for years, Edison’s lab and mental efforts were focused on doing everything necessary to bring a single, revolutionary product to market. On the other hand, GE Research usually had many separate courses of research underway at once. These projects all sought to improve the science and production of existing lighting systems, but they were otherwise often unrelated to each other. Additionally, GE’s work could be categorized as more traditional “applied research.” The lab was not actively looking to create a field of technology from scratch as Edison did. GE Research's projects were often novel and ambitious, but in a different way than Edison's.

Later, I will explore the types of novelty the GE Research Lab pursued. First, I’ll give the reader a more fine-grained idea of how Edison’s lighting project actually operated.

Lessons from Edison’s Work on Electricity

Edison’s lighting work provides great management lessons for those looking to direct a large chunk of a lab’s efforts toward a single, big idea.

Edison’s major contribution to the field of electricity was not inventing each of the components in his lighting system, but in turning a mass of disparate gadgets, scientific principles, and academic misconceptions into a world-changing system. The burden of doing “night science” — as Francois Jacob refers to it — largely fell on Edison. In the late 1870s, nobody knew much about electricity yet. The existing academic literature had more holes than answers, and many of its so-called “answers” turned out to be wrong or misleading. From this shaky starting point, Edison proceeded. He combined his unique mix of attributes and experience to deliver a world-changing system. These included: knowledge of several adjacent scientific fields, deep knowledge in then-overlooked experimental areas, market knowledge, manufacturing knowledge, and the ability to adequately operate a small research team.

In large part, Edison created his lab as a way to scale himself. As a result, to understand how his lab operated, one needs to know how Edison himself carried out his explorations. Edison was one of the more stubborn experimentalists of all time. He spent most of his waking hours carrying out one experiment or another. While he did pore over scientific literature, for him, nothing was settled until he proved it for himself at the lab bench.

I write in my Works in Progress piece:

Edison respected scientific theory, but he respected experience far more. In Edison’s era of academia as well as today’s, many professors had a certain preference for theory or ‘the literature’ over hands-on improvement. Because of this Edison did not care much for professors. He was even known to go on long diatribes, during which he had assistants open up textbooks, locate scientific statements that he knew to be untrue from experience, and quickly rig up lab demonstrations to disprove them. ‘Professor This or That will controvert [dispute with reasoning] you out of the books, and prove out of the books that it can’t be so, though you have it right in the hollow of your hand and could break his spectacles with it.’

Contained in his head was a database of countless experiments and results that made it seem as if his “intuition” was far beyond his contemporaries. This left him with an unparalleled skillset and body of knowledge. If anyone could feel comfortable pursuing a project that others had previously failed at, it was Edison. Edison’s confidence in his skills was never more on display than when he chose to pursue his lighting work. Many in the scientific establishment knew electric bulb lighting was technically possible, but claimed they had proven that it could never be economical. Edison disagreed.

On top of Edison’s admirable approach to experimentation, he brought a high level of practicality to his process. He knew his inventions needed to make commercial sense in order to make it out of the lab. So, even in early courses of experimentation, he kept factors like manufacturability in mind. He wouldn’t commit much time to something that didn’t make commercial sense. With that being said, Edison wanted to change the world with his technologies more than he wanted to get rich. So, the practical factors he paid aggressive attention to were primarily treated as constraints. He did not optimize for profitability, but he knew his ideas needed to be profitable. Nobody who wanted to optimize for profit would have pursued lighting in the way Edison did. The technical risks were too great.1

Edison was able to imagine an ambitious system that required many technical advances. It was so futuristic that maybe only he was capable of coming up with it. But just as impressively, he was able to do it profitably and on schedule. His dogged commitment to experimentation seems to be largely responsible for this. Edison and “the boys” constantly experimented on every piece of the process to improve and learn more about all the sub-systems in Edison’s grand system. They wanted to know how every piece of every sub-system performed in all conditions. I’ll share just two excerpts from my Works in Progress piece as examples.

The first is from Edmund Morris' biography of Edison. It recounts how thoroughly Edison and his trusted aid, William Batchelor, were in carrying out round after round of filament experiments:

For week after week the two men cut, planed, and carbonized filaments from every fibrous substance they could get — hickory, holly, maple, and rosewood splints; sassafras pith; monkey bast; ginger root; pomegranate peel; fragrant strips of eucalyptus and cinnamon bark; milkweed; palm fronds; spruce; tarred cotton; baywood; cedar; flax; coconut coir; jute boiled in maple syrup; manila hemp twined and papered and soaked in olive oil. Edison rejected more than six thousand specimens of varying integrity, as they all warped or split…

In the dog days, as heat beat down on straw hats and rattan parasols, the idea of bamboo suggested itself to him. Nothing in nature grew straighter and stronger than this pipelike grass, so easy to slice from the culm and to bend, with its silicous epidermis taking the strain of internal compression. It had the additional virtue, ideal for his purpose, of being highly resistant to the voltaic force. When he carbonized a few loops sliced off the outside edge of a fan, they registered 188 ohms cold, and one glowed as bright as 44 candles in vacuo.

This approach went far beyond bulb filaments. The following excerpt describes the work of one of Edison’s lead mechanics in turning the Menlo Park yard into a 1/3 scale model of what they would later install in Lower Manhattan. I write:

[Kruesi, Edison’s mechanic] along with a group of engineers and a team of six diggers, turned the excess land of the lab in Menlo Park, New Jersey…into a one-third-scale model of Edison’s first lighting district in lower Manhattan. This team tested and re-tested the electricity delivery system, digging up Menlo Park’s red clay to lay and re-lay an experimental conduit system. The team carried out countless tests to ensure that they found materials to efficiently carry the electric current while also keeping the delicate materials safe from water and ever-present New York City rats.

The entire process was marked by the classic trial-and-error of the Edisonian process. The first subterranean conducting lines and electrical boxes the group laid were completely ruined by two weeks of rain — despite being coated with coal tar and protected with extra wood. While the diggers dug up the failed attempt so the damage could be examined, Kruesi and a young researcher…studied and tirelessly tested unbelievable numbers of chemical combinations — making full use of the laboratory library and chemical room — until, finally, a blend of ‘refined Trinidad asphaltum boiled in oxidized linseed oil with paraffin and a little beeswax’ was found that protected the electrical current from rain and rats.

Edison built his own style of dogged experimentation into the culture of his lab. Since the lab was meant to scale Edison, this makes perfect sense; he was a man with far more ideas than hands. So, he hired more hands. Edison did not search far and wide to hire the world’s best research minds, and many of those he employed did not even have scientific backgrounds. This didn’t matter much to Edison because most of them were employed to undertake courses of research that he had directed them to pursue. A couple of his Menlo Park employees had advanced scientific degrees, but far more did not. For the most part, the lab and its activities were steered by Edison and his ideas. As a result, the productivity of his lab followed wherever his attention went. After some time working on a project area, Edison would often grow antsy and wish to move on to the next thing — he craved novelty. The lab’s resources and extra hands would move with him. As we’ll see in the next section, this stands in stark contrast to how the GE Research Lab recruited and chose problems.

Menlo Park's electrical activities provide a great management playbook for what it looks like to direct a lab’s efforts toward a single, major system. If Answer.AI does not want to go all-in on one thing, it can still find a way to apply this playbook to a certain focused team of employees while leaving the others to tinker around with exploration-stage ideas. In Edison’s less-focused experimentation periods, his lab served as more of an “invention factory,” doing this sort of fiddling. Additionally, Edison's preference for application and commitment to experimentation over theory in a young area of science can surely provide Answer.AI some inspiration.

Of course, Edison did some things better than others. Edison’s most easily-spottable “deficiency” is that his lab was largely dependent on him. Without him and his big ideas, the lab would have probably ground to a halt. While Edison’s technical vision, practicality, and experimental approach are absolutely worthy of emulation, the lessons of GE Research should probably be added into the mix as well. GE operated as more of a prototypical industrial R&D lab with an approach quite suited to the fact that the science of electricity was beginning to mature in the early 1900s.

Lessons from the Young GE Research Laboratory

The young GE Research lab took a different approach to electricity research than Edison. The lab worked on many unrelated projects at once, recruited more talented researchers, and allowed these talented researchers more freedom to exert the scientific method on commercializable projects. The lab did not undertake projects that were as purposely futuristic as Edison did. Nobody from the lab earned nicknames like “the Wizard of Menlo” or “the Wizard of Recorded Sound.” But early GE Research was still responsible for a Nobel Prize and making the light bulb a much-improved, more cost-effective technology.

Elting Morison wrote the following on the lasting impact of GE Research’s early decades:

There seems little doubt that…much that was done in Schenectady in electrical engineering and some parts of physics was both better done and more interesting than what was being done in those fields in any American university.

In its heyday, even great researchers like Karl Compton hoped to shift their academic departments to operate more like GE Research.

While GE did simultaneously pursue diverse projects, there was a unifying thread holding all of the projects at GE Research together. Each project aimed to improve the quality and profitability of GE’s products and manufacturing. Under that unifying theme, all kinds of projects were encouraged. Much of the research was very applied, particularly in the early years when the lab was still proving itself.

William Coolidge was one of the lab’s most talented applied researchers in its early years. Coolidge joined the lab in 1905, part-time while teaching courses at MIT. Coolidge had the kind of toolkit typical of many MIT professors in that era. He had a far greater grasp of the science of physics and metallurgy than somebody like a blacksmith; he was simultaneously far closer to a blacksmith than one would ever expect a university researcher to be. With this differentiated toolkit, he did science in a way that was not typical of academics. In describing the process that led to his successes at GE, he claimed that he was, “guided in the main by experiment itself rather than by metallurgical knowledge.”

Coolidge expertly applied practical skills in concert with scientific knowledge to pursue the problem. Elting Morison described a small sample of Coolidge’s workflow:

He suspended tungsten powder in an amalgam of bismuth, cadmium, and mercury. He then passed the resulting substance through tiny dies — drawing it — and obtained a silvery pliable wire. At that time, he thought he had reached ductility and the search was over. But when a current was passed through this wire the mercury, cadmium, and bismuth distilled out, leaving, unfortunately, a nonductile tungsten. But it also proved to be tungsten in the purest state he had yet produced.

I continue in my FreakTakes piece, writing:

He eventually iterated his way to a workable process where…the more pure tungsten was put through a specific combination of metal-working processes at a temperature that worked that produced rods of tungsten about 1 mm in diameter. These 1mm rods could then be drawn and re-drawn through rods of decreasing size until you were left with wires of tungsten .01 mm in diameter. When put in the vacuum-sealed bulb, electricity ran through the tungsten filaments and demonstrated an efficiency of 1 watt per candle — extending the life of a bulb up to 27x.

Within 5 years, 85% of all lamps would be made from tungsten. As the project went on, more and more research chemists and technical assistants grew to be involved in the wide-ranging steps and combinations involved in Coolidge’s experiments. But it worked. GE had the factories re-fit and deployed the new bulb. Coolidge moved on to other research.

The success of Coolidge’s hybrid work style, not dissimilar to Edison’s, is surely a useful data point to Answer.AI. But GE Research also did work that went far beyond Coolidge’s technically adept, applied science. The lab was fantastic at making use of talented individuals who were very academic. Irving Langmuir was a prime example. I described his interests in my original piece:

It should be noted…Langmuir did not even care about lightbulbs. Well, I guess that is not technically true. The bulb interested him because 1) he thought a metal like tungsten was cool because it could accept really high temperatures which opened up options to the scientist working with it and 2) these vacuum-sealed bulbs provided a pristine environment for controlled scientific investigations.

To Langmuir, light bulbs were primarily a playground in which to do his science. But Willis Whitney knew how to take an individual like that and direct his energy towards productive ends. The lab deployed a principle that I call extending a “long leash within a narrow fence” to basic researchers like Langmuir.

The way the lab facilitated this was rather simple. On his first day, Langmuir was told to walk around the applied end of the lab and ask people about their projects. Whitney permitted him to undertake any course of investigation of any phenomenon he wanted, but it had to be directly related to an existing problem/limitation/constraint that the applied folks were working through. These applied folks were working on projects that rather directly plugged into GE’s operations, so there was minimal risk of Langmuir’s work not amounting to anything useful if he succeeded and found answers. With that assurance of applicability, Langmuir was given extensive timelines to find answers to open questions.

Langmuir’s first course of research focused on the constant bulb-blackening problem common to bulbs at the time. The problem was generally attributed to a bulb’s imperfect vacuum. Langmuir found this problem to be a great excuse to carry out a course of experimentation he found interesting. Morison described Langmuir’s thought process as follows:

If residual gases — imperfect vacua — produced a bad effect — blackening — here was a fine opportunity to study the effects produced by different gases introduced one by one into the bulb. What he wanted to do, he told Whitney, was simply to plot the interactions of various gases exposed at low pressures to very high temperatures in the filament. Nobody knew very much about this phenomena and he wanted to look into it simply “to satisfy [his] own curiosity.”

Langmuir carried out this course of research over three years. There were many gases and temperatures to test, which took time. But unforeseen results constantly took Langmuir off in different directions. Exploring these unforeseen results often entailed new courses of experiment altogether. With his long leash, Langmuir was able to figure out that imperfect vacua were not what caused bulb blackening at all. Rather, it was that tungsten vapor particles were finding their way onto the wall of the bulb. Temperature was the issue.

He also discovered that different gases markedly changed the rate of evaporation. One extreme example was nitrogen, which reduced the evaporation rate by 100-fold. However, adding nitrogen to the bulbs caused the electrical efficiency of the system to decrease drastically. So, the existing bulb design with nitrogen added was less cost-efficient than the normal bulbs. But Langmuir was undeterred. This was progress.

Existing fundamental research in this area led him to believe that this efficiency issue could be alleviated by increasing the diameter of the filament. Further experimentation proved this to work. He also found that coiling the filament in a certain way could mitigate the heat loss issue. The final result was a novel bulb that used an inert gas instead of a vacuum to reduce bulb blackening. Along with the coiled tungsten filament, this new bulb only required .5 watts per candle and lasted three times longer than any other bulb.

Once he passed the bulb project onto the engineering team at GE Research, Langmuir set his sights on an anomaly he had come across talking with the lab’s more applied staff. The bulbs in the lab had a design that depended on only a few milliamperes of current flowing across the space between one end of the filament and the other. Langmuir noted this anomaly in a letter to Scientific Monthly, writing:

This fact seemed very peculiar to me, for the work of Richardson and others had indicated that at temperatures as high as those used in the tungsten-filament lamp, currents of many amperes should flow across the space. In other words, according to the then-accepted theory of the electron emission from hot filaments, a serious difficulty should have been encountered in the construction of tungsten-filament lamps. The fact that we did not meet any such difficulty therefore seemed to me a peculiar fact that should be investigated.

In the brief course of exploration that followed from Langmuir, he discovered what is now known as the space-charge effect. This work combined with follow-on work from Coolidge to produce an entirely new kind of GE X-ray tube.

Under this “long leash within a narrow fence” guideline, Irving Langmuir would go on to be partially responsible for a handful of new and improved product lines at GE. Additionally, the knowledge he created with his tungsten filament work went far beyond padding GE’s balance sheet. Over the course of his project, he noted that the way tungsten vapor condensed did not gel with existing academic theory. His subsequent exploration of this phenomenon led Langmuir to be credited with founding the field of surface chemistry. Langmuir earned himself a Nobel Prize for his efforts.

There was a symbiosis in the GE lab between Langmuir types and the Coolidge types — the latter skillset being more standard in the lab. I imagine Answer.AI will have no shortage of Coolidge-like individuals: bright, Kaggle Grandmaster-type individuals who understand academic theory but whose specialty is in expertly applying their craft in dirty, practical situations. Someone like Jeremy Howard will likely have great intuition about how to utilize these individuals. The GE playbook — with its “long leash within a narrow fence” principles — can help Answer.AI think through how to deploy basic researchers in its operations

Langmuir’s career at the GE Research Lab provides a clear roadmap for how to optimally leverage a basic researcher’s energies in an applied context. Langmuir getting paid to investigate any anomalies would likely have satisfied his curiosity. However, it was his investigation of the right anomalies that made this a beneficial arrangement for GE Research.

In general, there is a time and place to apply insights from either Edison’s playbook or GE’s. The maturity of a given research field or technology area has a strong hand in dictating which set of principles is more applicable. Edison came first and had to shoulder the burden of developing an extensive technical system to power the “killer app” that was his bulb. GE Research had the benefit of working on an existing technology area with moderately developed science and existing user technology (thanks to Edison), but the technology still needed a lot of work to become reliable and economical.

A lab can simultaneously employ both playbooks. Even most of Edison’s projects were modest in relation to his lighting work. When inventing for existing fields, such as telephony, Edison contained his inventive streak to working within existing technical systems. He knew nobody would rebuild entirely new telephone infrastructure just because the young inventor had rigged up a moderately improved but completely different version. When adding to Bell’s telephone, he simply invented a carbon transmitter that could plug directly into the system. This device made voices come through much clearer. That was it: one gadget that cleanly plugged into the existing system. Technologies like these may not be as earth-shattering as Edison’s lighting system, but they were still enough to make him a world-famous inventor in his own time.

It was about impact. In optimizing impact, I thoroughly suspect Answer.AI to make great use of the playbooks of both of these small industrial research giants.

Learning From Other Historically Great Industrial R&D Labs

I’d now like to highlight applicable lessons from other research operations covered on this Substack. I’ll cover the orgs in no particular order.

Striking the Balance of BBN and CMU’s Autonomous Vehicle Group

FreakTakes recently covered two historically great DARPA contractors who expertly balanced the competing pulls of project novelty and deployable technology. The first was Bolt, Beranek, and Newman (BBN), the contractor primarily responsible for the ARPAnet. The second was Carnegie Mellon’s autonomous vehicle groups.

BBN embodied what it meant to be a “middle ground between academia and the commercial world.” The firm was initially set up by MIT acoustics professors to pursue their contracting work more ambitiously. In its early decades, the firm gradually expanded its contracting efforts into the computing space, initially under the leadership of BBN VP J.C.R. Licklider. BBN soon became a common home for the best researchers in Cambridge, abandoning their academic positions to work for BBN. The firm's growing reputation even earned the monicker the “third university of Cambridge.”

The firm’s revenue was primarily sourced from research contracts given out by orgs like DARPA, research grantmakers, and aerospace firms. BBN’s positioning was somewhat unique; when compared to industry, the firm emphasized novelty and cutting-edge technology work. This insistence on novelty helped the firm recruit individuals who felt a bit too talented to waste away working on derivative projects at Westinghouse. When compared to academia, BBN emphasized working on real technology that people would use in the near term. J.C.R. Licklider is just one prominent example of an individual who left a tenureship at MIT to work on more useful technology down the road at BBN. Leveraging this positioning, the firm was able to recruit the best talent.

BBN also provided its most talented individuals latitude to ply their minds broadly. Many projects at BBN showcased the extreme potential of small teams of talented individuals with broad technical knowledge. Only eight BBNers were primarily responsible for pushing the early ARPAnet into existence. The size of the team was no accident; Frank Heart, the engineering lead of the project, described why he preferred a team of this size in his oral history:

I tend to believe important things get done by small groups of people who all know all about the whole project. That is, in those days all the software people knew something about hardware, and all the hardware people programmed. It wasn't a group of unconnected people. It was a set of people who all knew a lot about the whole project. I consider that pretty important in anything very big. So I suppose if you call it a management style, that would be something I'd state. I think also that they were a very, very unusually talented group. I think things tend to get done best by small groups of very, very good people — if you can possibly manage that. You can't always manage it. So if you again want to call it a management style, it is to get the very, very best people and in small numbers, so they can all know what they're all doing.

Lockheed Skunk Works legend “Kelly” Johnson also held quite similar beliefs when putting together teams to build experimental aircraft. Particularly in the early stages of novel projects, there is a strong case for keeping things small, with specialists who understand the fields that touch theirs. To me, Jeremy’s belief in small teams seems well-validated by technical history.

BBN demonstrates the ideal case of a research firm that wholly embraces technical novelty. CMU can be thought of as the flipside of that coin: a university that wholly embraced systems-building and used firm-like management practices to do so. The highlight of CMU’s later-1900s systems work was its autonomous vehicle projects. The academic group staffed itself with researchers responsible for technical integration and management-style work to effectively carry out novel technological systems building.

Similar to BBN, CMU’s positioning was differentiated from both industry and academia. This fact became very clear as DARPA’s mid-1980s autonomous vehicle work progressed. CMU was seemingly the only contractor excited about technical novelty and systems integration. Martin Marietta — the defense prime in charge of DARPA’s Autonomous Land Vehicle — obsessed over ways to hit DARPA’s demo benchmarks while using unambitious, dated technologies. Simultaneously, the academic vision research groups cared more about using the camera data to write papers than helping directly contribute to building a functional driving system. CMU was the only contractor involved in the project who truly cared about building a novel, functional system. DARPA eventually recognized this and gave them ample funds to build successive generations of autonomous vehicles. The rest was history.

CMU carried out this work with a management structure that was more firm-like than most academic labs. For example, Chuck Thorpe did project management-style work for the team with firm-like incentives — he was a researcher promoted based on vehicle performance, not his h-index. While this was a firm-like position, the group also had academic-style positions. Its use of graduate students on the projects is one prominent example. Each student on the project had to own a piece of the project that was all their own and could be written up as a thesis.

(I explore how the team mitigated the risks of theses not panning out in my interview with Chuck Thorpe.)

These academic incentives partially enabled the CMU team to continually innovate. Oftentimes these students’ theses perfectly plugged into existing systems, such as a thesis on reducing the processing time of an existing sensor’s data from 15 minutes to 90 seconds. But on the most extreme occasion, in 1988, this incentive structure led a grad student named Dean Pomerleau to successfully train a neural net to steer the vehicle. In that particular case, the requirement to allow each grad student to try something new changed the world.

Answer.AI similarly cares about deep technical novelty and building deployable technology. As such, Answer.AI could benefit from emulating BBN and CMU’s strategies to balance the two. The success of BBN and CMU should hopefully embolden Answer.AI’s founders to trust in the priorities they have set. This balance of goals is uncommon today, but orgs from history have expertly balanced the two to world-changing effect.

With that said, time elapsing without world-changing results might be unnerving. This ambiguity is partially what pushed academia to rely on near-term outcome variables that incentivize the incremental. Fear of wasting time and money is real. The Answer.AI founders would surely like some way to ensure that they are spending theirs on good problems. To deal with that, I think the Bell Labs’ approach to problem selection has a lot to add to the approaches I’ve already covered.

The Bell Labs Approach to Problem Selection

Bell Labs’ management of researchers in its golden era is famous — and it should be. However, Bell researchers were not left to their own devices to pursue whatever they wished, despite what many think. Bell managed their researchers with an approach similar to the “long leash within a narrow fence” approach of GE — which one long-time Bell chemist called “circumscribed freedom.” The most effective tool it used to do this was its corp of excellent systems engineers.

Bell had an expansive product line with massive scale — even more so than GE. Even modest improvements from the research team could have outsized returns. This, of course, is not the case with Answer.AI. However, Bell’s use of systems engineers can still be extremely instructive to Answer.AI, even if Answer.AI may deploy them in different ways.

As I covered extensively on FreakTakes, Bell’s systems engineers often combined several knowledge bases to expose the right researchers to the problems that most needed solving. Within one mind systems engineers often combined STEM backgrounds, knowledge of the nitty-gritty details of Bell’s manufacturing, an understanding of Bell’s implementation problems, detailed knowledge of Ma Bell’s expenses, and familiarity with the researchers at Bell Labs.

Bell knew these systems engineers were a massive part of their secret sauce, ensuring Labs deployed its limited resources on the right kinds of problems with sufficient upside. I wrote in the conclusion of my Bell Labs piece:

Finding a problem in these systems is not so hard for those familiar with the systems. That’s why many researchers and engineers do not feel the need to bring in help. But finding a set of good problems is not finding the best problems. Finding the best problems is a profession in and of itself. A systems engineer is worth it when, under the right scrutiny, it might turn out that the best problem is 10X as financially valuable, does 50X the social good, or is 2X as likely to work as just some run-of-the-mill good problem.

My prior piece delves deeper into the specific problems towards which Bell’s systems engineers led Bell’s researchers. For now, suffice it to say that I think that any new applied science org that can dedicate an (ideally full-time) individual to doing the work of a systems engineer should strongly consider it.

Of course, these systems engineers would need somewhat clear marching orders on what sorts of technologies they should be exploring. Nowadays, many existing roles in industry and academia train individuals to equate revenue or potential citation count with impact. Answer.AI will not be satisfied with these metrics as proxies of impact, and they shouldn’t be. What to direct these systems engineers toward instead should surely be up to Jeremy and Eric.

As a firm that pursued novelty and was not attached to a large industrial operation, BBN might be an interesting source of inspiration. Several of its hallmark projects started with systems-engineer-style contributions. Three examples are DARPA PM Larry Roberts putting out the ARPAnet contract, J.C.R. Licklider’s visionary Libraries of the Future Project, and BBNer Jordan Baruch’s early-1960s pitch to the NIH on a system to build a computer system to facilitate modern hospital operations. All three project initiators had Bell systems-engineer-like exposure to the people and problems of their field — technical backgrounds, regularly spoke with the best academic researchers, knew modern industry’s issues, were able to project the costs and complications of potential projects, etc.

Few have used systems engineers as effectively as Bell Labs, with Bell-style goals, since the great lab was broken up. It would be amazing to see a lab like Answer.AI commit significant staff time to this purpose.

The Cautionary Tale of Thinking Machines Corporation’s Funding

The case of the Thinking Machines Corporation (TMC) is not as directly instructive as the examples above, but TMC made one key mistake that makes it worth mentioning. TMC put itself in the unfortunate position of raising some of its funds from investors whose incentives were not aligned with theirs.

For those who don’t know, TMC was a complete failure as a commercial firm. As a result of its bankruptcy, many write the firm off as a holistic failure. But the firm did accomplish many of the technical goals it set out to achieve. Since the company was conceived with technological goals in mind, rather than a specific market, this was no small feat. The company was founded by Danny Hillis, a PhD from Marvin Minsky’s lab at MIT. Through TMC, he sought to build the machine he conceptualized in his graduate thesis: a truly parallel computer to improve the capacity of all scientists. The young company recruited the best researchers — including scoring Richard Feynman as its “intern” for several summers — and achieved many technical goals that helped pave the way for the modern field of parallel computing. Jeremy, who knows far more about the technical aspects of parallel computing than I do, sang TMC’s technical praises in our first conversation. He emphasized how shocking it was that TMC seemed to be the first to employ so many methods the field still uses today.

However, TMC’s high-powered team and great technical work were not enough to overcome their management follies. The firm spent money as if its financial standing was in line with its technical reputation, which it was not. In retrospect, some of these management decisions — such as a comically expensive long-term lease — could have been avoided without modifying the company’s general approach. However, on a deeper level, there was a dissonance between the company’s technical goals and the funding it raised.

TMC’s two major funders were DARPA’s computing office and private investors. The goals of Hillis and his top-flight technical staff were only aligned with DARPA’s goals. In looking to build the technically most ambitious parallel computer possible, DARPA funding was ideal. The DARPA computing office also felt that TMC’s work was progressing exceptionally well for most of its early years. However, the level of enthusiasm Hillis and the technical staff had for building a machine for science did not bleed into the most profitable areas — like deploying the machine on banking databases or managing logistics for Walmart. As time wore on, it seems that pressures were beginning to mount for TMC to pursue work more in line with those areas. It seems highly likely that TMC would have had to disappoint one of its funders sooner or later, even if it spent funds more wisely in its early years.

Had TMC just raised funds from DARPA and spent them much more modestly, the company might still exist today; it may have even earned a reputation beyond that of NVIDIA.

Copying mid-20th C. Industrial R&D Models is Hard for Incumbents

The middle 20th Century saw both the rise and fall of ambitious American industrial R&D labs. It is not just nostalgia that makes modern researchers look back on these labs with fondness. Not long after these labs were formed, it was becoming clear to many top researchers that the model was special.

In 1927, Karl Compton wrote a prescient letter to Science that praised these labs. At the time of writing the letter, Compton was the head of Princeton’s Physics Department and a part-time GE contractor. His letter asserts that these organizations were doing some of the best science in the country, even though the top universities often had the best men. A portion of the letter, in which Compton praises the labs’ management of scientific projects, reads:

This has been strikingly demonstrated in some of the big industrial research laboratories, from which the output has greatly exceeded the individual capacities of the research workers.

Compton believed there were many lessons that university departments should steal from these exceptional industrial labs. The first was the need to specialize when building a portfolio of researchers and projects. Why should every department attempt to loosely approximate the makeup of researchers and research questions in the field as a whole? No company would ever do such a thing. He believed that, “Such a policy dissipates effort, and if every institution followed it we should have the spectacle of a great many universities all very much alike and all with struggling, mediocre departments.”

On an organizational level, Compton believed the equilibrium of N autonomous professors with N separate budgets and a few grad students in their lab under their control was just not efficient for most projects. It would be silly for all departments to function that way. In the letter, Compton proposes what I’ve taken to calling a “Compton Model” research department. This model is far more structured than a department of mostly autonomous professors doing ad-hoc research with their own separate funds. Compton describes it as follows:

There is another direction in which more effective organization is possible within the universities themselves! Departments of a somewhat more flexible nature than those to which we are accustomed and which could, more than now, be built around one or two outstanding men in the department, could give these men an opportunity for organization and concentration of effort which is now rarely possible.

Another way to describe this model is as a “fund department heads, not projects” model. The model allows one or two individuals to largely shape the research vision, hiring, project selection, capital purchasing, etc. of an entire department at once. In addition, it allows these individuals the latitude to replace salaries spent on additional professors or grad students with full-time engineers or discretionary capital expenses as needed.

The model makes perfect sense. However, it’s remarkably difficult to make happen at a real university. Compton couldn’t succeed in doing so when he took over as MIT President. The existing stakeholders and structures are just too hard to shift in this direction. Absent some special circumstance, most university administrators wouldn’t find the idea even worth considering. However, CMU’s President Cyert was able to build a department that loosely resembled a Compton Model department. And its results were exceptional! But this was the exception. Pivoting an existing department to run like a Compton Model department has proven infeasible in almost all cases.2

The great, old models of managing labs didn’t disappear from industry because they lacked scientific merit. These labs largely began to disappear in the 1970s and 1980s. The 1970s saw a deep recession, which usually hurt R&D budgets. In the 1980s, new corporate management trends surfaced that led to companies being managed more myopically than ever before or since. These, along with other non-scientific trends, were largely responsible for the labs going away when they did. With them, the operational know-how that had slowly been built up throughout the century dissipated. Now, to learn how these orgs operated, one must read oral histories or talk to now-retired engineers.

The first thing these retired engineers will often tell you is that we should bring the old models back! Bell’s long-time researchers had confidence in the Bell model even as a late-1900s court case ended Bell Labs as we know it. In the 1920s, Compton dreamt of copying much of the GE model to use within a university. One should not be afraid to put their time and resources behind bringing these models back. Great engineers largely maintained confidence in these models from their inception until they faded into the background.

Many in academia know it should operate more like BBN, CMU, or GE research. They just can’t change the structures to make it happen. Many R&D leaders know that their company should think more in 10-20-year time horizons when planning research expenditures, but shareholders and shareholder-wary CEOs often do not find this view actionable. Answer.AI can be free from all this. The org can apply the great old models in a new era on new technology.

(Ries being a cofounder of Answer.AI should be comforting. Ries has railed against the types of myopic management trends that make activities like maintaining an expansive industrial R&D lab difficult for firms. He founded the young Long Term Stock Exchange, in part, to help mitigate issues like this.)

Conclusion: Running Your Own Races

It’s up to Answer.AI’s two-headed management team to stick to the organization’s comparative advantages. There is no need to race academic CS researchers or corporate R&D departments in races those two groups feel incentivized to run. Corporate R&D, NSF-funded CS researchers, and AGI-focused labs all have areas in which they clearly will and won’t operate. And those areas don’t come close to covering 100% of the good ideas somebody should obviously be working on. Answer.AI is free to run its own races, uncontested.

Many will feel the org is using an untested model. However, Answer.AI’s founders — like me — believe that this model is proven, but has simply gone away for a while. So, shouldering this “organizational risk” that others are seemingly not willing to do, they have the chance to work on problems without much competition. If their $10 million experiment works, it has the chance to spark a rush of emboldened researchers and engineers to found small research firms, leveraging the models of the once-great dragons of American industrial R&D.

I wish Jeremy and Eric luck in the early stages of their mission. To any researchers and engineers — across all areas — reading this and wishing a BBN, CMU, TMC, or Answer.AI existed in your area, please reach out to me on Twitter. I’d love to see if there’s anything I can do to help.

Thanks for reading:)

Subscribe now

Share

As a preface to the interview, I wrote the following ‘Introduction’ section to highlight why CMU is a singularly important department in autonomous vehicle history. I leave the reader with three options as to how to consume the rest of the piece after reading the introduction. For the bookworms, I have provided an abridged transcript of my meeting with Chuck Thorpe — which picks up after the introduction. For the audiophiles, the full audio of the interview is available — both on Substack and on Spotify. And for the YouTube afficionados, I share a YouTube link of the Zoom interview.

Subscribe now

Share

Introduction

The first piece in the all-time great DARPA contractors trilogy explored the importance of BBN’s unique structure, embodying the “middle ground between academia and the commercial world.” BBN did so well in this role that it earned itself the monicker “Third Great University of Cambridge.” BBN, as a firm, demonstrated what was possible if a firm staffed itself with the very best engineering researchers and cared deeply about technical novelty more than profit maximizing. The flipside of the coin is CMU and its AI work in the later-1900s. Much of CMU’s AI work in this period answers the question, “What if an academic engineering department made its comparative advantage novel systems contracts?” CMU was an academic department eagerly embracing a style of work more common to the defense primes in the DARPA ecosystem.

In DARPA’s Autonomous Land Vehicle project — covered in the ALV section of the ARPA playbook — CMU stood out as the lone contractor involved who was excited to take on both the burden of novelty and the burden of systems integration work. The academics and the project’s main commercial firm had very different incentives when it came to these issues. As I wrote in the earlier ALV piece:

The contractors who found themselves working on the reasoning system vs. the vision system had very different interests and incentives. Most of the reasoning contractors were commercial firms and had the incentive to fight for a broader scope of work and more tasks getting assigned to the reasoning system because that meant better financial returns to the company. Most of the vision contractors, such as the University of Maryland, were academics and largely content to let that happen…if it meant they got to focus primarily on specific vision sub-tasks that were more in line with the size and scope of projects academics often undertook. For example, the private companies felt that building up much of the systems mapping capabilities, knowledge base, and systems-level work should fall to them.

CMU and one of its project leaders — Chuck Thorpe — seemed to be the strong voice of dissent on many of these matters from the academic community. Ever since CMU had become a major contractor in DARPA’s Image Understanding work, it had taken a rather systems-level approach to attacking vision problems — working with civil and mechanical engineering professors like Red Whittaker to implement their vision work in robotic vehicles — rather than primarily focusing on the component level. The CMU team far preferred this approach to things like paper studies — in spite of the increased difficulty.

As the ALV project wore on, Martin Marietta — the lead systems contractor for the ALV — gradually succumbed to its ‘demo or die’ pressure. As the frequency of demos increased, Martin became much more concerned with technically hitting the benchmarks DARPA specified than pushing forward truly ambitious technological changes. Additionally, many of the academics focusing on vision research concerned themselves more with using their DARPA research funds to improve their performance in somewhat simplified environments that helped them write research papers than to do research that would directly plug into the ALV. With this equilibrium becoming clear, Ronald Ohlander — a former student of Raj Reddy at CMU and the DARPA PM responsible for funding vision research — designated CMU as the systems integrator for his DARPA SCVision portfolio. The Martin Marietta team clearly did not care enough about integrating and testing cutting-edge algorithms. So, he tapped CMU and gave them the funds to properly integrate insights from the research ecosystem.

In the prior piece, I described the situation as follows:

CMU had already, unlike most of the component contractors in the SCVision portfolio, been testing its component technology on its own autonomous vehicle: the Terregator. CMU’s used the Terregator — for often slow and halting runs — to test the algorithms developed for its road-following work. The Terregator proved extremely useful to this research, but its use had also helped the CMU team learn enough to outgrow the machine.

In CMU’s ongoing efforts to implement sensor information from a video camera, a laser scanner, and sonar into one workable vehicle, the CMU researchers had come to realize that they needed funding for a new vehicle. CMU needed a vehicle that could not just accommodate the growing amount of sensor equipment, but also one that could carry out its tests with computers and grad students either on board or able to travel alongside the vehicle. This would allow bugs to be fixed and iterations between ideas to happen much faster. The vehicle that the CMU computer vision and robotics community was casting about trying to get funding for would come to be known as the NavLab. Fortunately, what they were seeking money for seemed to be exactly what Ohlander felt the SCVision portfolio needed. The funding for the NavLab began around early 1986. The funding set aside for the first two vehicles was $1.2 million, and it was estimated that any additional NavLab vehicles would cost around $265,000.

As Martin was somewhat frantically taken up with its demo schedule, CMU, with its longer time windows, was free to focus on untested technology as well as technology that ran quite slowly but seemed promising in terms of building more accurate models of its environment. As the CMU people saw it, if anything proved promising they could upgrade the computing machine used to operate that piece of the system later on…the vision researchers likely figured that a piece of hardware powerful enough would likely exist soon to help push the idea along — even if not in the next year. Lacking the hard metrics of yearly demos with speed requirements loosely matching how fast the vehicle would need to go in the field, the CMU team could afford to think more along the lines of “how do we build a machine that is as accurate as possible, even if it moves really slowly today?”

As CMU took up the mantle of building SCVision’s true test bed vehicles, it also took over the role of SCVision portfolio’s true “integrator.”

Things continued like this for a while. Then, after a successful ALV demo in 1987, Martin’s role as the prime DARPA autonomous vehicle contractor suddenly ended. This abrupt shift happened after a panel of DARPA officials and technology base researchers met to discuss phase two of the program. Roland and Shiman wrote in their book on this era of DARPA computing:

Takeo Kanade of CMU, while lauding Martin’s efforts, criticized the program as “too much demo-driven.” The demonstration requirements were independent of the actual state-of-the-art in the technology base, he argued. “Instead of integrating the technologies developed in the SC tech base, a large portion of Martin Marietta’s effort is spent ‘shopping’ for existing techniques which can be put together just for the sake of a demonstration.” Based on the recommendations of the panel, DARPA quietly abandoned the milestones and ended the ALV’s development program.

The CMU researchers and the NavLab — which were remarkably cheap compared to Martin Marietta — continued to be funded. In this way, CMU took its place as DARPA’s top systems integration contractor for driverless vehicles. And it would stay on top of the heap for several decades.

The CMU driverless vehicles were responsible for many — some might even say most — of the practical driverless vehicle breakthroughs in the field's early years. Much of CMU's earliest AI work on these vehicles revolved around the use of things like radar and laser sensors, path planning, and finding ways to integrate inputs from different models into a single set of instructions. But, in the late 1980s, CMU's work abruptly became quite recognizable to the modern eye. In 1988, then-CMU grad student Dean Pomerleau successfully integrated the first neural net-based steering system into a vehicle — his ALVINN system. In 1995, Dean and fellow grad student Todd Jochem drove the successor to Dean’s ALVINN — the more complex RALPH — across the country on their “No Hands Across America” tour. The RALPH system employed neural nets but also put much more effort into model building and sensor processing. Holding Dean and Todd, the upgraded NavLab was able to autonomously drive 98.5% of the way on its successful cross-country journey. In a sense, much of CMU’s work culminated in the CMU vehicles' performances in the second DARPA Grand Challenge, held in 2005. Many only remember that Stanford's team won this landmark challenge. But CMU's two vehicles finished a close second and third to Stanford’s vehicle.

At the helm of Stanford’s team was Sebastian Thrun. Thrun had, the year before, left his CMU research position — which he had held for almost ten years — before joining Stanford and leading their team. In the early years of the field, an astonishing proportion of the best researchers were either trained at CMU or were long-time professors there. In a 2010 oral history, Thorpe described Sebastian Thrun and many of the early Google driverless car team’s CMU ties, saying:

Now, it’s the Google group, and we’re very grateful to Google, but we think of it as the Carnegie Mellon West group. Sebastian, of course, used to be here, and then went to Stanford and Google. Chris [Urmson] is a Carnegie Mellon person on leave. James Kuffner is a Carnegie Mellon faculty member on leave. If you go down through the list, they – everybody except for one on that group either is a Carnegie Mellon person or has a Carnegie Mellon heritage, so that’s our boys. [laughs] And we’re delighted that they’re off with Google, and Google’s given them the researchers to do really cool stuff.

Carnegie Mellon has its fingerprints all over the breakthroughs and researchers from the early decades of autonomous vehicles. It would be hard to argue that CMU’s efforts didn’t move forward the timelines of this field by some number of years. It also seems clear, looking at the history, that the incentives and management structures CMU used to manage projects like this played a major role in setting the university apart.

The university clearly had a comparative advantage in this style of work. So, I decided to ask Chuck Thorpe some questions about how exactly CMU did what it did.

Chuck Thorpe Interview

In the following transcript, I made some cuts to make reading the text version of the interview more efficient. The audio and video files contain the interview in its entirety — which is about 15% longer.

Spotify podcast link

Finding his way to CMU and the early years of the Robotics Institute

Thorpe opened the interview by detailing how exactly he found his way to CMU as a grad student and how CMU’s approach to computing research set it apart from many other universities at the time.

Thorpe: So, I showed up wanting to work on robots. They gave me a book and the robots were all these robot arms and I thought, “Well, that's not what I want to do.” I want to do mobile robots. The reason I came to Carnegie is I was backpacking with my professors from North Park.

One of them said, “What do you want to do?” I said, “Well, artificial intelligence.” “Oh. You ought to go to Carnegie Mellon. Herb Simon just won the Nobel Prize for inventing AI.” I'll tell you the truth. I had to look up Pittsburgh on a map. I had to go to the library and find out what Carnegie Mellon was. And Carnegie Mellon in the seventies was reinventing itself.

It was a very good Western Pennsylvania engineering school that looked at the demographics of Western Pennsylvania and said, “We're going to be a very small, very good Western Pennsylvania engineering school,” and decided that they needed to go national. Now “national” meant that in 1972, once a year, they would go to Chicago to try to recruit students. And every other year they would go to DC.

So, Dick Cyert, the president, bet his presidency on turning Carnegie into a national brand and really expanded it and grew it. He also, Cyert, was a professor of business and his doctrine was comparative advantage. Find out where you can be one of the top two or three in the world and go for it — and don't try to compete broadly.

And a mid-sized university like Carnegie, that makes a lot of sense. So Raj Reddy was talking with Cyert and convinced Cyert that it was his idea, Cyert's, not Raj's, that we could be top two or three in robotics. And Raj said, “But I'm not interested in playing catch-up small ball. If we're going to do it, let's go big.”

So Cyert asked Raj to run the Institute. He got a million-dollar grant from the Navy and he got a million bucks from Westinghouse. And that was the origins of the Robotics Institute.

Gilliam: And how important was it for him to get a big industry contract and a big public sector contract? Was that specifically the goal? To make sure they had one from each?

Thorpe: Specifically the goal was to do both, really crack into ONR (Office of Naval Research) and then into DARPA because they had a lot of money to do very interesting things. But, also, Carnegie has these deep roots in Western Pennsylvania. And if we could work with Western Pennsylvania industry and really help Westinghouse succeed that was a win all the way around.

This was at a time when the steel industry was visibly dying and trying to figure out what was going to save Western Pennsylvania. They thought it was Westinghouse. It turned out that was not completely the story. Westinghouse had its own financial problems. But that notion of trying to see what we could do locally, what we could do with industry, as well as tapping into the US Government was a big deal.

I, then, interjected to provide some historical context and ask a question.

Gilliam: It's interesting to hear you talk about the comparative advantage, because even Karl Compton, who ran the Princeton physics department, was MIT President for a while…he, at one point, wrote a letter in Science in the 1940s. Essentially, he was trying to be polite as he wrote it, but he said (roughly), “We see the talent going into some of the industrial R& D labs from the physics departments. We feel like, to the man, we have higher average talent, but we are not really touching what they're outputting.”

He thought that the firm structure and the concept of comparative advantage, organizing teams in a way that made sense, is what departments should do. It's what made sense. The concept of 20 different budgets [one for each professor] and 20 different fiefdoms [lab groups] to do small research was silly. It made no sense. It's for pure theoretician-type people. But he later became an MIT president and could not succeed at that.

But it seems like Carnegie in running a lot of this work — it seems like Alan Newell was calling it project style work, the Raj Reddy style of work — it seems like Carnegie succeeded in a lot of that ethos. Would you say that's fair?

Thorpe: So, a lot of this starts with Herb Simon saying, “What can we do really well?”

Herb was a founder of the business school, was a founder of the computer science department, was a founder of cognitive science (the psychology department). Herb then brought in Newell. And the two of them saying, “How do people think and how can we encode that in a computer?”…that sort of set the framework for so much of what has made Carnegie successful.

That whole AI and cognitive psych paradigm…if you look at the growth of the computer science department and then school, there's very good theoretical computer science, but the rest of it is all “AI+”. Yeah, AI + linguistics and you get machine translation. AI + learning and you get computational learning. AI + human interaction and now you get the Human Computer Interaction Center. AI + mechanical engineering and you get robotics.

So, it was the combination of leadership from people like Cyert saying, “I'm going to invest a little bit of money here.” And the intellectual leadership of Simon and Newell saying, “This is going to be a growth area.” Simon and Newell, in turn, hired Raj Reddy, poached him from Stanford. That group of them hired [Takeo] Kanade, poached him from Kyoto. And that sort of set us off and running.

The importance of Raj Reddy, what it means to be ‘project-oriented’, and how CMU utilized its grad students

Gilliam: Could I read you an excerpt from Alan Newell's oral history where he talks about Carnegie Mellon's project-oriented style and Raj Reddy?

Because he uses some terms that I'm sure you're going to hear and you're going to go, “I know exactly what he means!” But to me…I knew what I thought it meant — or maybe it's what I wanted it to mean — so I'd love to run it by you. So I'll read you two remarks and then I'll lead you in with the question.

Alright, so the first remark is:

NORBERG [Interviewer] : You had the money along with Green and Perlis [talking about an extra injection of DARPA funds]. What did you people decide to do with it?

NEWELL: I don't know... support people... just spend it. We didn't decide to do anything with it. One of the features of this environment was that it was decidedly un-entrepreneurial. That seems, in one respect, like a contradiction in terms, but we never took these funds and decided we were going to go out and do big things with these funds. That was an attitude typified the operation around here up until Raj [Reddy] shows up.

And remark two is in reference to the 1970s with DARPA. There were a lot of changing political tides, as you’re aware, and they’re talking about the changing goals of DARPA when DARPA is giving out funds. Newell says:

Newell: ...And so, therefore, we have been pushed towards being applied. A response I have, which is not an attempt to change that, that is exactly what's happened. CMU has gone along with that in spades. We started going along with that in the 1970s. I can remember having conversations with Howard Lackler as we were sitting there under the fixed funds with inflation and still trying to get an extra PDP-10. We were allowing that environment to shape us up so that we were not in fact anything like? And we became project-oriented all in that period. We were not project-oriented in the 1960s at all.

…
Newell:..The corresponding thing is the cats that want to play that game -- the Raj Reddys, the Canatis, and so forth, the Coons -- [find] the game they play is how to do that and do basic science at the same time within the same budget.

So I guess my initial question with that is, as Newell’s talking about ‘project-oriented,’ what does it mean to be project-oriented? How does that differ from business-as-usual AI research on a DARPA grant, the way you might see in a Marvin Minsky lab or something like that?

Thorpe: Oh, so Minsky is a very interesting contrast. Minsky wrote this book, The Society of Mind, where you could read…it was, I don't know, 50 two-page chapters. And you could read it in any order and made just as much sense in any order as in any other order.

That's kind of the MIT notion [presumably Minsky’s MIT AI group] of how ‘mind’ works, and their notion of how research works. It’s that you've got 50 independent projects and, together, something may emerge from that.

Carnegie shows up with people like Raj, who says, “I want to conquer speech. I'm going to put together a big team, and these people are going to be working on the phoneme level, and these people are going to be working on the lexeme level, and these people are going to be working on the syllable level, and these people are going to be working on the word level, and these people are going to be working on the sentence level. And there's going to be an architecture which connects them up and down. And we're going to have to have specialized hardware for all of this to work on.”

And so, Raj says, “Let's build this, this project.” And it was a project that had 16 different dimensions, all of which came together. Now, Raj was also flexible enough to say…when somebody showed up and said, “You know, I've got this idea for a completely different method. Let's not bother with all of the structure. Let's just have a little system where everything sort of learns and it figures out all of those mechanisms.” So that was Hearsay (demo), and that was Harpy (demo), and that was the two different systems.

That competition between big structured AI and machine learning, data-intensive AI has been going on at Carnegie for the last 50 years.

Gilliam: To get very in the weeds here…you talk about putting together a big team. And there's a lot of grad students involved in that. And it seems like on the one hand, ‘project-oriented’…it almost sounds like more of a firm structure than an academic team. A lot of the grad students are the employees. But there seems to be some kind of balance where the grad students are the employees — for example, with the ALV or NavLab where you had “on board grad students” — but also the grad students need a thesis.

With a thesis like yours [working to improve the operating speed of a sensor already in use on a test vehicle]…I could see how your thesis would have folded into a big team with that project's goals quite well. Because you were working on a specific sensor technology, and with Blackboard you could fold all that in. But then there's somebody like Dean Pomerleau [whose 1988 ALVINN system implemented neural net steering on CMU’s vehicle] on the other end of things. It's unclear how his project can mesh with everything.

Can you talk about how those management decisions are made? Because I guess the big goal of what I'm trying to do is to help people understand what the CMU approach is on a management level. So this would be great.

Thorpe: So let me back up and say a couple more things about the big CMU picture, and then I'll talk about Dean and how we organized the NavLab, in particular.

The big CMU picture was fundamentally changed when DARPA sent a telegram —this tells you how long ago it was — saying, “We've got a million bucks to spend. If you can send us a 350-word proposal by the end of the week, we'll send you a million bucks.” (~$5-$10 million today)

And that was the grant in basic research for computer science. And that grant lasted for, I don't know, 15 years or something like that. And even after that grant disappeared, that set the ethos of the computer science department. There's this one big chunk of money that supports all of the graduate students. Therefore, the graduate students are jointly sponsored by the entire department.

Do you know about Black Fridays?

Gilliam: No.

Thorpe: Twice a year, the entire computer science faculty sits down and talks about every graduate student. And the student has written a letter. And they [the faculty] read the letter. And they turn to the advisor and say, “What has your student been doing?” And if the advisor says, “Well, the student is making good progress and they did this and this and this,” and it matches what the letter has said, good. Then the student gets a letter saying satisfactory progress.

If the faculty member says, “You know, I've been trying to work with this guy and he's just so stubborn, he won't do what I tell him to do,” then the faculty write a letter saying, “You must, by next Black Friday, do this and this and this.”

If the student is co-advised and the one advisor says, “Well, I thought that Eric was working on it,” and Eric says, “Oh, I thought Chuck was working on it.” Now the faculty turn on the advisors and say, “You bozos, you have to get your act together and do a better job of advising this guy!” But it prevents one student from being abused by their advisor. If you're not happy with your advisor, because you're sponsored by the department you can change your advisors. But it also prevents one soft-touch advisor from saying, “Aw, poor student, he had a rough semester.” No, the entire department gets together and writes something. And there are these very carefully crafted letters of satisfactory progress, unsatisfactory progress, the dreaded ‘n minus one’ letter, and then being kicked out of the department.

So, the ethos is that we're all in this together; that we've got some joint resources; that the graduate students are joint resources; that we need to properly take care of these students and properly supervise them. So that gives the graduate student a large amount of flexibility. But this responsibility to not just their advisor, but to the entire department.

So that's great because students can find interesting projects. There is the marriage process where you walk in day one and each advisor says, “Here's what I'm up to. Here's how many students I'm looking for.” And then the students get to pick who they would like to work for. And the marriage process matches advisors who have interesting projects with students who are interested in working on them…

That's very different from almost any department that I've seen any place. Almost any place you're an indentured servant to the faculty member who happens to have graduate support. And if you don't get along with your advisor, you have the burden of seeing if there's somebody else who has money who will support you and seeing if you can switch advisors.

So that has set this attitude that we're all in this together. We'll have individual little projects, but we'll also have some of these big projects — like the speech projects — that a lot of people will work on.

Newell himself, at his height, had both his human-computer interaction/user interface stuff, but also his SOAR project — which was a pretty good-sized project with a number of graduate students working on it — trying to figure out how they could all fit together.

Newell said that, “You can do science in lots of different ways. One of the ways you can do it is just by doing something that's an order of magnitude bigger, or an order of magnitude faster. If you can process ten times the number of rules, then you're in a fundamentally different space, and you're doing something different, and the science is different. If you can run your computer ten times as fast, then the nature of the problems that you're dealing with changes in ways that you can't even predict.”

So, doing something 10 times bigger is great. So, he was very supportive of that kind of an attitude.

The Origins of the NavLab

Thorpe: Now, how did we get going on the NavLab? So I was sponsored on the original ONR project to do cool stuff. And one of the cool things that we did was to build Takeo’s direct drive robot arm. Another thing we did was to build this stereo vision for navigating across a room.

Side notion. We were sitting there working one day — graduate students, T-shirts, empty pizza boxes. And we heard the door open and we looked over and there was Raj and President Cyert arguing over who could open the door. And in walks, this Navy admiral in full dress whites with all of the…Nobody had told us the head of ONR was coming to visit…

Larry Matthews, who's now Head of Computer Vision at JPL, Hans Moravec, myself…with robots and pieces. And in walks the head of ONR to see what he's been spending money on. Like, “Guys, if you told us we would have at least cleaned up the lab and probably had a demo ready to run.”

Gilliam: Or put on your good T-shirt.

Thorpe: And put on my good T-shirt! But he was impressed. And many years later I ran into him and he remembered that visit. And so he was happy to see robots.

When I was finishing my thesis, we had a meeting and Takeo asked each of us, “What do you want to do next?” I said, you know, we've been doing a good job indoors. I want to go outdoors and really build driving cars. And when we had gone around and said what we wanted to do next, Takeo came back and said, “Good, because DARPA has this money for Strategic Computing. They're going to be building the world's best computers, and then they need ways of showing it off.”

And, being DARPA, they have to have a way for the Air Force, and a way for the Army, and a way for the Navy… (Thorpe then goes on to explain the Battle Management and Pilot’s Associate systems covered in a prior piece.)…And the Army wants to build a robot scout. “So Chuck, you're in luck because what you want to do next is exactly what DARPA is going to have money for. So let's write a proposal for it.”

We had just hired a guy named Duane Adams. Duane had been Deputy Director of one of the DARPA offices — IPTO. And he came to be the Associate Director of Robotics and to help us write proposals. So, Duane said, “Chuck, start writing.” And we wrote a proposal for what it would take to do road following. And that was the beginning of the NavLab project.

So we submitted the proposal, one for road following and one for the architecture of what the system would have to look like. I went back, finished my thesis, and defended my thesis. I thought I would take a couple weeks off to figure out what I wanted to do next. Raj came up and said, “Congratulations, you passed your thesis. There's a meeting in my office to talk about how we start the NavLab project.” And that was my gap between my thesis and the NavLab project, walking from the defense room into Raj's office to start this meeting, to start planning what we were going to do.

Departmental decisions that made big systems projects feasible

Gilliam: To stop you for a quick second to add to the queue of operational questions. So I have the question of 1) grad students as ‘employees vs. novel thesis generators.’ And now 2) in one of the DARPA year-end reports for the NAVLAB from CMU, I saw them refer to you as a ‘project manager.’ Which is interesting…that's not an academic designation. That sounds very industry. So I would love for you to talk a little about that, too. If I should read into that or what that means to you.

Thorpe: Shouldn't read much into that. But what you should read into is when Newell, Simon, and Reddy really got computer science going — and before that Perlis and Shaw, etc…They said, “If we're going to do big projects,” — and we did big projects like the Andrew Project, that turned into the Andrew File System and things like that. They said, “We need to have more faculty than we have courses to teach. We can support them on soft money, so we need to have research scientists who have full faculty privileges.”

So if you're at MIT and you're not tenure track, you're nobody. Carnegie said, “No, we want to have non-tenure track people, supported on soft money, but who have advising privileges and PI privileges and all of the privileges that faculty have. They just may or may not be teaching courses at this time.” So, unique to computer science, there's this whole research track which is fully equivalent to the tenure track. And that was my underlying title. When I defended my thesis in October, Raj said, “I can't make you a faculty offer. You're going to be a postdoc until July 1 and then I'll make you a research scientist.”

So I came up through the research ranks after having spent that eight months or so as a postdoc. And it's partly that business of trying to figure out what research scientists were that caused computer science to first break out of the Mellon College of Science and be a free-floating department. And then form themselves into a school so they could really have this parallel track.

‘Project manager,’ that was just sort of a title because that's what I was doing on the NavLab project. Raj took me aside. He said, “I don't care how many papers you write. I don't care how many awards you win. That vehicle has to move down the road. If you do that, you're good. I'll defend you. I'll support you, I'll promote you.”

Because I was a graduate student writing this proposal, I couldn't be a PI. So Takeo was a PI and I think Hans Moravec was a PI. And then as I became a faculty, I became a co-PI and then eventually the PI. But it was pretty clear that Takeo was interested in the project because it was a cool project. He was also interested if he could have a few people working on individual projects. But he didn't want to worry about scheduling stuff, and who got time on the machine, and writing reports. So, he was very happy to turn all of that over to me and let me run it the way that I wanted to run it.

Gilliam: And how was that? Can you describe your approach? Let's say, describe your approach to…somebody who knows how an academic lab works usually, but doesn't do anything like this. They output papers, they play the h-index game a little bit, etc.

Thorpe: Well, you pointed out a very key feature here, that the whole system has to work and that each student has to have something that they can claim as their own and that they can write up as their own thesis topic.

So, we got ready to put the system together and I said, “How are we going to see the road? Jill [Crismann], how do you want to do it?” Oh, well, she wanted to work on electrical engineering, signal processing, color classification scheme. “Great, you go do that. Here's what your interface needs to look like. You need to tell me where the road is in the image. If you tell me that. Someone will take that and will steer the vehicle.”

“Karl [Kluge], do you have any ideas?” “I think that we ought to be able to track edges and track color features. You ought to be able to track yellow lines and track white lines.” “Oh, that sounds like a good idea.” So Karl went off to do that. Great. “If you can tell me where the road is in the image, then I'll have someone figure out how to steer it.”

“Tony [Stentz], what do you want to do?” “Well, I really want to steer the vehicle.” “Good. You can do the path planning for that kind of thing.”

“Martial [Hebert],” he was a postdoc who came over from France, “what do you want to do?” Well, he was a 3D vision guy. “Cool! They're sending us a laser scanner. Do you think you could use that to determine where obstacles are?” “Oh, great. If you can find out where obstacles are, then feed that to me.”

“Jay [Gowdy] what do you want to do? I think you're a good systems guy. If someone tells us where obstacles are and someone tells us where the road is, could you figure out how to put the whole system together and give them useful user interfaces?”

So, really trying to build the system in a modular way, so that each person had something that they could do and be proud of. And if it worked, great! Then we plug it into the system and it runs. And if it doesn't work, well, I've got two other people working on other approaches to that same thing.

Gilliam: And if somebody like Dean [Pomerleau] came in with neural nets — wanting to apply Geoff Hinton's somewhat theoretical methods to this — how do you fold them in? Do they sit off a little further to the side in the lab? What does that look like?

Thorpe: Well, first of all, we said, “Neural nets? Oh, come on, nobody's ever built a neural net that does anything. Haven't they proved that neural nets aren't going to work? Um, you want to try it? Sure, try it. Let's see what happens. What would you like? You want some images? We'll collect you some images. Come on in. We'll see.”

And then talking to our people at DARPA, because there was a different and competing office at DARPA doing neural nets, and saying, “Look, I know you're skeptical about neural nets. We've got this neural net guy, separately funded. I want to give him a shot on this. You don't mind if he uses our robot?” “No, that's fine. We'll disprove neural nets once and for all. So come on in and try it out.”

Dean turns out to be an extraordinarily bright person. And you wouldn't know it to meet him, but he's extraordinarily competitive. And he's willing to get his fingers into everything. So, mostly what he wanted was this recording of images and steering wheel position. And then he went off and started to work on that and built a system with 100 hidden units and it seemed like it could work, but really slowly. And so he cut it down to 50 hidden units and it seemed like it could work as well, and a lot faster. He cut it down to 25 hidden units. And he eventually got down to four hidden units, and now he was working in a believable amount of time and it still worked pretty well. He cut it down to three hidden units and it didn't work anymore. So, okay, four hidden units.

Back to this theme of Strategic Computing and DARPA building supercomputers. One of the supercomputers they built was at Carnegie, and it was a thing called the Warp machine. The Warp machine had ten pipelined units, each of which could do ten million adds and multiplies at once. (The Warp machine and BBN’s butterfly processor were briefly covered in an earlier piece in the ARPA series.)

H. T. Kung was working on systolic computing, where things just sort of chunk through the pipeline. And we said, “Here's this big box. Because we're sponsored by Strategic Computing, we'll put it on the NavLab. Can anybody figure out what to do with it?” “Well, it's a really complicated thing and really hard to process.” But Dean could figure out what to do with it. Partly because Dean is a really, really smart guy.

Partly because the inner structure of a neural net is, “Multiply, add, multiply, add.” You take these inputs, you multiply them by a weight, you add them together, you sum it up, run it through your sigmoid function. So, Dean was, as far as I know, the only person… well, that's maybe a bit of an exaggeration. Some of the speech people were able to use the Warp also, because they were doing similar kinds of things…But Dean got the full hundred megaflops of processing out of the Warp machine! And then he put it onto the NavLab and we got the whole thing to work. So, after that, Jill got her thesis for doing her thing. Carl got his thesis for doing his thing. But Dean's neural nets seemed to work in most situations better than either Jill's or Carl's one. And so that's the one that got all the attention.

Going from ALVINN to RALPH and CMU’s complementary systems projects

Gilliam: What did you consider a bigger practical breakthrough? There's obviously the ALVINN system, which to a modern machine learning engineer’s eye, would give them a little tear. “Wow, it's so minimalistic It can do all this (with mostly just neural nets).” And then RALPH comes around. It’s a little more manual. Did you consider RALPH a much bigger practical step? Or once ALVINN came around, did it seem somewhat clear to you that something like RALPH would be possible and better?

So that was part of it, but the other part of it is..how do you take something like ALVINN…which at its core is just turning steering wheels. It's not doing any representation of the world…how do you take that and merge that with obstacle detection and avoidance? How do you take that and merge that with a map? And so, layering architectures on top of this that can take something which shouldn't really have a geometric representation and extract some sort of a geometric representation so we could do other things with it. That was really the fun part of it.

Gilliam: Is that when it became a proper, all hands on deck, CMU-type project — going from ALVINN to RALPH? It sounds like ALVINN was not a one-man job, but smaller. But then maybe RALPH became a proper CMU systems effort.

Thorpe: So Todd [Jochem] came along and worked with Dean. And part of what Todd did was to say, “How do you get some geometry out of this so that you can train multiple different neural nets and switch between them.” So that's even separate from RALPH. That was still in the ALVINN stuff. How do you train this thing up and have things work together? But if you look at the NavLab '90 videotape where we're going from my house to Keith's house, that was the obstacle detection and stopping. That was landmark recognition. That was the annotated map saying when we're supposed to do what. That was high-accuracy dead reckoning when we were doing a sharp turn and ALVINN couldn't work. Getting all of those system pieces to work together was a lot of fun. And this is on top of an experimental vehicle where you never knew what things were going to break.

Gilliam: I’d like to put something on the record, and you let me know if I'm wrong. So I think if some of my readers read what you said, they would say, “Oh, how fortunate that CMU had this systolic array machine going on and that it happened to be able to — obviously some computer structures were somewhat specialized back then — that it happened to be able to work with the natural language project and the vision project.”

But my reading of the history is that this doesn’t seem like circumstance. Now, this was a period when a lot of people weren't pursuing building machines at universities anymore. Obviously, this is not long after Slotnick [and the ILLIAC IV machine] at U of I. And Slotnick's gone on the record some years before saying “The era when you could build big, good machines at a university is over. Like, “this is for industry now, it's not for universities to do.” It seems like CMU’s systems approach helped prolong the era where you could build a good workable machine at a university.

Also I read that the Warp folks made sure that their machine could work with the other ongoing CMU systems efforts like vision and like natural language. Is that fair? Or is that me giving too much credit to the CMU structure?

Thorpe: So the CMU structure, when I got there, they had just finished building CM*, and before that C.MMP. CM* was a powerful array of 50 PDP-11s. But the difference between C.MMP and CM* was they were playing with…if you have multi-processor systems, do you connect a big crossbar switch with every memory connected to every CPU? Or do you connect them in some sort of a hierarchy? And so this notion of building big systems, big hardware systems, was there.

And of course, computer vision has always been a hog for CPU cycles. So in the early days, they built these systems and then they said, “Computer vision people and speech people, have at it!” Because the computer vision people and the speech people knew that we needed the cycles. And we were willing to put up with a fairly klutzy interface in order to see if we could take advantage of these sorts of things.

So, that ethos was there. The names C.MMP and CM* come out of the Newell and Siewiorek book. So here's Al Newell, a pioneer of AI, writing a textbook on computer architecture because he understood that AI needed multiprocessors in order to work. So that ethos was there.

We had a guy in the computer vision group named Rick Rashid, who was working on how you make the hardware more efficient, and how you make the software more efficient, and how you build microkernels. And he's the guy who went out and started Microsoft Research. So there's this long history of computer vision people having to have the highest possible processing power and being willing to put up with a fair amount of klutzy-ness in order to take a wire-wrapped prototype and really use it for for high performance.

Concluding Thoughts: CMU’s applications-focused culture, its origins, and the legacy of its grad students

Gilliam: This has all been fantastic. Is there anything else you think people should understand about how CMU managed its projects? It was a quite exceptional contractor there for a while…To me, CMU in this era seems like one of the all-time great DARPA contractors. Right up there with BBN in the ARPAnet era and Skunk Works when it built STEALTH. In many ways, CMU seems quite similar to BBN.

(Here, Chuck and I went on a 5-10 minute tangent discussing some of his friends from BBN, DARPA’s Clint Kelly, and others from the DARPA Strategic Computing era. Check that out on the audio if you’re interested. But it’s been skipped in this abridged version.)

Thorpe: …Back to Carnegie. Carnegie had a couple of aspects of the culture. One is that doing big things and doing practical things makes a difference. So if you look at what Carnegie has done…Duane Adams, his first proposal he wrote was for the NavLab. The second proposal he wrote was for the Software Engineering Institute. Running an SEI, that's not a typical academic thing.

But there are places like Berkeley. Berkeley runs the Berkeley labs and the Lawrence labs. So running an FFRDC (federally funded research and development center) is something that a handful of universities do, but not everybody. You don't get a lot of published papers out of that, but what you do is you provide a venue to take the basic research you've done and to turn it into practical things which change society.

So the SEI does big software engineering projects for DOD, but they also do a lot of training. They also run the CERT (Computer Emergency Response Team.) So, when some new virus breaks out, SEI jumps in and tells the world what the virus is and how to fix it. And that's the kind of thing that is a little bit unusual for a university to do. But Carnegie had the culture to say, “Making a real impact in society is important. And this is one way you can do it. Let's do it separately, off campus, so we don't dilute the graduate students and writing the papers, etc. But let's have this pipeline.”

Within robotics, NASA came to us, I don't know, 30 years ago and said, “NASA is spending a lot of money on robotics. We're not getting good PR out of it. Partly because our robots are, they go off to places like Mars. Could you take the technology that NASA is developing and work with people like John Deere?” So we formed the Robotics Engineering Consortium. It sits off campus. It employs more staff and fewer graduate students. It can do confidential projects. And it takes basic research done on campus and turns it into practical projects that Deere can take and turn it into products.

So that whole notion of a halfway house to take basic research and turn it into something bigger, something more practical, something applied…that this is a good thing for a university to do…that's not a universal university ethos.

Gilliam: Two-part question. The first is: why do you think CMU was able to maintain that ethos for so long? And the second part is interesting because, now, you’re a university president. You have the benefit of all sorts of institutional knowledge. Why do you think schools struggle to manage projects this way, if it seems, in many ways, to be a natural default approach for technical subjects?

Well, let me give you two reasons why Carnegie succeeded in doing that. The first one, I think, is really more Newell than Simon. Simon wanted to do great intellectual projects. Newell wanted to build things. So Newell caught on that building big stuff and seeing how it worked in the real world was important. And then Raj pushed that even further, starting out with speech. But then also the hardware projects etc. The other reason why Carnegie did this…if you go back in the history, there was the Carnegie Institute of Technology, and there was the Mellon Institute of Science.

The Mellon Institute of Science was the contract research shop. When the Mellons owned Gulf Oil, an aluminum corporation, and big chunks of various steel companies, they needed a chemistry research shop. And so the Mellon Institute was this chemistry research shop, and it existed to do contract research. If you got to produce some papers, that was good, but it was 50 Ph.D. chemists doing industrial research. In the great merger of ‘67, which formed Carnegie Mellon, we absorbed the Mellon Institute and had to figure out what to do with it. And partly what we did with it was to keep it running as a contract research shop.

So, there was already this notion of an off-campus, sponsored by soft money, more practical research shop that we didn't quite know what to do with. But in an ideal world, you would take basic research from the main campus and then figure out how to apply it, etc. So that was a little bit in the DNA of Carnegie since the merger in ‘67.

Gilliam: It's not dissimilar to MIT circa 1920. It had a program called the Technology Plan. For context, when MIT was young — from 1860 to 1920 — they were perpetually poor and living on shoestring budgets. Back then, more than now, MIT really lived on the ethos of, “We serve industry above all else. That’s what we’re here to do”. So, the Technology Plan came about because, at a certain point, they said, “The most honorable thing an institute of technology can do is provide a great service for a fair price.” So they let their applied researchers loose to work a lot of these contracts to the point where, at a certain point, their chemical engineering/applied chemistry department was 75-80 percent industry funded.

So this is very interesting.

Thorpe: Sure. And MIT has this history of spinning out the Draper Labs and the Lincoln Lab. And again, that's stuff that's too applied to be done in a university lab by graduate students, but that should have some connection to the university. Less connection in the case of Draper Labs because of some firewalls. But MIT has some of that same same ethos. Yeah, Carnegie had this very practical streak to it.

Paul Christiano was the provost for a while, and his dad was a Carnegie Tech graduate in civil engineering back when the civil engineering curriculum had courses like bricklaying. This is really hands-on, get out there and get your hands dirty.

Gilliam: Yeah. Back when they used to be able to teach industry machinists at night and things like that, at places like MIT or Carnegie. Different world we live in now.

Thorpe: So, when I was director of the Robotics Institute, I said, “We have won the battle to be the biggest robotics research group. We're very good at producing smart robots. We're very good at producing smart graduate students. We're not very good at producing smart industry. How do we build up a robotics industry in the city of Pittsburgh?” And we started working with the big foundations there. We started working with the little spinoff companies. There were a bunch of them, but none of them were very big. And we started working with the university management to say, how do we pull all of this together?

And when I say the foundations…like the Hillman Foundation, which is one of the richest foundations in the city…John Denny from L. C. Hillman's office came and sat with us and said, “How are we going to build a robotics industry?”And that led to the Robotics Foundry, and that led to Pittsburgh becoming RoboBurgh, which eventually led to these bigger clusters and some federal manufacturing research centers and so forth.

Gilliam: So in that case, do you take a lot of pride in the following? In your one previous oral history, you referred to the early Google group as “Carnegie Mellon West.” You said this talking about how Sebastian [Thrun], who of course, built his DARPA Grand Challenge vehicle at Stanford, but he came up at CMU. And you pointed out how everybody but one in that early Google autonomous driving group kind of came up or had a deep affiliation with CMU. And it seems like that's held even recently. There are a lot of departments that do pretty good autonomous car work, but Uber came to CMU, specifically, to essentially attempt to buy up a whole swath of the university and its projects wholesale to fold into itself.

Do you take a lot of pride in the projects and people that CMU has been outputting being considered applied enough that they could be a direct input into an industrial process? Rather than what the current applied researcher’s credo seems to be at a lot of universities, namely, “Somebody could pick this up and within a handful of years, it can then be applied.”

You all are kind of directly plugging in.

Thorpe: So, Matt Mason — who was on my thesis committee and was my successor as Director of the Robotics Institute — he's a much more theoretical roboticist. And he says, “You know, if all of the applied people left Carnegie Mellon, we would still be a famous robotics institute for all of the more theoretical stuff that we do.”

But they had to kind of live in the shadow because, face it, it's much easier to do a videotape of a robot flying down the road and to explain to the Pittsburgh Post Gazette what it is that we're up to than it is for the theoretical people to say, “Ooh, and look what I can do with a quaternion, I can represent this thing…and this is a fundamental, advance that people are going to be using for the next century!” And they're right! And we are grateful to them. They just have a harder time getting the publicity out of it.

So the DARPA urban challenge was just a family feud. Sebastian and I have a best paper award together. Sebastian and Red [Whittaker, the engineering lead on the DARPA Grand Challenge teams] have a best paper award together. Chris Urmson, who is Sebastian's number two, was Red's graduate student. So this was just a family feud. Those guys happen to be out at Stanford, but they’re still buddies of mine.

(We then talk for a bit about semi-related topics before I ask him if he’s also proud of the number of NavLab/CMU grad students that have gone on to be technical leads at companies or found companies.)

Thorpe: Yeah, you know, I had expected more of my alumni to go off into academia. If you're an academic yourself, you sort of think, “Shouldn't everybody want to do what I do?” And, no! They've gone off and started their own companies. Jill Crismann was the AI guru for the Department of Defense and now has gone back into private industry. Jenny [Kay] went into academia and still teaches. John Hancock went off into the computer game industry, which is a good thing. Hardware keeps breaking for him. So we thought that actually working on real robots was not as good as working on simulated robots.

Parag [Batavia] started his own company. Raul [Valdes-Perez] went off to Google. So it's been fun watching these guys. They have this attitude that, “We can do it! Here's a cool problem. Let me figure out how to make something work. Oh, cool. That works. Uh, now that it works, let me see if I can figure out how it works and write a paper out of it.”

And that's just a fun attitude and fun to watch them go and succeed.

Gilliam: “And if it works, how do I make it usable? How can it be productized?”

Thorpe: The next thing, and the next thing, and the next thing. Some of them have figured out, “How do I take this and turn it into money?” Others have said, “How do I take this and turn it into robots that run on the surface of Mars?”

I cut off the recording there. Over the course of the interview, I came to admire Carnegie’s applied-minded approach to engineering work even more than I did coming into the interview.

Hopefully, you enjoyed this exploration into how CMU managed its exceptional autonomous vehicle research work from the mid-1980s through the early 2000s. CMU’s strategies are, in practice, quite logical. However, comparing their methods to academic norms, it is clear that CMU continually made decisions to go against the academic norm. Doing this, they maintained their major comparative advantage. The result, in the field of computer science and robotics, was a systems research powerhouse that helped shape the field of autonomous driving as we know it today. In many ways, this period of CMU’s history is the closest university example I have seen to the fabled early years of MIT.

Thanks for reading:) See you next time!

Subscribe now

Share

Pattern Language Tags:

• Utilizing a contractor made up of individuals with research-style goals and training working within a ‘firm’ structure.

• Building out an experimental test bed for a research field.

• Putting out contractors for specific “integrators” in addition to prime contractors and basic researchers

This piece is a part of a FreakTakes series. The goal is to put together a series of administrative histories on specific DARPA projects just as I have done for many industrial R&D labs and other research orgs on FreakTakes. The goal — once I have covered ~20-30 projects — is to put together a larger ‘ARPA Playbook’ which helps individuals such as PMs in ARPA-like orgs navigate the growing catalog of pieces in a way that helps them find what they need to make the best decisions possible. In service of that, I am including in each post a bulleted list of ‘pattern language tags’ that encompass some categories of DARPA project strategies that describe the approaches contained in the piece — which will later be used to organize the ARPA Playbook document. These tags and the piece itself should all be considered in draft form until around the Spring of 2024. In the meantime, please feel free to reach out to me on Twitter or email (egillia3 | at | alumni | dot | stanford | dot | edu) to recommend additions/changes to the tags or the pieces. Also, if you have any ideas for projects from ARPA history — good, bad, or complicated — that would be interesting for me to dive into, please feel free to share them!

Specific Links:

• Chuck Thorpe’s IEEE oral history (video)(transcript)

• Red Whittaker’s IEEE Oral History (video)(transcript)

• Raj Reddy’s Computer History Museum Oral History (Parts One and Two)

• CMU Strategic Computing Vision Project Report: 1984 to 1985 

• Dean Pomerleau’s 1988 paper, ALVINN: An Autonomous Land Vehicle in a Neural Network

• RALPH paper

General Links:

• Strategic Computing: How DARPA Built the Computer Age

With that, let’s get into it.

Subscribe now

Introduction

DARPA has no lab benches, research facilities, or staff scientists doing science. Instead, it has contractors. DARPA uses contractor ecosystem to execute any ideas the office chooses to fund. Throughout this series I will generally write project histories from the POV of DARPA’s PMs and office directors. This is a natural choice because, depending on the era of the organization, either the project managers or the office directors are usually tasked with driving projects forward.

However, in the coming trilogy of pieces I’ll tell stories from the POV of some of DARPA’s best contractors ever. Projects are selected, funded, and often steered by the DARPA office; but ideas are entirely executed by contractors.

In many cases, which contractor wins a contract might only have a moderate impact on the cost efficiency, speed, and overall results of a project. In those cases, it might be hard for a contractor — whether it be a university research lab, private company, or a non-profit institute like RAND — to prove it is truly top-notch.

One the other hand, there are cases where many in a field do not even bid on a contract because they do not find the project specs feasible. In these cases, a level of excellence can be required to deliver on a proposal. This is the category the now-famous ARPAnet fell into. As most readers know, the ARPAnet was the precursor to the internet. It was as the main contractor on this project that the MIT faculty spin-off research firm Bolt, Beranek, & Newman (BBN) would cement its name in the history books.

The coming trilogy of pieces exploring DARPA contracts from the contractors’ POV provide an essential perspective to this series. Contractors’ work on projects like BBN with the ARPAnet, CMU’s driverless vehicle projects, and Lockheed Skunk Work’s STEALTH project have many management insights in common — despite the obvious differences between those projects and contractors.

Today’s piece dives into the contractor primarily responsible for DARPA’s most famous project: BBN.

What was BBN?

BBN’s stellar results on the ARPAnet project did not shock many in the computing community. In the late 1960s, the firm was known to be special. It was an interesting kind of research firm, of a sort we don’t see much of today. BBN’s staff found themselves natural members of the elite Cambridge, Massachusetts research community despite the firm’s for-profit status. The firm was home to some of the best theoretical researchers in Cambridge — formerly professors at Harvard and MIT — as well as top-tier engineering researchers who often defected from the large computing projects at MIT’s Lincoln Lab.

In general, many researchers defected because they believed BBN was a better place to work on big problems in more exciting ways than at Lincoln. Theoretically inclined researchers, like Robert Kahn, found this a great place to do proper research in an applied context. Great engineers who cared less for theory, like the project’s lead engineer Frank Heart, found the firm an ideal structure to work on implementing real technology that still had an extreme level of novelty.

To be sure, BBN was a company that found ways to make a profit and grow at a considerable rate in its early decades — 26% year over year. However, it was also a firm that went out of its way to ensure that its researchers were happy. It wanted the best problems — as long as it could cover the costs and salaries of the project with some kind of funding — not necessarily the most profitable problems.

Robert Kahn and J.C.R. Licklider are two familiar names that called BBN home for a time. From his view as IPTO chief and minor celebrity in the computing research community in his day, Robert Kahn had a deep understanding of what went on at every notable computer research center in the world. So it should be taken as no small thing when, in an oral history, Kahn had this extreme praise for the firm in this era:

BBN was a kind of hybrid version of Harvard and MIT in the sense that most of the people there were either faculty or former faculty at either Harvard or MIT. If you've ever spent any time at either of those places, you would know what a unique kind of organization BBN was. A lot of the students at those places spent time at BBN. It was kind of like a super hyped-up version of the union of the two, except that you didn't have to worry about classes and teaching. You could just focus on research. It was sort of the cognac of the research business, very distilled. The culture at BBN at the time was to do interesting things and move on to the next interesting thing. There was more incentive to come up with interesting ideas and explore them than to try to capitalize on them once they had been developed.

As we’ll explore, some at the firm would often try to capitalize on big ideas. But this was not the focus of the firm.

From a legal point of view, BBN was legally structured as a partnership for its early decades — kind of like a law firm — before going public in 1966. Teams at BBN were not exactly permanent. Individuals were often recruited simply because BBNers felt they could contribute to some coming project, or simply “raise the average level of competence of the firm.”

The firm’s culture was far more similar to a university lab than an incumbent firm like Honeywell. If a talented researcher found an area of exploration interesting, the firm did what it could to find a payer for that research. For example, this was the case with a young J.C.R. Licklider looking to further explore the world of interactive personal computing on a very expensive machine. Bolt and Beranek did not say no. Rather, they said yes, bought a machine, and found a way to fund the project through a grant from the Ford Foundation. The long research report — later turned into a book — by Licklider and other BBNers titled, “Libraries of the Future” was far more than it seems. The report did more than just carry out a forward-looking analysis by information researchers and engineers assessing what digitized libraries could look like for a library non-profit. The report contained much of the early technical exploration that would grow to become Licklider’s legacy.

Beyond BBN’s whole-hearted embrace of the “research” in “research firm,” its structure as a “firm” allowed it to carry out extended, elaborate implementation projects like the ARPAnet project that were ill-suited to an academic lab. The firm’s expertise in early computing systems — built up by contract research projects as well as the team’s pre-existing experience at places like Lincoln Lab — allowed it to push technology areas forward in a self-funded way. For example, in the 1960s the firm designed, implemented, and maintained one of the first commercial time-sharing systems for non-engineers. The project gave Massachusetts General Hospital a time-sharing system. The system allowed the hospital to handle patient administration in a modern way and automate some of the data analysis in their research. This, of course, was not the first time-sharing system. But projects like this allowed BBN engineers to continue honing expertise with young technology and earn private sector wages in doing so.

The ARPAnet project was another implementation project optimally suited to BBN’s somewhat unique structure and makeup. But, to gain a full understanding of how the firm came to be what it was, let’s take a step back and start at the beginning. BBN was not started as a computing research firm. Bolt and Beranek were not computer scientists. They were acoustics professors who, steeped in the applied research tradition of early MIT, began to become overwhelmed by their contract research work in the 1940s.

BBN’s early years as an acoustics firm

BBN began in a fashion not unlike many early-20th-century MIT spin-off companies. A team of professors and graduate students worked extensively on consulting and contract work. As time went on and word spread, the demand for their contracting services grew. For a while, renting an additional room in their lab building and hiring more graduate students was enough to keep up with demand. Eventually, the young company had to get its own quarters and hire its own full-time employees.

In the case of BBN, the original professors involved were involved in the relatively young area of psycho-acoustics. In the wake of World War II, Richard Bolt was set to become the director of a new acoustics laboratory at the Institute to continue several courses of research that were largely funded by the Navy. Faculty and staff with backgrounds in physics, electrical engineering, architecture, mechanical engineering, aeronautical engineering, and psychology would all contribute to the contracts. In these post-war years, Leo Beranek was recruited to join the lab as its Technical Director from Harvard’s electro-acoustic laboratory. He was also given a professorship.

Since the inception of MIT’s Technology Plan in 1920, the Institute had made a habit of actively facilitating research contracts to its faculty. Professor’s contract salaries covered only four days a week during the academic year and left weekends, holidays, and summers entirely free to the professors for contract work. Professors could wholeheartedly spend their time on contract work with close to zero red tape or permission required from MIT. Unlike places like Harvard — and most universities — the Institute, in this era, encouraged this double life for its professors. Negligible amounts of paperwork and permission-asking were involved. MIT saw no more suitable activity for faculty at a professionally-focused institute of technology than to ply their craft for paying customers. In doing so, professors simultaneously kept up with industry trends, solved industry’s problems, learned how to keep their university research industry-relevant, and brought money into the Institute.

In the heyday of this arrangement, requests from industry regularly came into MIT’s office of the president for help in various problems, such as acoustics. When acoustics queries came in, they were routed to Richard Bolt. In 1946, one of these requests came from an architecture firm responsible for building the UN Headquarters in New York City. Bolt submitted a bid and won the commission. In 1948, the drawings arrived from the firm with the request for Bolt to begin his acoustics consulting work. Given the massive size of the drawings, Bolt realized the project was not a one-man job. He asked Leo Beranek if he’d like to work on the contract as a team. The two had papers drawn up, and the partnership of Bolt & Beranek came into existence.

For this first contract, Bolt and Robert Newman — one of four part-time graduate students hired to work on the project, who was brought into the partnership upon graduation — were responsible for figuring out acoustical treatments for the UN building. Beranek was in charge of the more troublesome task of sound system design and equipment selection for the unorthodox building. Despite the difficulty, the project turned out to be a very public success. Beranek wrote that the firm’s “name was known to architects everywhere, and business boomed.”

To this point, the firm had simply been run out of an extra room in the Acoustics Laboratory. As Beranek described the arrangement:

The firm, Bolt & Beranek, had the blessings of MIT’s new President, James Killian. He offered to help us get started and rented us two rooms in the MIT Acoustics Laboratory for our use, but warned us that we would have to seek space outside of MIT if our needs expanded.

By the end of the business’ first twelve months, the room was already full of equipment the firm had bought to continue to service the growing number of contracts.

In late 1949, the young firm moved to another address in Cambridge. Gradually growing, in 1951 the firm would move again to take up two apartments and a basement in a mostly residential building on Elliot Street. By 1956, the acoustics-focused firm had built up to 50 full-time employees — many of them former Cambridge graduate students — along with an additional office in the architecture hotbed that was LA. This all happened without outside financing beyond the firm’s line of credit with the local bank.

In this period, the firm also added two more of the four founding part-time graduate students to its partnership. These new partners, Jordan Baruch and Samuel Labate, had worked with the firm for years and also completed graduate theses in areas relevant to the firm. Additionally, in 1953, BBN incorporated in order to shield itself from liabilities associated with the growing subsection of its business associated with controlling aircraft noise. The National Advisory Committee on Aeoronautics and companies that manufactured jet engines hired the firm to work on problems such as designing structures to minimize noise during engine testing. Projects like this had many practical uses, but also represented cutting-edge uses of the theory and experiment in acoustics that excited acoustics researchers. (Projects like this also benefited from the insights of psychologists. As it turned out, projects in noise reduction or concert hall design often turned out to be more about “perceived” decibels rather than actual decibel measurements.)

As all of this was happening, Beranek gradually reduced his teaching load at MIT — to 75% in 1951, 50% in 1953, etc. He resigned his tenured professorship in 1958. More and more work — from government contracts, private industry, and research grants — steadily rolled in. As the business grew and contracts rolled in, the MIT and Harvard research communities provided an ample supply of part-time minds and hands. Sometimes individuals were brought in because they were acutely needed. In other cases, the firm simply saw a talented individual and would find some excuse to bring them on.

As the business kept expanding, in size and scope, Beranek set his sights on building up more of a talent base in computing. During the war, Beranek remembered how much of the work of his Harvard Electro-Acoustic Laboratory and its partner lab, the Psycho-Acoustic Laboratory, related to problems dealing with information handling in warships. BBN already combined psychological insights in its acoustics work, working on problems like pilots’ ability to process noise in cockpits. Expanding into the arena of man-machine systems felt natural. To Beranek, this felt like a fascinating area of research with the potential to amplify human labor.

BBN enters the computing business

Beranek’s motto in making hires at the firm was, “Each new person hired should raise the average level of competence of the firm.” At its core this was a research firm. And in staffing the research firm, he felt it was vital that they only continue to hire people who they felt were as smart as them. With Beranek’s sights now on computing, he seemingly only had one man in mind to help lead the firm’s computing efforts: J.C.R. Licklider.

Licklider had been a young researcher in the Psycho-Acoustics Laboratory at Harvard during the war. Beranek was so impressed with him that, soon after being brought over to MIT, he pushed for Licklider to be hired there as well. Now, he was going to push Licklider to give up his tenured position at MIT to come to BBN. With the promise of generous stock options, the understanding that the firm did cutting-edge research work, and the title of “vice president in charge of man-machine and information systems,” Licklider came aboard in 1957.

One co-worker said of Licklider’s move that, “he was becoming entranced by computers at this point and felt he could pursue these interests best at BBN.” In the end, Licklider would go on to draw many of his former friends and colleagues from graduate school, the MIT faculty, and elsewhere in his academic travels to come to BBN. After moving to BBN, some simply continued with their pre-existing work under research contracts they’d won before joining BBN that paid for their services. Some were folded into BBN’s regular operations. Others found contracts to do odd things like write textbooks using BBN’s rare equipment.

But before this collecting of colleagues started, it was just Licklider joining the firm from his MIT position. No formal roadmap had been agreed upon with BBN management. Licklider quickly got things rolling, though. Beranek describes Licklider’s early tenure at the firm as follows:

Lick, as he insisted that we call him, was outgoing and always on the verge of a smile; he ended almost every second sentence with a slight chuckle, as though he had just made a humorous statement. He walked with a gentle step, often with a Coca-Cola in hand, and he always found the time to listen to new ideas. Relaxed and self-deprecating, Lick merged easily with the talent already at BBN. He and I worked together especially well: I cannot remember a time when we disagreed.

Licklider had been on staff only a few months when he told me, in the fall of 1957, that he wanted BBN to buy a digital computer for his group. When I pointed out that we already had a punched-card computer in the financial department and several analog computers in the experimental psychology group, he replied that they did not interest him. He wanted a then state-of-the-art digital machine produced by the Royal McBee Co., a subsidiary of Royal Typewriter.

“What will it cost?” I asked.

“Around $30,000.” (~$300,000 today) he replied, rather blandly, and noted that this price tag was a discount he had already negotiated.

I exclaimed, “BBN has never spent anything approaching that amount on a single research apparatus. What are you going to do with it?”

“I don’t know,” Lick responded, “but if BBN is going to be an important company in the future, it must be in computers.”

Although I hesitated at first — $30,000 for a computer with no apparent use seemed just too reckless — I had a great deal of faith in Lick’s convictions and finally agreed that BBN should risk the funds. I presented his request to Labate and Baruch, and with their approval, Lick brought BBN into the digital era. Lick sat at that computer many hours each day, literally hoarding the machine, learning how to do digital programming.

To be sure, this turn of events was probably not surprising to Beranek. Licklider had been deeply involved with computers at MIT through his affiliation with Lincoln Lab computing projects as well as interactions with Wes Clark and other early computing gurus. It was at MIT where Licklider, as a technically focused psychologist, was becoming an evangelist for work we would now categorize as human-computer interaction. But Licklider may have grown expensive more quickly than Beranek anticipated. As contracts and know-how were gradually building up in the computing area of BBN, the firm bought the first PDP-1 from the Digital Equipment Corporation two years after it purchased the first machine.

After describing the Royal McBee purchase in his chapter of “A Culture of Innovation” — an internal history of BBN by BBN alumni — Beranek described the obvious next step:

Then Lick and I took off for Washington D.C. to seek research contracts that would make use of this machine, which carried a price tag of $150,000 (~$1.5 million today). Our visits to the Department of Education, National Institutes of Health, National Science Foundation, NASA, and the Department of Defense proved Lick’s convictions correct, and we soon secured several important contracts.

When it came to talented researchers and academic engineers, BBN’s instincts as researchers often led the way. The organization was a firm and would not willingly light money on fire to pursue its interests. But if a talented group of researchers and engineers felt a problem area was promising, BBN might spend money to pursue it as long as management felt it could probably earn the money back.

In the case of purchasing the PDP-1, this was a more risky leap than the company usually took. But the risk was quickly mitigated, with BBN quickly selling government contracts in areas like man-machine cockpit system displays and computer-based learning. However, the primary grant driving Licklider’s work on computers at the firm was not a government grant. The grant that proved futuristic enough to support Licklider’s grand vision came from a grant-maker the reader would likely not expect.

Libraries of the Future

The Council on Library Resources was established by the Ford Foundation to study “libraries of the future.” The Ford Foundation was something like the Gates Foundation of its day and, thus, funded a broad swathe of areas. In particular, the libraries of the future contract arose from the growing “information problem” talked about in the research community in the 1960s. Namely, much more work was being published than any researcher could possibly make use of. Licklider and BBN were tapped to explore whether or not modern computing could do something about this problem.

(For those interested, the Council on Library Resources was advised by Warren Weaver, John Pierce, and Richard Bolt in the early stages of its existence. Weaver and Pierce have been featured in earlier pieces on this Substack.)

Contracts like this two-year contract were not likely where BBN made the lion’s share of its money. However, projects such as this — or projects such as those where NASA paid the company a fee to perform exploration and produce a textbook — allowed BBN to offset the costs of explorations its researchers already wanted to do.

Going above and beyond what the Council on Library Resources likely expected, Licklider’s team used this contract as an excuse to explore seemingly futurist technology. For example, Licklider and the project team dedicated much of the second half of the report to technical explorations under the assumption that future library-goers might interact with the library as a store of knowledge with a question-answering front end, rather than as a means to check out books.

The project seems to have gone a long way in offsetting the costs in a variety of explorations by BBN staff. Below, BBN-alum John Swets describes a question Fisher Black — of Black-Scholes Model fame — pursued:

Fischer Black, a mathematics graduate student at Harvard and part-time BBN employee, produced a series of question-answering systems that involved symbolic logic and computer programming, including one in which a non-human system first solved the “airport problem” posed by McCarthy. Based on some statements about a person’s whereabouts, transportation resources, and local geography, the system answers the question of how to drive to the airport (in part: walk from the desk to my garage, drive my car to the airport.)

(Eventual Turing Award winners John McCarthy and Marvin Minsky were also part-time members of the Libraries of the Future project team.)

Other ideas explored in the report include:

• The importance of improved random access memories for optimal information retrieval on large datasets — like one containing the entirety of scientific papers.

• Black’s question-answering system which successfully used first-order predicate calculus to represent information.1

• Several prototype indexing methods and iterative information retrieval methods for searching academic papers.

• And many qualitative aspects of PC user experience which Licklider is known for.

Many ideas in the report were not novel on their own. However, many of the systems outlined in the report were novel, practical advances that both fulfilled BBN’s contract obligations and the research ambitions of its employees. The report paints a remarkably prescient picture of how personal computers would eventually organize knowledge and interact with users. In reading the report, one can easily recognize many of the ideas for which Licklider would later become famous as the PC revolution got underway.

In this period, BBN was also working on more engineering-focused computing projects. The practical experience and staff the firm acquired in undertaking these projects gave the firm the much-needed experience and talent it would later need to complete the ARPAnet project. The most important of these projects may have been BBN’s work developing small machine time-sharing systems for hospitals.

BBN’s Hospital Time-Sharing Systems

In 1958, Ed Fredkin had only briefly been at Lincoln Lab before deciding to go into business for himself. To get started, he ordered himself a Royal McBee. He did not yet have the money to pay for the computer, so he went seeking out contracts. He consulted with his friends Frank Heart, then at Lincoln Lab, and Tom Marill, who had recently left Lincoln Lab for BBN, to see if they knew of any customers. Marill told Fredkin to see if BBN would be interested in doing some business with his company. So, he set a meeting with BBN’s new computing czar, J.C.R. Licklider. Fredkin’s meeting with the newly-joined Licklider ended with Licklider pitching Fredkin to, “come work at BBN.” Licklider convinced the partners to purchase Fredkin’s Royal McBee as part of his hiring arrangement. With that, Fredkin was added to Licklider’s growing team of computer engineers.

Licklider’s computing team buildup was still in its early stages, as Fredkin remembers it. If you said “computer,” people would say “analog or digital?” to show they at least knew something about computers. In Fredkin’s estimation, even Licklider was very much still learning the ropes. It was clear that Lick’s higher level ideas were fantastic. But as an engineer, Fredkin had this to say about him:

There was nothing I could do to get Lick to be a good programmer. He insisted on being a coder, and had wonderful high-level ideas; but what he always chose to code never made sense to me. I tried to straighten him out a number of times but couldn’t succeed. It’s just that at that time he didn’t have a knack for coding, either in the style of code or in the things he chose to code.

In short, in Fredkin’s estimation, Licklider seemed to be more bird than frog at that time.

But Fredkin may have had high standards. Licklider referred to Fredkin as a “young genius” and said the decision to bring him in to help grow the computing efforts was a no-brainer. Fredkin, who at the time was not the most fit for a corporate environment, found Licklider and BBN very willing to find a way to make the arrangement work. Licklider remembered that “he was having all kinds of psychological problems about getting work done.” So, BBN worked out an arrangement with Fredkin where, at the end of the month, they would figure out what his salary would be for the month. To Lick, this was just one of BBN’s many “fantastically interesting and flexible arrangements” that it worked out with its sharp employees.

Fredkin was instrumental in convincing BBN to buy the first PDP-1 computer and to bring on professors Marvin Minsky and John McCarthy as part-time employees. Soon after this arrangement began, Fredkin surprised McCarthy by saying that he believed some of McCarthy’s time-sharing concepts could actually be implemented on the rather small PDP-1. Fredkin’s combination of engineering design skills and professional connections from Lincoln Lab helped push this work forward. As McCarthy remembers:

Fredkin designed the architecture of an interrupt system and designed a control system for the drum to permit it to be used in a very efficient swapping mode. He convinced Ben Gurley, the chief engineer for D.E.C, to build this equipment.

Fredkin and his colleagues made multiple technical improvements to prove McCarthy’s ideas could be implemented on such a small machine — including inventing the concept of a swapping drum. Fredkin left the firm in 1961, with McCarthy taking over directing some of Fredkin’s time-sharing projects. But Fredkin’s efforts did a lot to jumpstart BBN’s engineering efforts in time-sharing.

This early time-sharing work eventually led to BBN being hand-picked for an extensive course of funded engineering research. The work came about when Jordan Baruch, one of BBN’s partners, was doing acoustics work for the Clinical Center at the NIH. His project involved instrumentation for the cardiac and neurological staff. During his weekly trips to the NIH, Baruch would go to the home of the Clinical Center’s Director, Jack Masur, for gin and jelly beans. On one of these evening visits the topic of computing in healthcare came up because Masur was asked to give a speech on the topic. Baruch seized the opportunity and shared his vision for the possibilities of computers with patient medical records. Masur used many of Baruch’s ideas in his speech. After the speech was a hit, Masur phoned Baruch and told him, “You should go do it.” Masur recommended Baruch apply for a grant from the NIH’s Division of General Medicine to build the system. BBN did. It won the $1 million (~$10 million today), three-year initial grant.

The contracting relationship would continue over the years, growing from a proof-of-concept prototype to an actual system deployed in a hospital. The specific goal of the research project was for BBN to develop a hospital computer system that would automate “the information gathering, processing, storage and retrieval that takes place in the modern hospital.” The work eventually grew into BBN installing terminals around Massachusetts General Hospital (MGH) which were connected to a time-shared computer at BBN. The initial applications were admissions, discharge, medication ordering, test ordering, and test reporting.

As the project wore on, the medical professors at MGH began using the system for certain types of interactive (albeit primitive) data analysis for their medical studies. BBN staff worked with them to develop these capabilities in the system. As former project manager Paul Castelman noted, “At the 1965 RAND/SDC conference on advanced data-management systems, BBN’s Hospital Research System was the only system reported that operated interactively rather than in batch mode.”

Four years after the start of the MGH project, work on the project began slowly winding down. As a technological proof-of-concept, the project was a success. But it seemed that the system’s deployment in a teaching hospital found difficult reception with some of its (apparently many) opinionated faculty. As project member Bill Mann remembered:

The feelings at MGH ranged from “very interesting” to “it may kill my patients, get it out of here,” with a strong bias towards the latter.

In the mid-1960s, the project’s originator, Jordan Baruch, was leaving BBN to spin off a (BBN-sanctioned) joint venture with GE based on the work called Medinet. The MGH system itself was being handed over to the hospital to operate. In this period, Frank Heart joined BBN as a Vice President in charge of managing technical engineering, with a particular interest in BBN’s life sciences efforts. Heart, in his first year and a half at the firm, helped spearhead BBN’s life sciences computation efforts in various areas of biomedical research.

However, not long after Heart’s joining word was swirling around that Larry Roberts had been brought to ARPA to orchestrate a contract that, as an engineering problem, was too fun for the team at BBN to pass up. Heart and several other Lincoln Lab alumni who had joined BBN in this same period were ideally suited to carry out Roberts’ project. They had a toolkit few in the computing research ecosystem had at the time: deep experience in designing and implementing real-time systems.

ARPAnet Context and Beginnings

The computing team that had been amassing at BBN contained several individuals from the small community specializing in real-time computing systems. In his oral history, Frank Heart explained why so few in the computing community had real-time systems experience, saying:

You know, in that period of time, the computer world was really somewhat divided into people who understood real-time systems and those who didn't. I mean, there were people who built computer systems using operating systems, and there were other people who built very finely tuned machine language programs for dealing with phone lines. And those two camps didn't interact a great deal. So the world was not full of people who knew how to make computers run in real-time, and connect to real-time systems. We were not the only group, but it was a somewhat small universe.

Making computers work via communication lines was no easy task. Larry Roberts, the IPTO PM who oversaw the ARPAnet project, had his own difficulties with the technology while at Lincoln Labs. In his last round of research at Lincoln Labs before joining ARPA, Roberts had been attempting to outline the open computer and communication issues that stood in the way of achieving practical inter-computer communications. As Roberts remembered, the results of this work he’d done with Tom Marill showed “that computers could work with each other, and we figured out how to do that — but we couldn’t get the communications to work on it at all.”

This problem, inter-computer networking, is the primary problem Roberts was brought to ARPA by IPTO Director Robert Taylor to work on. Roberts, who initially rejected Taylor’s offer since he so enjoyed his life as a researcher at Lincoln Lab, eventually had his arm twisted by Lincoln Lab staff. He had turned down Robert Taylor’s offer in early 1966. But, after failing to find another candidate as good as Roberts, Taylor went with something of a Plan B. Then-ARPA Director Charles Herzfeld, at Taylor’s bidding, called up Lincoln Lab’s Director to point out that ARPA was 51% of Lincoln’s funding. Herzfeld then re-emphasized just how much they would like to have Roberts’ services. Roberts accepted. Or, as Taylor summarized these events, “I blackmailed Larry Roberts into fame!”

For reasons beyond the diet blackmail, Roberts was coming around to what he viewed as a bit of a research management job. Roberts explained:

I was also coming to the point of view…that this research that we [the computing group at Lincoln] were doing was not getting to the rest of the world; that no matter what we did we could not get it known. It was part of that concept of building the network: how do we build a network to get it distributed so that people could use whatever was done? I was feeling that we were now probably twenty years ahead of what anybody was going to use and still there was no path for them to pick up…So I really was feeling a pull towards getting more into the real world, rather than remain in that sort of ivory tower.

This general sentiment, coupled with BBN’s openness to interesting research problems if the money was right, is partially why the firm was so successful in recruiting Lincoln Lab staff.

Roberts’ willingness to dedicate the next years of his career to the specific problem of orchestrating the building of the ARPAnet is not surprising. He had started heavily thinking about the problem four years prior. The event that pushed Roberts to begin dedicating his research agenda to the idea was a series of conversations he had at a 1962 computing conference with, among others, J.C.R. Licklider. Roberts said this of the ideas that came up talking with the group:

I came to the conclusion that the next thing, really, was making all of this incompatible work compatible with some sort of networking. In other words, we had all of these people doing different things everywhere, and they were all not sharing their research very well. So you could not use anything anybody else did. Everything I did was useless to the rest of the world, because it was on the TX-2 and it was a unique machine. So unless the software was transportable, the only thing it was useful for was written technical papers, which was a very slow process. So, what I concluded was that we had to do something about communications, and that really, the idea of the galactic network that Lick talked about, probably more than anybody, was something that we had to start seriously thinking about. So in a way networking grew out of Lick's talking about that, although Lick himself could not make anything happen because it was too early when he talked about it. But he did convince me it was important.

According to Robert Taylor, the recent development of time-sharing was the reason he felt his tenure as IPTO chief was a particularly good time to make a push for the ARPAnet project. Taylor said:

The thing that struck me about the time-sharing experience was that before there was a time-sharing system, let's say at MIT, then there were a lot of individual people who didn't know each other who were interested in computing in one way or another, and who were doing whatever they could, however they could. As soon as the time-sharing system became usable, these people began to know one another, share a lot of information, and ask of one another, "How do I use this? Where do I find that?" It was really phenomenal to see this computer become a medium that stimulated the formation of a human community.

The ARPAnet, in Taylor’s mind, was meant to make this “interactive notion” work one level higher — across universities rather than just within them.

Prior to Roberts’ arrival, Taylor had already gotten the necessary funds for the early stages of the project approved and set aside by Director Charles Herzfeld. The next step in the project is common to many major ARPA projects like this. Larry Roberts went around talking to anybody in the ARPA computing community who could conceivably be useful. He wanted to have an understanding of all the opinions from all the top people before he put a concrete proposal out into the world.

Many researchers and engineers contributed ideas that made it into the final proposal, such ideas to improve the network’s host-to-host protocols. But one researcher who was consulted in the planning stages of the RFP can be considered of particular importance. That researcher was Wes Clark of Lincoln Labs — who mentored several BBN employees while they were there. Clark, who his contemporaries considered a bit of an inventive genius, is a bit of an unsung hero of this generation of computing for his work in building early small computers — such as the TX-0, TX-2, and LINC. Clark’s thoughts helped convince Roberts that instead of using a large machine in the center of the country to help “run” the ARPAnet, Roberts should have a small computer at each site handle that. Clark’s advice was singularly important in determining the final structure the ARPAnet took — which was remarkably decentralized.

These small computers Clark proposed came to be known as “IMPs.” In the ARPAnet, the IMPs were meant to connect to both the large host computers at the ARPAnet site as well as the corresponding IMPs at other ARPAnet sites. If they could be successfully made, the use of IMPs would substantially ease the burden involved in connecting host sites to communication lines. Instead of each site having to find a way to connect directly to the telephone lines, they’d simply connect to an IMP. It would be the main contractor’s job to make an IMP reliable and easy enough to use to make the ARPAnet work.

Issued in 1968, the RFP was seeking a prime contractor to design, build, and install these IMPs. In essence, whoever won the contract would be principally in charge of the ultimate success or failure of the project. The ARPAnet nodes — largely research sites with a lot of IPTO funding already — would receive their own ARPA contracts to perform some amount of set-up, debugging, and research on their own end. The IMP contractor would be in charge of building IMPs that would be able to interface with most of the large computers at the time — which were not built to be interoperable. This IMP contractor also had to make sure that IMPs could communicate across noisy telephone lines, across thousands of miles, in real-time. This was the problem that had stumped Larry Roberts on a small scale while he was still at Lincoln Lab. This was a difficult research project as well as an intricate kind of engineering implementation problem.

The team at BBN thought this sounded like great fun. Each member of the proposal team — Frank Heart and Severo Ornstein on hardware, Dave Walden and William Crowther on software, and Robert Kahn on theory — make a point of repeatedly emphasizing that the project just seemed to be great “fun” and a fascinating engineering puzzle to solve. One or two felt it was a problem of massive importance. More often, they did not quite grasp how significant the project would become. Regardless, it was clear that this was a contract BBN wanted in on. So, Frank Heart and Robert Kahn — who had been consulted during the RFP process by Larry Roberts — took the lead in BBN’s proposal efforts. Kahn spearheaded the conceptual aspects of the proposal. Heart, who would eventually be put in charge of the project, spearheaded the more practical engineering aspects.

Kahn, who would later become IPTO chief, had also found his way to BBN around the same time Heart had. In 1966, Kahn took leave from MIT and joined BBN on the advice of a fellow engineering professor. Kahn’s research focus on mathematical aspects of communications was very theoretical. Despite Kahn having worked stints at Bell Labs during his Princeton Ph.D., he was still driven mostly by theoretical problems. So, Kahn’s senior colleague felt he would benefit from additional practical experience. Kahn referred to this piece of advice as “the best advice I ever got.” So, he headed to BBN. He initially intended to be there for only a year or two — he ended up being there for around six years. At BBN, he chose to work on networking.

He described his intentions as to “get my hands a little more dirty with some of the practical problems of everyday engineering.” He seemingly felt BBN was the best place to do this. He described BBN as follows:

It was a marvelous little firm at the time, a fraction of the size it is today. It was largely a group of very professional people doing an extension of what Harvard and MIT were doing, except on a full-time basis, without the responsibility of dealing with students and teaching.

With the RFP out, BBN put far more person time into the proposal than was normal for the firm. Five or six full-time people worked “very, very long hours” on the project for approximately three to six months. As Severo Ornstein remembered, “I spent a lot of time, I remember nights till 3 or 4 in the morning, working with Bob Kahn in the back room of my house in Newton on the proposal — designing the system and figuring out how it was all going to work.” Ornstein also noted that, “more dollars were spent preparing that proposal, more man hours charged to it, than I think had ever been done for any [BBN] project.”

As a reference, the firm strove for 70% of an individual’s time to be billable to some contract in some way. This inordinate use of time on a single proposal was a special case. To be sure, the special case was probably partially driven by the understanding that the contract could lead to longer-term contract extensions. But it seems that the primary reason the team spent as much time as they did was because both BBN employees and management believed this to be an exciting technical problem area — one which the firm felt they knew a lot about.

Many researchers on the team had known Larry Roberts at Lincoln Lab, knew him to be exceptionally smart, and figured he would understand how important their real-time computing experience was. Heart’s Lincoln Lab’s work with the Whirlwind computer required the processing of radar data in real time. As he put it, this meant “the computer had to accept the data at phone line rates and deal with each radar scan before the next one came along.” Heart continues, describing what he believed made his BBN engineering team so attractive in his chapter of “A Culture of Innovation,” saying:

This kind of computer use was unusual at the time, but at Lincoln Laboratory the group of people working with me became unusually expert at the real-time use of computers. At Lincoln, in a long series of projects, computers were connected to various radar antenna systems, radio antenna systems, sensors at underground seismic arrays, and sensors at underwater acoustic arrays. Each such project required a detailed understanding of the computer timing relative to the sequence of data arriving from the various sensor systems. This experience at Lincoln — tying computers to phone lines, and constructing hardware and computer programs that involved the timing constraints of such data handling — was a crucial attribute of my group at BBN that years later bid on and won the ARPAnet contract.

BBN’s main competition for the contract was large contractors like Raytheon and Western Union. Companies like this were significantly less focused on cutting-edge research than BBN but had deep experience managing large systems integration and implementation projects. Roberts stated in an oral history that, on paper, he thought that Raytheon’s bid looked quite reasonable. It wasn’t in a completely inferior league to BBN’s. But Roberts, himself having come up at Lincoln Labs with many BBN staff, did not ignore the advantages that the firm’s structure offered. He recalls:

Raytheon had a good proposal that competed equally with BBN, and the only distinguishing thing in the long run for my final decision was that BBN had a tighter team organized in a way that I thought would be more effective than a very steep commercial structure with lots of managers and vertically – so I didn’t believe they’d keep the schedules as well, but they had a good proposal.

To be clear, ARPA often trusted Raytheon with a variety of implementation contracts. So, it is not likely that Roberts meant that Raytheon would not reliably stick to an implementation schedule. Rather, he likely meant that BBN was far more likely to solve its way through all of the novel engineering challenges that stood between these proposals and an actual system being implemented. This was an extremely novel system. That was likely why BBN won the day.

Of course, all engineering projects are at least, in part, research projects; they all have many “unknown unknowns,” even once they are solved on paper or have been implemented before. Those standard challenges obviously existed in this project. However, the ARPAnet system had so many major “known unknowns” that this was as much a research project as an implementation project. The whole system was an experiment. In fact, the research-focused Robert Kahn stayed on with BBN for three more years after BBN won the contract because of this. There were practical, ongoing research problems — such as those in the area of systems design — that were going to require continuous re-working as the project progressed. As Kahn explained:

My notion originally was "Help them get the award; let them go build it and I will go back to doing what I was doing." But I pretty quickly came to realize that that just wasn't a practical notion. There were too many things that had to be thought about.

According to Roberts, companies like IBM even ‘no bid’ the contract because they didn’t think the request, as it was written, was possible. The price for the computing tasks required, they thought, was completely infeasible with the technology that existed. Roberts remembered:

IBM and CDC 'no bid' it because, they told me: "If we used Model 50s at these sites, you're going to go broke, and we think you're crazy." And what else is there, of course, besides Model 50s?

Many corporate researchers in the communications industry felt that the packet switching idea, which the network’s IMP-IMP communications relied upon, was completely infeasible. To players like AT&T, this was an academic idea that head-in-the-clouds researchers came up with; this was all fine and good on paper, but would just not hold up to the practical engineering reality. BBN obviously disagreed. But, as Frank Heart put it, “There was no proof. You couldn’t go to some book and find the answer; it was an experiment.”

The challenge, which some thought impossible at ARPA’s price point and given the noisiness of telephone lines, was set. BBN won the IMP contract. It was now their job to make it all work.

ARPAnet Operations

The initial one-year contract Roberts awarded BBN was to develop the IMP and deliver a four-node network connecting UCLA, the Stanford Research Institute, UC Santa Barbara, and the University of Utah. The contract was for about $1 million in 1969 (~$8 million today). The initial team that worked on the contract that year was made up of the initial five members who drew up the proposal along with about three more full-timers, primarily to work on hardware, and four or so additional part-timers. Several of the hires were Severo Ornstein’s best students from his course at Harvard.

The proposal team had, at least conceptually, worked its way through many problems in the extensive time it spent working on the proposal. So, the team was able to hit the ground running and make a push to have the four-node network operational by the end of the year. In all likelihood, Roberts likely expected the team might not make the deadline. In many cases, ARPA contracts would set extremely ambitious benchmarks that PMs, office directors, and contractors would later re-negotiate. But this did not turn out to be one of those cases.

Other Contractors on the Project

A number of subcontractors were also involved in the ARPAnet project. Each of the node engineering departments was a subcontractor. To a certain extent, these departments were responsible for getting their host computer set up with the IMP. BBN would do what it could to make this as easy as possible for them. But given the state of the technology and the talent present at the nodes, this was deemed to be the optimal approach. Additionally, ARPA used a contractor called DECCO to take charge of the logistically annoying process of purchasing time from AT&T on their telephone communication lines.2 Of course, DECCO purchased what BBN told them to purchase. Only BBN knew what it needed to successfully hook the IMPs up to the phone lines. But one would imagine BBN was still thankful to have DECCO to keep them somewhat insulated from this process.3

Two other contractors of note were the Network Analysis Corporation and the Network Measurement Center. ARPA wanted the Network Analysis Corporation to run simulations to help design optimal network topologies. In theory, this was reasonable. But, in Frank Heart’s estimation, the company did not prove as useful as ARPA expected in determining which sites to add next. In his eyes, practical matters such as whether or not departments were properly staffed or were ready to work on their IMP were mostly responsible for determining the network’s shape as it grew. The Network Measurement Center was run by Leonard Kleinrock at UCLA — the first node of the ARPAnet — and was responsible for taking a variety of measurements to study the network. The Network Measurement Center was, to Heart, somewhat academic in nature and not pivotal in the initial engineering or design of the ARPAnet. But it was useful in taking measurements pertaining to the health and efficiency of the network in its early stages.

Lastly, Honeywell was the subcontractor most pivotal to the contract’s success. BBN had no manufacturing facilities. Thus, they needed to contract with a computer manufacturer to have an IMP of their design actually built. At the time, few computer manufacturers did this. A standard company’s manufacturing process was not built to do wholly custom jobs. The processes were suited to be fiddled with, but not much more than that. But Honeywell had recently acquired a company called the Computer Control Corporation (CCC) that made them an exception. The team from CCC, as Heart put it, “understood a bit about the special systems business.” This was a unique enough skill that at least two contract bidders had used Honeywell as a subcontractor — BBN and Raytheon. Larry Roberts listed this as a reason that Raytheon’s bid was being considered alongside BBN’s.

Some of the Major Technical Problems

A variety of technical problems awaited the BBN team.

One of the major early problems the BBN team spent its time working through was the error control and correction problem. To work through this, the team was able to design special checksumming hardware that would be installed at both the transmitting and receiving end of the circuit. Every packet of information sent would have a 24-bit cyclic checksum appended to it which would be recomputed by the hardware at the receiving end to determine if there were any errors. If there was, the packet would be retransmitted.

How the nodes got there was the most theoretically-involved piece of this engineering experiment. The packet switching technologies we know, love, and hyper-optimize today were much more un-tested back then. This was one of two areas in which Robert Kahn’s presence proved extremely useful to the engineering-focused team. Kahn was able to quickly explain to them various aspects of packet switching algorithms as problems came up. (The other major theoretical area Kahn contributed to was in some mathematical aspects of phone line error problems.) The technical details of the routing algorithms the team initially deployed are described in the project’s completion report to ARPA as follows:

The approach taken was to use a distributed adaptive traffic routing algorithm which estimates on the basis of information from adjacent nodes the globally proper instantaneous path for each message in the face of varying input traffic loads and local line or node failures. Each IMP keeps tables containing an estimate of the output circuit corresponding to the minimum delay path to each other IMP, and a corresponding estimate of the delay. Periodically, each IMP sends its current routing estimates to its neighbors; whenever an IMP receives such a message it updates its internal estimates. Thus, information about changing conditions is regularly propagated throughout the network, and whenever a packet of traffic must be placed on the queue for an output circuit, the IMP uses its latest estimate of the best path.

Another major area of problems came from the technical aspects of the IMP-Host interfaces. The engineers spent substantial time hitting their heads against the wall to find ways to successfully connect IMPs to all different kinds of computers while, also, minimizing the amount of engineering work required by the node contractors to connect their machine to the IMP.

Solutions to all of these problems had to work in close to real-time. Individuals from remote machines were going to use the network to do much more than just send batch messages like emails. They were going to use their computers to remote into another computer as if the whole network was a single, time-shared machine. So, to make this all work at time latencies that made sense, the BBN team had to do things like “write very carefully tailored programs in machine language to optimize the capacity and the low delay of the data path in the switching node.” The team emphasized in its final report that, “Great attention was paid to minimizing the operating time of the inner loops of these programs.” Many areas of minutiae were all-important.

Many problems — from IMP design, to Host-IMP connection software and hardware, to error-correcting software — required this level of care. All of the high-level concepts — computers sending a message across a telephone line, linking two computers together via a communication line in a lab setting, basic message handling across noisy communications lines, etc. — had been demonstrated as feasible before. But BBN had to invent and engineer all that was required to make it go from technically possible to actually usable.

Viewed naively, the implemented packet switching technology was the team’s major “new” contribution. But, as most engineers will tell you, a project like this can be one big pile of unknown unknowns. That, in part, is probably why industry did not consider touching it in their R&D efforts. But, given the BBN staff’s continuous experience in recent years working on real-world time-sharing systems and real-time computer communications, they were probably the team that had the best sense of what the unknowns of the project might look like and how to deal with them as they came up.

The Honeywell Partnership

To get the IMP designed and built, the BBN team heavily leaned on Severo Ornstein. Ornstein had previously worked with Frank Heart and others on various computer projects at Lincoln Lab, worked with Wes Clark on smaller computers at WUSTL, and had since returned to Cambridge to work at BBN and do some lecturing at Harvard. Prior to the networking project, at BBN Ornstein had been working on projects related to the use of computers in education. Ornstein’s interest in the ARPAnet RFP caused him to dedicate more of his time to BBN.

While designing the IMP during the proposal process, Ornstein estimated that he and the team “did 90% of the design then.” But even when BBN had won the proposal, Ornstein still had to put substantial effort into working with Honeywell to get the computers made. Ornstein remembered:

The man from Honeywell who was assigned to build those interfaces from my drawings didn’t understand the drawings well and was not really careful. We ended up having to redo much of his work.

Much of BBN’s interactions with Honeywell may cast some doubt as to whether an industry actor could have carried out this contract nearly as effectively as BBN had. One account by Ornstein regarding the BBN team working their way through a confusing problem will help elucidate why exactly this is. The story both shines some light on what exactly the BBNers were spending their research time on prior to the implementation stages and how Honeywell dealt with problems. Ornstein explains:

They were not used to doing some of the kinds of things we were doing…For example, we discovered some design flaws in the machine. One of the reasons we chose the Honeywell 516 was that we thought it was a mature machine…that was not going to gives us grief. Well, we were wrong. We pushed it very hard…and we uncovered a bug that they could hardly believe, a synchronizer problem…Synchronizer problems are very, very subtle. We had to dig and dig and dig at them, and finally from their back room they produced a really smart enough guy — they do have a few — but, it was very hard to get this guy. We finally got him…when he came and we sat down and I finally had someone I could talk to who would understand what I was talking about and believe me. It was a subtle problem. The program would run fine in the machine for days on end, and at the end of three or four days suddenly the machine would just die, inexplicably. It was a very, very low frequency failure, so infrequent that you could never look at it on a scope; you could only see the effects afterwards. In fact, we had to build special hardware that beat on the trouble spot many, many times faster than normal usage would. Then finally, with all the lights out in the room, we could faintly see on the tracing scope an occasional failure. That was when the Honeywell people finally became convinced that there was a real problem…We showed them the trivial fix they could make to the machine…So, their people weren't absolutely top-notch people; they were okay. They were industrial-strength people, not research-strength people.

Of course, there were industrial operations with plenty of “research-strength” individuals back then — assuming Ornstein meant individuals with the brains and toolkits to consistently find answers to vexing problems like the one mentioned. Bell Labs, Kahn’s old employer, was just one example.

Not assigning this systems engineering-style contract to an academic department makes sense. Most academic departments are just not staffed and structured in a way to make projects like this feasible. But it’s unclear how an outfit like Raytheon would have done working through problems like the one described above. Would they have been able to solve them? If so, how quickly? We see that the Honeywell team was great at reproducing duplicate machines once they had already made one. But there were certain computer engineering principles and skills utilized in the project that some of the builders — from places like Lincoln Lab and Wes Clark’s WUSTL team — had much more experience with than most other groups. That was to BBN’s great advantage.

All of that being said, the formerly-university-employed engineering team still conceptualized this as more of a novel engineering problem than a research problem. Dave Walden, one of the project’s two founding software engineers who later became General Manager of BBN, described how the team viewed the problem as follows:

My view is that mainly what we were doing was we were very pragmatically doing engineering. We had to send these bits down the wire: how do you put a header on the front; how do you put a trailer on the back. There was theory of how you put error-correcting codes on it. Bob Kahn knew that theory and told us what it was. There were some constraints: this is the way that the 303 (or the 301 or whatever the Bell modem is) has to be interfaced to, but after that it was all pretty pragmatic. Not lots of theory of coming from someplace else.

The Team’s Management

Before diving into how the one-year contract’s implementation went, we should first get a better understanding of how the team was managed at BBN. Firstly, it was much smaller than one would expect.

As the first year of work progressed, the team remained small — about eight people. This is how Frank Heart preferred it. As he saw it, that small team of elite talent in the relevant sub-areas was all BBN needed. He described the project’s work style further, saying:

I think mostly I tend to believe important things get done by small groups of people who all know all about the whole project. That is, in those days all the software people knew something about hardware, and all the hardware people programmed. It wasn't a group of unconnected people. It was a set of people who all knew a lot about the whole project. I consider that pretty important in anything very big. So I suppose if you call it a management style, that would be something I'd state. I think also that they were a very, very unusually talented group. I think things tend to get done best by small groups of very, very good people — if you can possibly manage that. You can't always manage it. So if you again want to call it a management style, it is to get the very, very best people and in small numbers, so they can all know what they're all doing.

Heart was extremely averse to the team growing so large that communication had to be done on paper. He preferred their style of “very, very frequent interaction on problems.” As Walden remembered, “I don’t remember such a thing as we had a weekly progress meeting. We probably were more in tune with the progress than that; we probably did it hourly.” Heart and the entire, small team did their best to stay technically involved with each other’s work. Heart, as a rule, insisted on technically understanding every bit of the project. The individual team members attempted to do the same, which they felt was quite easy to do. The team all had offices and work stations immediately in the vicinity of each other. As things came up, they’d simply go talk with the relevant individual or call an impromptu meeting.

The Early Node Installations

With some level of difficulty, BBN worked their way through problem after problem like those in the technical problems and Honeywell sections. With a lot of persistence, the team got the IMPs specified in the proposal built and de-bugged. By September of 1969, the hardware and software for the first IMP was ready for installation.

Unlike many ARPA computing contractors, the UCLA department had a core of individuals who were particularly excited about the ARPAnet project. While many of the professors who owned computers saw the ARPAnet as an opportunity for ARPA to force them to further split their computer time, Leonard Kleinrock and others at UCLA had research interests relating to the network itself. Kleinrock, most notably, had written about the theoretical aspects of packet switching in his prior work. Because of this interest, he formed the Network Measurement Center to monitor certain aspects of the network as a subcontractor for the ARPAnet contract.

In early September 1969, BBN sent its first IMP off to UCLA. When shipping the finished IMPs to the host sites, BBN even went as far as to have staff ride alongside the machines on the plane, just in case. For these early sites, Ornstein and some of the software researchers like Dave Walden traveled to the sites to hook the IMP up to the phone lines and work through any setup issues. In the lead-up to BBN sending the machines, the firm shared a set of specs with the host sites so the sites could do as much work as possible beforehand. On the node end this work was usually led by some kind of hardware lead and software lead.

BBN had apparently done quite an effective job of communicating its technology to UCLA. Also, UCLA had apparently done a good job of internalizing the information and building accordingly. According to Leonard Kleinrock, within a day of initially connecting the IMP and UCLA’s SDS Sigma 7, messages were moving back and forth.

This was a massive success. Still, many questions could not be answered until the second installation was done. Only at that point could BBN see what happened when two nodes tried to communicate with each other. The next month, the BBN team installed its second node at the Stanford Research Institute — which had also been subcontracted by ARPA to do some research on the network itself. Once BBN and SRI had the IMP hooked up, the two node research teams attempted to send the first message over the ARPAnet. As Leonard Kleinrock remembers:

On October 29th, 1969, one of my programmers, Charley Kline, and I were in this room and we decided to log on to this [SRI’s] machine. Now, to log on, one has to type in “LOG” and the remote machine will type in the “IN” So our job simply was to type in “LOG.” Now, up at the other end, there was another programmer waiting to watch all of this. And we had a telephone connection between these two…so they could talk to each other. What happened is Charley typed the “L” and he asked, “You get the ‘L?’” And the answer was, “Got the ‘L.’” He typed the ‘O.’ “You get the ‘O?’” “Got the ‘O.’” He typed the ‘G.’ “You get the ‘G?’” Wacko! The system crashed. This machine [SRI’s] went down.

So the very first message on the internet ever was “LO,” as in, “Lo and behold!” You couldn’t ask for a better, more effective message.

Surely the BBN team considered it enough of a success that any letters were sent at all. The team proceeded to install a new IMP per month for the rest of the year, making four in total. Problems continuously came up through both BBN’s testing and the node researchers’ testing. BBN fixed the problems as they came up. It does not seem like any of these problems proved particularly devastating.

Kleinrock excitedly described the arrangement as follows:

So we had a four-node network running, and we began to test it and find out what some of the problems were. And we could break this network any time we wanted, we found faults, we had BBN test it, etc.

As problems were worked out, there were regular hardware modifications and software releases that had to be rolled out. The software releases were done, initially, by sending Dave Walden around to each of the sites with his paper tapes to get the software working on the machines. In this way, the first batch of nodes became operational and steadily improved. BBN did this on budget and within the one-year timeline of the contract. Larry Roberts and ARPA were, surely, extremely pleased with the progress. By the end of the year, ARPA contracted BBN to expand the network to 19 nodes.

Before discussing how the project scaled, let’s briefly explore Larry Roberts’ working relationship with the BBN team. Roberts was a very active manager in the ARPAnet project, not just its funder.

Working with Larry Roberts

In putting together the RFP for the ARPAnet, Roberts had looked deeply into many technical points and consulted the best members of the computing community. In doing this, he made several key decisions before the contract was ever in BBN’s hands. As Heart put it, “They [DARPA] picked the baud rate of 50 kilobits, they picked the sites, they picked the issues about the checksums. A lot of the work had been done by DARPA in advance.”

As Dave Walden framed things, BBN could surely be considered the ARPAnet’s operator and the primary driver of the engineering work, but ARPA was the high level manager. This entire operation, the ARPAnet itself, was meant to facilitate new capabilities for ARPAnet contractors. So, Roberts providing a constant guiding hand made sense. Also, the ARPA contracting community contained some of the best minds who could make technical suggestions to the BBN team. So, Roberts and ARPA continually facilitated meetings between BBN, node contractors, and others as they saw fit. The now-famous Network Working Group which debated many ARPAnet-relevant ideas, like what later came to be the TCP/IP protocols, was one example of these efforts.

Once the project was underway, Roberts largely left BBN in charge of making day-to-day decisions. They did not need to seek his approval on many things. But he did keep close tabs with the team and would raise objections from time to time. Roberts could technically “pull rank” on BBN as the project’s funder. But the arrangement did not really work like that. If Roberts didn’t agree with the team’s approach to a particular problem, they would discuss it over the phone or in person until they could come to some kind of agreement. In a contentious case, that might take some number of days. But, in most cases, a clear agreement was reached.

The BBN team held extreme respect for Roberts as an engineer. In turn, Roberts seemed to respect the BBN team greatly. Roberts had a direct line of communication not just with Heart, but with individuals in charge of component technologies on the team such as Severo Ornstein. According to Heart, normally a week wouldn’t go by where Roberts didn’t talk with somebody on the team. Ornstein, who was particularly casual, said he simply conversed with Roberts as he would any other colleague.

Beginning to Scale the Network

The initial contract to build the four-node network was a success. So, ARPA opted to give BBN a contract to spearhead the effort to 19 nodes.

In early 1970, BBN itself became the ARPAnet’s fifth node as the network continued to expand at roughly one node per month.

As the expansion continued, new engineering problems continually arose. While Frank Heart was known to be an extremely defensive engineer, things were still bound to work imperfectly. This whole project was an experiment and was a first-of-its-kind network, so one could only be a certain level of defensive without halting the project’s progress entirely.

In many areas, the BBN team shipped technology that it initially felt was merely good enough — with the understanding that they could continually improve component technologies on an ongoing basis. One case of this was the network’s protocols. The initial NCP protocols were deemed by the Network Working Group to be quite sub-optimal.4 So, over the years the NCP Protocols protocols were improved until they were entirely replaced by Kahn and Cerf’s now-famous TCP/IP protocols.

As the ARPAnet completion report notes, on occasion, some of the problems present in the operational network were more than small. In 1970, a major problem was demonstrated with IMP flow control and storage allocation. Node hosts, if they didn’t heed the error, could accidentally halt the network’s entire operation. Yet, while BBN and others took the many months required to fix the problem, the ARPAnet continued to provide adequate service to its nodes.

As Dave Walden recounted in his oral history:

Walden: …it almost immediately became obvious to anybody who hadn't believed it before that what we had originally implemented was insufficient. And what we had specified to the host and what they had implemented had a lot of ramifications. They were now done and now it didn't work all that well. So for the first year or two years, quite a long time, there were rules, informal rules in the network that you didn't pump stuff into it from the hosts so fast it swamped the net. Because we all knew the algorithms didn't work, I mean they broke, patently they broke when you pumped stuff at them too hard. And so this interim solution worked in that network — this is the kind of thing that somebody who is working on proving programs correct and stuff can't imagine. It actually was quite useful in those early days of experimentation because everyone agreed not to break it. You know how to break it; let's not break it. Meanwhile, let's go figure out how to fix it…

Interviewer: So it was on the order of a gentlemen's agreement. Just not to go break it.

Walden: Engineers' agreement.

Given that all the users were well-trained and capable engineers and researchers themselves, BBN could also get away with only gradually scaling up its service capabilities. Dave Walden running around with his tapes to implement software updates was one example of this. As the network expanded and the network began to operate more reliably, BBN was able to dedicate more of its team’s efforts to helping the network operate a little more automatically.

In these middle years of the project, BBN also began to expand its efforts to finding more efficient solutions to manage network issues, answer questions from nodes, etc. In the beginning, BBN’s answer to this problem was to throw Dave Walden at it. As Walden fondly remembers:

Up until September of '70, I had the program listing beside my telephone at home and would get telephone calls at home whenever something stopped. My telephone number was quite literally on the front of the original packet switches and they called my house over here in Allston and I would talk to whoever was there, using my listings. So that's what happened.

With ARPA sponsorship, BBN began running the Network Control Center (NCC) in service of this goal. In building up the NCC, BBN designed and implemented tools and software to regularly ping the network and keep tabs on network problems, monitor the status of hosts, etc. Alexander McKenzie became the “ARPAnet generalist” BBNer put in charge of the customer service aspects of the project as it grew. McKenzie proudly recounted stories of calling telephone companies and telling them a line was about to go down…and then it would. The companies were quite impressed because this was not an ability that they had yet.

Starting in 1972, BBN hired staff to offer de-bugging and other customer services to network nodes full-time — three people worked the day shift, two the late evening shift, and one the midnight shift. Through BBN’s and the NCC’s efforts, nodes were soon able to (in most cases) automatically report their status. Maintenance and debugging were then typically carried out remotely from the monitoring center. As the NCC’s capabilities grew, BBN was able to successfully place IMPs at locations where the staff had less and less computer training.

Additionally, as the network expanded BBN conducted research in an attempt to expand the types of IMP technology as well as the types of communication lines IMPs could be hooked up to. In 1971, BBN finished designing its first terminal interface message processor (TIP). These terminals were able to connect to the ARPAnet and do work on ARPAnet host machines without having to be run on a host computer of their own. On their own, TIPs were more expensive than IMPs. But the idea was extremely exciting to ARPA because the TIPs becoming operational meant one could hook up to the ARPAnet without having a large computer on hand. The first two TIPs, shown below, were installed at MITRE and NASA’s AMES research facility. The early 1970s saw the first TIPs installed, IMPs connecting to the network via satellite technology, the design of standard IMPs ten times more powerful than the initial ones, and the deployment of Ornstein’s Pluribus multi-processor IMP.

As the ARPAnet hit its 19th node, ARPA had already opted to give BBN a contract to expand the network to over 50 nodes. The first two years of the project had all occurred largely on schedule and on budget. With this next extension, the network even began to include nodes from remote geographic areas like Hawaii and Norway. This required the use of some of the project’s new technological capabilities as well as the project’s first interactions with foreign telephone operators. While these remote nodes did tend to be more buggy, they were largely functional and proved some of the new capabilities to be useful.

Throughout this period, Larry Roberts’ continual arm-twisting of node sites to do what they needed to do to make the node installations a success proved extremely useful in keeping things on track. As many of the sites’ primary computing funder, he could exercise this power quite easily. By 1975, the ARPAnet project was clearly on stable footing. So, responsibility for the project was transferred to the Defense Communications Agency (DCA). According to DARPA Historian Richard Van Atta, at the time the project was transferred to DCA the total budget outlays throughout the project had been $25 million in total (~$150 million today). That number covers the entire project, not just BBN’s portion. By anybody’s metrics, this should be considered a remarkably cheap success.

The First International Conference on Computer Communication

How exactly the outside world — meaning those in the technology community not affiliated with an ARPAnet host site — became aware of the ARPAnet’s success deserves a brief explanation. Even in 1972, many in the communications community doubted this cross-country packet switching network idea would be technically possible anytime soon.

Robert Kahn’s final project as an official member of BBN helped rectify this. In October 1972, he put on a massive demonstration of the ARPAnet at the first-ever International Conference on Computer Communication (ICCC). The conference itself was an extremely useful forcing mechanism for BBN and each of the node contractors. This provided tangible pressure, with a concrete deadline, to thoroughly debug many pieces of technology, sure-up certain protocols, etc. The academics responded much better to the deadlines or threats of their funding being diminished than they did to the mere presence of bugs in the system. Additionally, the existence of the conference provided a push to ensure that as many host sites as possible were functional across as many use cases as possible. To this point, many nodes had only been outfitted with the hardware and software to use some of the ARPAnet’s possible applications.

Dozens of nodes contributed computer hosts to the demonstration. The conference was a sort of DIY showcase where individuals could hook up to the computer hosts via TIPs and work with them the way node users did every day. This could be for things as simple as e-mail, a rapidly growing use case of the users, as well as much more complicated programs. As Roberts recalled:

I think this was the watershed event that made people suddenly realize that packet switching was a real technology. We had thousands of people who went through that — I don't know what the exact number is — at least a thousand who went through that particular exhibit.

Or, as the ARPAnet completion report put it:

The demonstration itself was a spectacular success; with everything working amazingly well, many visitors remarked that the ARPANET technology "really is real."

ARPAnet Results

On a technical level, the ARPAnet experiment was a success in many ways. The project proved that a large network could be built in such a way that node failures were localized and did not crash the rest of the network. It convincingly demonstrated adaptive routing algorithms and packet switching theories. To a large extent, the ARPAnet proved that a decentralized network like this could run itself without a central command center. Finally, from a user perspective, the IMPs and TIPs proved reliable and easy enough to use to facilitate the time-sharing of diverse computers, thousands of miles from one another, for a variety of tasks. The working ARPAnet also succeeded in its role as a test bed for the entire field of computer communications technology research.

But, of course, the success of the ARPAnet project goes beyond the technical successes of the project. As BBN’s ARPAnet completion report stated:

The ARPANET project also proved the feasibility of achieving closely knit communities of technical interest over a widespread geographic area; it is possible that this social feasibility demonstration is as important as the many technical feasibility demonstrations.

Possibly the clearest indicator of this growth is how rapidly internode traffic grew over the network through the mid-1970s.

As the usage of the network grew, BBN feared that even the new algorithms being used to handle routing, flow control, and congestion control would not be able to keep up. But, as the team noted, “Luckily, the improvements in the algorithms managed to stay slightly ahead of the growth in network size and traffic.” In BBN’s estimation, the diminishment in growth rate was because at a certain point, the existing host machines being accessed over the network were simply reaching capacity.

Throughout this growth period, acute problems present in the network lessened, BBN sharpened its processes, demand increased, and more research was being done that relied on the network in some way. Several of the BBNers, in their oral history interviews, talked excitedly about how the network gradually became a “utility.” When prompted by the interviewer to say more, Heart explained:

A utility is something people depend upon. Like the electricity, or the phones, or the lights, or the railroads, or the airplanes. Yes, it was a utility. That's the thing that was the amazing surprise. It was started as an experiment to connect four sites, and it became a utility much, much faster than anybody would have guessed. People began to depend upon it. And that was a problem, because that meant when you changed it, or it had problems, they all got mad. So that was a two-edged sword. But it was also very exciting.

Alexander McKenzie noted that, while the ARPAnet was never as reliable as a power company, the team did eventually succeed in keeping the IMPs up 98% or 99% of the time. As this transition to “utility” took place, NCC-style activities took more of an emphasis. Research and operational activities that optimized the reliability of the network slowly became prioritized over network research activities that created knowledge but interfered with service. This was likely disappointing to some BBNers. But it can also be seen as merely a symptom of the project’s success.

The network was becoming so convenient that a substantial percentage of network traffic was intra-node traffic. Computer hosts at one site were communicating with others in the same location. In other words, communication over the ARPAnet had become, for many tasks, preferable to even walking next door to see a colleague.

In 1972, BBN even began attempting to form a sanctioned spin-off to commercialize the technology in the wider market. ARPA was spiritually supportive of this effort even if they did not fund it. In fact, helping get this firm, Telenet, up and running was Larry Roberts’ first job upon leaving ARPA. He had gone to ARPA to make inter-computer networking a reality. Upon leaving, he was heading into a role where he could attempt to get the technology to as many users as possible. Telenet would go on to have a level of success as a growing network provider — but would not be considered a smashing success. The fact that the ARPA-funded technology showed commercial potential as little six years after the project’s initial funding should also be considered a major success.

If one wanted to nitpick, one could point out that ARPA’s major intended application of the network was as a way to reduce computing costs. This never did become the primary use of the network. As computing costs gradually decreased, this particular use of the ARPAnet became relatively unimportant. The network found better uses.

ARPAnet Lessons Learned (and Caveats)

Many small-scale, PM-relevant lessons are present in this report. As one example, the BBN team maintaining extremely close connections with the academic community while implementing sooner rather than later proved to be a key decision. The academically-driven Network Working Group, with its debates on network protocols in the group’s early years, was at a stalemate regarding what protocols were best. The BBN team could have waited far too long to get the perfect set of protocol recommendations from the group. Instead, BBN implemented the NCP protocols and gradually, with much input from the group, upgraded to the much-improved TCP/IP protocols. Another interesting lesson was Roberts’ use of a demonstration as a forcing function to get all the nodes outfitted with the full suite of ARPAnet capabilities.

Minor learnings aside, it is important that the reader fully appreciate just how important having a contractor like BBN was to the success of the project. Not only was the research and prototyping stage of this difficult project done well and on schedule, but BBN seamlessly transitioned it into a utility. Appreciating why they were able to do this is extremely instructive.

The BBN team was staffed with top minds who cared deeply about the research enterprise but operated within the structure of a firm. Dave Walden, explaining why exactly BBN had earned its nickname as the “third university of Cambridge,” emphasized the following:

BBN provided a support structure for self-motivated researchers who were able and willing to find sponsors for the research they wanted to do. The work was not directed in a top-down fashion; within broad limits, senior researchers were free to pursue their own interests, if they could find the necessary financial backing.

To complement this, BBN also had the necessary staff, experience, and structure to manage successful projects as they grew into larger implementation projects. The firm’s hospital time-sharing work is just one example of BBN managing an implementation project with cutting-edge technology. A project at BBN could start in the deeply exploratory research stage and naturally scale into a utility.

The ARPAnet project showcased exactly what BBN was optimized to do. When given the right kind of problem and the right situation, BBN’s model could help facilitate marvelous feats. The ARPAnet itself, just like Edison’s light bulb 90 years prior, proved to be both a field-changing test bed for scientific experiments as well as a world-changing commercial technology. BBN was ideally suited to make a breakthrough of this nature possible. While the mission creep that comes with a profit motive can often steer a commercial firm off of the course of truly ambitious research, BBN was less susceptible to this in these years. If its research somehow became too derivative, the firm would not have been able to recruit staff anywhere near the competence level it sought.

If every scientific/commercial area had a small research firm like BBN, the scope of what would become possible with an ARPA-like PM’s funding would expand greatly. In one’s role as a PM, if the opportunity to help incentivize or facilitate an organization to operate a little more like BBN, there seem to be few downsides — and massive upside — to do so. The more novel a systems contract becomes, the more suitable it is for a firm like BBN.

BBN created such a compelling, applied environment in which to do research that they convinced many professors to give up tenured positions at MIT to come work at the firm. The firm knew exactly how to make use of these bright, applied researchers. Dave Walden put extreme emphasis on how much the “native wit” of the initial technical team he joined — Frank Heart, Severo Ornstein, and Will Crowther — was required to make the project a success, saying:

They are smart guys. There wasn't much theory in how you build a packet switching network. There was a communications theory, but that was all pretty abstract. One just got out there and did it. All the stuff that is now taught in courses in communications about networks and protocols and all of that, I would say we were mainly (as part of the entire community of the host people) inventing it. The academic analysis tended to come later in a lot of cases.

I’ll conclude with Frank Heart’s fond words describing the organization in “A Culture of Innovation:”

First and foremost, BBN was a great place for a technical person to work, and most people really liked working there. It was a middle ground between academia and the commercial world, with the meritocracy and individual freedom of the academic world, along with the potential reward structure and potential impact on the world of commercial ventures. In the case of the ARPAnet and the subsequent explosion of network activity, it was an extremely unusual opportunity for a technical person to “ride a rocket” of change in the world.”

Thanks for reading:)

Share

Subscribe now

If there is demand, I can write a separate piece on how BBN’s culture and management emphasis changed from those described in the piece as the decades wore on. But I considered that exploration out of scope for this particular piece.

Pattern Language Tags:

• Utilizing a contractor made up of individuals with research-style goals and training working within a ‘firm’ structure.

• Building out an experimental test bed for a research field.

• Promoting a coordination/service mechanism to reduce material costs and increase research feedback cycles.

This piece is a part of a FreakTakes series. The goal is to put together a series of administrative histories on specific DARPA projects just as I have done for many industrial R&D labs and other research orgs on FreakTakes. The goal — once I have covered ~20-30 projects — is to put together a larger ‘ARPA Playbook’ which helps individuals such as PMs in ARPA-like orgs navigate the growing catalog of pieces in a way that helps them find what they need to make the best decisions possible. In service of that, I am including in each post a bulleted list of ‘pattern language tags’ that encompass some categories of DARPA project strategies that describe the approaches contained in the piece — which will later be used to organize the ARPA Playbook document. These tags and the piece itself should all be considered in draft form until around the Spring of 2024. In the meantime, please feel free to reach out to me on Twitter or email (egillia3 | at | alumni | dot | stanford | dot | edu) to recommend additions/changes to the tags or the pieces. Also, if you have any ideas for projects from ARPA history — good, bad, or complicated — that would be interesting for me to dive into, please feel free to share them!

Specific Links:

The following oral histories and related materials were a joy to read through and primarily what the detailed ARPAnet portions of the piece were based on. The ARPAnet Completion Report was simply used to fill in the gaps. These give a far more detailed account of how things actually worked on the project than the completion report ever could.

• Frank Heart’s UMN Oral History

• Larry Roberts’ UMN Oral History

• Larry Roberts’ Computer History Museum Oral History

• Robert Kahn’s 1989 UMN Oral History

• Robert Kahn’s 1990 UMN Oral History

• Dave Walden’s UMN Oral History

• Severo Ornstein’s UMN Oral History

• Alexander Mckenzie’s UMN Oral History

  • Contains many insights on the customer/user service mechanisms BBN utilized as the network grew in the early-to-mid 1970s.

• Robert Taylor’s UMN Oral History

  • Taylor dives into why exactly he felt his tenure as IPTO chief was a good time to attack the network problem and how he went about the early stages of the work.

• Leonard Kleinrock explaining early ARPAnet node installation and testing

• The History of Telenet and the Commercialization of Packet Switching in the U.S.

  • For more on the early history of commercial packet switching, check out this IEEE paper in which Larry Roberts is a co-author

• Licklider et al.’s Libraries of the Future Report

• J.C.R. Licklider Oral History

General Links:

• ARPAnet Completion Report

  • This report was a treasure trove of information related to contracting details, specifics on technical accomplishments, and various graphics summarizing ARPAnet’s growth.

  • The report also highlights how the actual work deviated from the plan, what was added/removed, etc. as the project went on.

  • Lastly, the report lists many of the technical problems the BBN team had to work through at various points.

• A Culture of Innovation: Insider Accounts of Computing and Life at BBN

  • Besides being the source of many quotes and pieces of information in the piece, this is the most thorough account of how BBN actually operated on a day to day basis. Each chapter is written by a different, long-time (and often high-ranking) BBN employee.

  • As one example, the book contains a long section from Frank Heart on how overhead rates impacted project selection and how BBN’s billing system worked. Explanations like this for all sorts of aspects of the operation are included in the book. 

Pattern Language Tags:

• Building out an experimental test bed for a research field

• Pushing forward the technological base via speculative machine building contracts

• Putting out contractors for specific “integrators” in addition to prime contractors and basic researchers

• Using customer-relevant applications as a demand pull mechanism for a research field

• Setting up your testing environment in the same machines and locations you will deploy in

Introduction

DARPA’s customer relationships and the individual agency of its PMs are possibly the two biggest differentiators of the DARPA model from bigger budget science funders like the NSF and NIH. Even in DARPA’s most basic research funding decisions, the question of “how can this be used by the military services?” finds its way into the discussion. In some generations of the organization, being “useful to the military services” is interpreted quite broadly, with justifications such as, “what’s good for the nation will be good for DARPA.” This approach was often taken by DARPA’s Information Processing Technology Office (IPTO) in the 1960s. Arthur Norberg and Judy O’Neill — who wrote an in-depth history of IPTO from its earliest days, including interviews with prominent individuals such as Robert Taylor — summarized the priorities of early IPTO directors as follows: “A contribution to the country which included a contribution to the military is what IPTO directors sought.”  With work such as the development of VLSI computer chips, that ethos has often proved to be an extremely correct heuristic.

However, in different generations of the organization — whether driven by wartime, fiscal austerity in DC, or the preferences of individual directors — being “useful to the military services” has been interpreted more narrowly. The first prominent example of this is DARPA under Director George Heilmeier. Another example of an era of DARPA where this goal was interpreted more narrowly was during Tony Tether’s years at DARPA as Director — starting in 2001. In this era, Tether went as far as to make many funding decisions contingent on a specific armed service putting in a budget wedge to continue developing the project past a certain stage. Neither approach is right or wrong. ARPA-like orgs often need to be able to adapt to changing situations like these.

Regardless, whether your customer is the US military, medical patients, or a certain subset of academics, all ARPA-like orgs serve some customer. This piece will dive into how DARPA’s Strategic Computing Initiative (SC) served its military customers in the 1980s — an era in Washington DC that demanded DARPA justify most of its spending as having clear, near-term value to the armed services. 

Context

The three projects covered in this piece are all examples of DARPA’s attempts to find early military use cases of the exciting advances that had been coming out of their computing portfolio since the 1960s. Some pieces of the IPTO portfolio clearly followed Robert Kahn’s — who headed IPTO at DARPA — vision of “pushing” technology out into the world to find applications. Meanwhile, other areas of IPTO’s portfolio relied more strongly on using projects to stimulate demand “pull” for technological capabilities. 

Robert Cooper — the DARPA Director at the time — was the counterbalance to Kahn and his beliefs. The program that Kahn envisioned was, put plainly, not considered sellable in DC. To sell the program, Cooper helped center the pitch around three new military systems that would result from the program if its individual research programs panned out in a moderately successful way. The three early descriptions of the applications became the centerpiece of the report that pitched SC to Capitol Hill. These applications were meant not just to provide the non-technical politicians a clear sense of what they were paying for, but, also, Cooper wanted to use the applications to help steer IPTO’s research funding in what the SC management hoped would be a coherent and politically explainable direction. 

Beginnings of All Three Programs

From an intellectual standpoint, Khan was considered the figurehead of DARPA’s 1980s Strategic Computing Initiative. The initiative was meant to expand on the prior decades of computing research breakthroughs as well as find useful military applications for the growing body of work. However, it would be DARPA Director Robert Cooper  — who had a Ph.D. background but favored industry-focused, goal-oriented engineering research more than Kahn — who would succeed in selling the program to Washington. A large part of the reason he proved to be a better fit for this sales task than Kahn was because he, more than Kahn, was willing and able to outline specific application technologies that SC’s technology portfolio could contribute to once all the work had been done. Khan felt that technological development was uncertain and that doing this was something of a fool’s errand. Cooper also seemed to understand that what they did at DARPA was an uncertain enterprise, but, also, that if DARPA wanted an expanded computing budget, then this was the kind of pitching that DARPA would need to do.

Roland and Shiman — who wrote a history of SC and interviewed many of its key players in the process — described Cooper’s thought process:

“Technology base” sounds fine but it does not mean much to those outside the computer community. Just what will that technology base allow you to do? What is its connection to the real world? How will it serve the defense mission? What new weapons and capabilities will it put in the hands of the commander in the field? These are the real metrics that allow you to measure the worth of such an investment. And any such goals have to be time specific. It is not enough to say that a robotic pilot’s assistant will be possible at some time in the future; there has to be a target date. Between now and then there must be benchmarks and milestones, metrics of progress demonstrating that the program is on course and on time.

Clinton Kelly, who originally joined DARPA as a PM in its Cybernetics Office before moving to the Defense Sciences Office when Cybernetics shut down, was asked by Robert Cooper to help define the applications efforts. Kelly, an engineering Ph.D. who had also run a company related to automating intelligence decision-making, had been funding a variety of somewhat odd mechanical machines while at DARPA — such as a robot that could do backflips. Kelly was known to be a great technology integrator and organizer, so he was very well-suited to the task Cooper had assigned him. The three applications that Kelly helped outline incorporated what Kelly and others saw to be the likely capabilities that would come from Kahn’s technology base and would intrigue the military services. The three major SC applications that Kelly helped jumpstart through his work on Cooper’s report were Pilot’s associate, Battle Management, and Autonomous Land Vehicle.

Progress towards these three applications would often be used to help steer and benchmark the program in the coming years. Many of the benchmarks included in the original SC report — such as real-time understanding of speech with vocabularies of 10,000 words understandable in an office environment or 500 words in a noisy environment — were set with the needs of these applications specifically in mind. Stated goals like helping a fleet commander plan for multiple targets or providing the military the ability to send an unmanned reconnaissance vehicle across a minefield helped ground the vision of the program to its funders and customers who were less likely to see the importance of the stale computer benchmarks — such as those used in the computer architecture section of the SC write-up.

While Khan eventually relented, he still saw these applications as Cooper imposing a NASA-style of operations on DARPA. NASA focused on working towards challenging point solutions rather than broad, flexible technological development — as IPTO historically had. As the years went on and work progressed, work on these projects proved that Kahn and Cooper both had good points. Regardless, the applications proved successful in helping DARPA win over an expanded computing budget for SC. In 1983, work on the program commenced.

Pilot’s Associate Operations

The Pilot’s Associate (PA) application was meant to assist aircraft commanders in all cockpit decision-making and planning that could reasonably be augmented by sensors and then-modern computing. The PA planned to incorporate the new hardware, AI/software, and natural language understanding capabilities expected to emerge from DARPA’s funding of the computer technology base. DARPA hoped that the PA would eventually help the aircraft commander perform tasks like preparing and revising mission plans. They hoped for the system to be small enough to fit into a cockpit and outfitted with a natural language understanding system so a pilot could deliver some verbal commands while they used their hands to operate the aircraft. The PA would be a “virtual crew member.” There were even comparisons, at the time, to the system functioning as a pilot’s own R2-D2 unit — although others in the military pushed back on this specific analogy.

Once SC was announced, DARPA’s first step in working on the PA application project was contracting Perceptronics — a research company that DARPA was currently using as a contractor for its SIMNET computerized combat simulation training project — and other contractors to carry out exploratory studies in the area. These studies were meant to more concretely outline how the system would work, as the description of PA in the original SC plan was quite vague. These studies established that, most likely, five separate “expert systems” could work in concert to carry out the functions of the final PA application. 

In the era of strategic computing, “expert system” was a common term to reference software programs — often built by combining the efforts of programmers and specific domain experts — which used codified sets of rules to help carry out tasks in a faster and more accurate fashion. In their simplest forms, expert systems might just be made up of meticulously constructed “if, then” rules. Expert systems were, at the time, considered a branch of AI research — albeit a less exciting one than more generalizable AI systems to AI research purists. However, in an era when generalizable AI was often not living up to any level of commercial promise, expert systems programs, which nowadays would simply be considered “software programs,” were proving interesting to the military and large companies alike. These programs showed real promise in bringing the fruits of the AI research community to a wide variety of sectors and customers. Unlike more pure AI systems at the time, expert systems work had actually even proven profitable in the private market.

So, it is not surprising that PA’s project planning was going to rely on the use of the more application-ready expert systems paradigm. In the case of PA, the preliminary studies determined that the five specific separate expert systems that would make up the final software of the machine would encompass the following areas: mission planning, systems status, situation assessment, tactics, and front-end communication with the pilot. As the PA project went on, most of the direct planning for the PA system was done by scientists and engineers at the Wright Aeronautical Laboratory at the Wright Air Force base in Dayton, Ohio.  The Air Force’s excitement to largely take the reins of the project from its early days was likely due to the fact, in the years leading up to this point, the Air Force had begun developing its own labs to explore the application of AI to Air Force matters. Major Carl Lizza, who had a masters degree in computer science, was the program manager on Wright’s end for much of the project. On DARPA’s end, John Retelle was the program’s PM until 1987, when another DARPA PM took over. With high level buy-in and excitement from the Air Force, minimal direction was needed from DARPA on the direct development and implementation of the PA system. The customer buy-in from the Air Force was a quite positive sign for DARPA. 

Following the study stage of the project, the implementation of the PA program began in 1986. McDonnell Aircraft Company was the prime contractor for the air-to-ground system. Three total companies made up this portion of the contract. Lockheed was the prime for the air-to-air system. Eight companies and university research teams were on that contract. McDonnell was awarded $8.8 million (~$23 million today) from DARPA and Lockheed $13.2 million (~$38 million today). Each company was also willing to cover 50 percent of the costs of the development work because they felt working on the contract would give them an advantage in bidding on the big-budget contract for the Advanced Tactical Fighter which would be awarded in the coming years. These performers would look to take advantage of the equipment and methods that had been coming out of the DARPA portfolio as well as industry in recent years — employing LISP, Symbolics, and Sun machines as well as many of the new expert systems development methods which DARPA often funded.

The project would also make heavy use of expert boards in planning its work schedule as well as work with the customers in developing successive versions of prototypes. As is common in military R&D applications projects, the team leveraged expert advisory boards from academia and industry on an ongoing basis. Help from these sorts of boards seems less common when making small decisions in projects further away from implementation — such as those made in the Defense Sciences Office in comparison to the systems technology offices. Also, as this was absolutely a product for a paying customer, the performer teams heavily involved the users of the eventual product throughout the design process. In fact, much of the development work by the industry contractors looked quite similar to modern software development to modern readers. The project relied heavily on the rapid prototyping approach to managing its workflow and benchmarking its progress. Several demos were set to happen in the first three years of the project that would prove the system’s function and value. After several successful demos and the proof of concept were completed, the work could then progress to doing the development work to ensure that the system worked as needed in live combat situations. From a high level, that was how the project was meant to progress.

Non-PA, but PA-relevant research and engineering work that was in earlier technology readiness stages was overseen by other PMs in the SC portfolio. This bit of bookkeeping is of practical importance to how the projects were managed. The PA-relevant work on the broader technology base — that was still related to the application and might, at some point, make it into the PA system — generally fell to PMs in IPTO. These PMs working on other pieces of the broader portfolio did not “answer to” the PMs of the PA, Battle Management, or ALV applications, in spite of the importance of those three projects to SC. The work in these other PMs’ portfolios, of course, was meant to prove useful to projects like the PA. However, the PA team did not have any direct power to influence other PMs’ priorities to be more in line with PA’s direct needs. The PMs in the broader SC portfolio were meant to help develop better component technology to help projects like PA in the future, but they were generally able to decide on the best possible deployment of their own portfolios. 

As the PA project was in its early planning stages, it seemed clear that a sufficient level of expert systems knowledge and experience was generally available to make an application like PA feasible at the time. But at least two notable technical risks existed that the PA team was relying on developments from the broader SC portfolio to overcome. The first risk was the need for a computer to be developed that was powerful enough — and small enough — to make the system work in real time. It was not going to be clear exactly how much more powerful a computer would need to be to make the PA work until more expert systems development was done by contractors on the project. But it seemed clear that a computer about an order of magnitude more powerful might be necessary before the system could be considered deployable in a cockpit. The second area of notable technical risk was the need to develop a natural language understanding system that could process a basic vocabulary of at least 50 or so words in noisy, cockpit-like conditions. This area of technical risk was likely less important than the issue of computing power, but it was still quite important if the system was going to work the way those in charge of the program wanted it to. 

In spite of there being notable areas of component technology that required improvement, Director Cooper felt it was important that the SC applications programs begin roughly at the same time as its work developing the broader technology base. Politically, this increased SC spending package was meant to be about applications. Waiting years to start working on the applications projects was not really an option. So, work on the three applications projects began, more or less, right after SC was announced. The SC team, however, did go out of its way to ensure that the more basic research projects got off and running sooner rather than later. Meanwhile, the applications projects often had study stages and preliminary planning periods that may have taken a year or two. I do not get the sense that DARPA made similar attempts to speed this process along the way they did to ensure that computer architecture researchers, for example, got the machines they needed for their research sooner rather than later. 

In March 1988, the first major operational milestone for the PA project was hit when the Lockheed Aeronautical Systems Company successfully demonstrated the system it had developed in a non-real-time cockpit simulation. In this demo, the system successfully performed flight tasks like identifying and dealing with a stuck fuel valve, creating a new mission plan as the situation changed, advising the pilot on evasive action, and deploying flares. The machine was able to complete the simulation’s 30-minutes worth of real-time computations in 180-minutes — meaning, in this early stage, the system was about one-sixth as fast as it needed to be. The slow performance was not considered behind schedule by any means. Firstly, much of the software was not even run on the most powerful machines possible at this stage. Additionally, in the early internal writings on the project, it was believed that by around 1990, some parallel computing technology from the DARPA architectures portfolio would be advanced enough to transition the expert systems to run on those machines. 

This bit of planning is quite interesting. Those planning SC’s applications very clearly understood the pace at which computing hardware and architecture was advancing in these years and incorporated a level of anticipation of that progress into their planning. For context, in this period the BBN Butterfly computer — which was a coarse-grained parallel processor and not so powerful yet — was one of the few operational parallel architectures. Massively parallel architectures were still in their very early stages at the time SC was getting underway. By 1993, those at the helm of SC felt that small computers would be able to run the types of systems required to operate the PA in real-time and then some. 

It was the job of the other PMs in the SC portfolio to seed investments and help orchestrate progress via the performer community to make this projected computing progress a reality. I will briefly give the reader an idea of what some of this work looked like. In the period between the mid-1970s and mid-1980s, technological development was occurring elsewhere in the hardware section of the SC portfolio and in industry that would end up becoming far more famous than the PA project itself — but that would, of course, enable many projects like the PA to become a reality in the coming decades. Some of this progress was funded by industry on its own, some by government and DARPA dollars, and often both. While NASA and the Air Force were buying chips from the young Fairchild Semiconductors, DARPA money had been finding its way to Fairchild through projects like the ILLIAC IV (covered in the FreakTakes Thinking Machines piece). Texas Instruments was hard at work on many of its own R&D projects, but many were also pursued in concert with actors like DARPA. For example, in 1983 DARPA approved $6 million to TI to develop compact LISP machine hardware — implementing a LISP processor on a single chip — as long as TI would fund $6 million in simultaneous software research for the device. Projects like the PA were at the top of mind when funding this TI project. Additionally, DARPA had funded much of the early Warp computer research at CMU which the university, later in the 1980s, worked with Intel to help miniaturize in developing the iWarp — whose first prototype was operational around 1990. Since this area of the history of computing progress is extensive and, to some extent, understood by many reading this piece, I will say no more on the progress in computing power in this period. Suffice it to say that the actual progress of the field did turn out to be in line with the lofty expectations of those planning SC as they outlined projects like PA in the early 1980s.

As this computing work was progressing — which would help the PA project team overcome its first major technical risk — SC’s portfolio of work on the second major technical risk, speech recognition systems, was also moving along quite well. While many consider speech recognition systems a relatively modern phenomenon, research in these systems was progressing quite steadily for several different performers in the SC ecosystem in the 1980s. By 1987, CMU had developed a speech system that could recognize 1,000 words with an accuracy of over 90% at real-time speed. BBN had also developed a similar system, and IRUS, which possessed a vocabulary of approximately 4,500 words in 1986 and was generally considered ready for testing in applications like the Navy-relevant Battle Management (the substantially larger vocabulary is because Battle Management’s speech recognition system was meant to be used in less noisy environments). Unsurprisingly, much of the progress occurring in the area was aided by improvements happening in computer hardware — in this period largely due to successes in VLSI development (covered more in the FreakTakes MOSIS piece). 

The progress in component technology was happening on the side of the work being done by the teams working directly on the PA project. Extremely pleased with the 1988 demo which showcased the general feasibility and usefulness of the system in practical situations, and with much of the work on relevant component technology in the SC portfolio going according to plan,  the Air Force elected to have the Lockheed/Air Force development team begin adjusting their development efforts. They hoped to have a system that could be put into aircraft to help them fly missions within three years. After one more successful November 1989 demo which showcased some additional functionality — such as reasoning using more imperfect sensor data, re-implementing programs in faster programming languages, and updating the machines used to run certain expert systems to faster versions —  the Air Force shifted the project into its further development phases. The coming stages were meant to get the machine ready to perform in more battle-like timelines and uncertainty conditions.

Unfortunately, the project would not ever make it across the finish line. The project’s fate seems to have become clear around 1991. Much of the component technology was strong, and some of these component improvements and learnings would find their way into similar future systems. For example, the Lockheed team produced extremely useful new insights when it came to the systems engineering of multiple coordinated expert systems combined in a single application. When the expert systems were combined, there were issues with each expert system following its own objective function and sometimes giving the pilot conflicting advice. The Lockheed team came up with a concept called a plan and goal graph to help the system better infer the pilot’s true wishes and deliver advice that was more coherent and in line with the pilot’s wishes. However, certain vital pieces of functionality were just not coming together the way the team had hoped as they attempted to make the system work in real-world situations. Certain pieces of the project that appeared conceptually feasible in the early stages were proving to be hairier than had been anticipated. The technology base was simply not capable of doing certain key tasks like recognizing certain maneuvers and accurately predicting the geometry that would result. But, in many ways, the program had served its intended purpose in DARPA’s portfolio.

Wright’s chief engineer on the project, Captain Sheila Banks, and its program manager, Major Carl Lizza, seemed to clearly understand that while the PA might not make it into a standard Air Force cockpit, it should not be considered a failure from the point of view of Robert Cooper’s DARPA. The two concluded a 1991 paper they had written on the state of the PA program by writing the following:

In accordance with the program goal to provide a pull on the technology base, the program often needed nonexistent technology to implement the design. Uncertainty-reasoning approaches, pilot modeling techniques, and development tools are areas that lacked sufficient research to implement the Pilot’s Associate vision. Verification and validation of knowledge-based systems constitute another area that is lacking techniques and is quickly becoming a program issue. The actual implementation of a system of this complexity uncovers many gaps in technology still to be addressed by the research community.

The project helped facilitate a level of practical learning in the SC portfolio and provided a level of clarity to many basic researchers trying to understand what contexts, roughly, they were attempting to produce technology for. But the project did not result in any immediate revolution in how Air Force pilots operated in their cockpits.

Battle Management Operations

I will give a slightly more high level accounting of some of Battle Management’s operational details. In many ways, the operational structure of the project was similar to PA’s, but the project resulted in more of a clear-cut win in the short run. Where certain project details are either instructive or interesting, I will be more thorough. But I would rather leave more time for ALV as its messy history and complex legacy are extremely instructive. (For more operational details relating to specific hardware, expert systems, and contractors see this document from Battle Management PM John Flynn.)

The Battle Management application project was launched with the goal of deploying the power of expert systems to help Naval decision-makers make better inferences about enemy forces, keep better tabs on their own forces, generate strike options, come up with operations plans, as well as a variety of other tasks — which any modern reader surely knows computers could assist Naval leadership in carrying out. The vision of the system was quite similar to the PA, but in the context of the Navy rather than the Air Force. As was the case with the PA system, the Battle Management system’s wish list of capabilities would likely require improvements in overall computing power from the rest of the technology base. Initial estimates for the computing power required for the system were around 10 billion operations per second — approximately a one or two order of magnitude increase in peak performance in comparison to what was available at the start of the program. 

While Battle Management was initially conceptualized as one program in SC’s founding document, it quickly became three separate programs: Fleet Command Center Battle Management (FCCBM), Carrier Battle Group Battle Management (CBGBM), and AirLand Battle Management. In this section, I’ll focus on the two projects that focused on the Navy as a customer —  FCCBM and CBGBM — for brevity’s sake. Both systems, similar to PA, were meant to use a GUI, natural language system, and expert systems to help assist Naval commanders in coming up with and executing operational strategies on the ship level (CBGBM) as well as on the theater level (FCCBM). 

CBGBM — the system that would be installed aboard a ship — was considered a continuation of a program by Allen Newell and others at CMU that had been ongoing since the 1970s. With the announcement of SC, CMU had proposed continuing the work area under the well-funded SC umbrella. DARPA believed this was a good idea and helped coordinate Naval buy-in to the project. The Naval Ocean Systems Center in San Diego officially bought-in to the CBGBM project early on. Similarly, for FCCBM — the fleet level battle manager — DARPA was able to secure buy-in from the HQ of the Pacific Fleet at Pearl Harbor. As the Air Force did with PA, the Navy agreed to provide a level of financial support to these projects at this stage. With that support DARPA had early buy-in from its customer into the project. DARPA’s working relationship with the Navy on the project would prove to be an exceptionally functional one — enabling a far more effective prototyping approach than was seen in the PA project.

The FCCBM’s (the system meant to help make plans on the fleet level) rapid prototyping strategy, implemented under PM Al Brandenstein, was executed in coordination with the team at Pearl Harbor. Roland and Shiman describe the project’s prototyping strategy as follows:

A test bed was established at CINCPACFLT [Pearl Harbor] in a room near the operational headquarters. The same data that came into headquarters were fed into the test bed, which was carefully isolated so that it could not accidentally affect real-world activities. Experienced Navy officers and technicians tested it under operational conditions and provided feedback to the developers. The resulting upgrades would improve the knowledge base and interface of the system.

This extremely tight customer relationship in the development process ensured several things in the early stages of the project: faster feedback cycles, increased buy-in from the customer, and extreme clarity on the requirements of the system. 

As the implementation stages of the projects got underway, the FCCBM team installed its computer hardware at Pearl Harbor in 1985. By March 1986, the test bed environment had been completed. With that infrastructure and scaffolding in place, the expert systems contractors could ensure their systems were tested on data with the highest level of fidelity to real-world data possible — because it was real world data. In June 1986, the Force Requirements Expert System — the first of the five expert systems for the project — which had been in development since 1984, was installed and began testing in the test bed. By the end of August the system was successfully demoed to DARPA and Naval leadership. This demo, as well as much of the work on the project, was heavily based on the development of BBN and the Naval Ocean Systems Center’s IRUS natural language generator — a project from DARPA’s technology base. By this time in 1986, IRUS had already achieved a vocabulary of almost 5,000 words and — being an experimental system developed with the knowledge that the Navy would be its initial user — even had a vocabulary and language generation style that could comprehend and respond to queries in language familiar to Naval operations staff.

By 1987, the FCCBM had already proved its worth to Navy staff. The system was capable of performing tasks that used to take fifteen hours in as little as 90 minutes. Research teams frequently seem to generate results like this on paper that either don’t turn out to be true in practice, or, alternatively, are never installed due to fear from customers that the paper results won’t hold up in deployment. FCCBM was able to avoid both of these issues. Since the system was already installed at Pearl Harbor and working on its actual data streams, the results were far from mere paper results. Additionally, the pain and annoyance of initial installation were much less of a factor in the decision than they might usually be because the system was already installed at Pearl Harbor. The first expert system quickly went into operation helping monitor ships in the Pacific while some of the other expert systems — which were further behind in readiness level — were tested until they were deployment ready.

Following a similar approach, CBGBM (the Battle Management system designed to be put on individual ships) was installed for testing on a ship in early 1986. Within a few months, the use of the machine in fleet exercises convinced the Navy that a modified version of the system would be as useful as they’d hoped. Improvements were made to the system before going out on a six-month tour later that year. The technology, at that point, was very much still in the prototyping stage because it could not operate in real time. But as a development project, CBGBM was proving its utility to the Navy. 

While these two projects generally progressed quite smoothly, there was one notable friction on the operations side of the project: classified information. Some individuals who worked with the contractors did not have security clearances because they couldn’t get them. Others just didn’t want them. The university research contractors, in particular, often didn’t want clearances. CMU, as an example, was one of the systems integrators for the CBGBM project and had a very good working relationship with the Navy, but largely did not have security clearances. So, this meant that one of the main systems integrators could not — for the most part — actually look at any of the golden, real-world data in the database at Pearl Harbor. Instead, their workflow had to look something like the following: discuss with the Navy what sort of information would be reasonable to use, do their R&D using fabricated data that they hoped would look like the real thing, and twice a year hand over their updated system to the Navy team and wait to see how it worked. While that was a hassle, it seemed to be a small price to pay for having the best minds on the project — the CMU team's work from campus seemed to port over to the real systems on the ships well enough.  So, the parties made it work. The Navy would even send staffers to CMU for long stays to ensure that as much knowledge exchange as possible was done. 

The work on the Battle Management applications, similar to PA, was enabled by the development work going on in the greater SC portfolio. This is particularly true of the following areas of the portfolio: computer hardware, natural language understanding, and speech recognition. Beyond the improvements in computing hardware going on in the broader SC portfolio — which was explored in the PA section and had similar implications to the Battle Management projects — much of the speech recognition and natural language understanding (NLU) research in the SC portfolio was done explicitly with Battle Management in mind. 

I will provide some clarity as to what some of this work looked like — which I will also cover more in a coming piece on DARPA workhorse computing contractor BBN — since several researchers and ex-DARPA PMs have personally expressed surprise that DARPA’s speech recognition and NLU portfolio has operational successes going back four decades. To many, this area of engineering feels quite young since many of the best methods in use today have been enabled by quite recent technology. However, as far back as the late 1970s, contractors in the speech recognition and NLU areas of the DARPA portfolio were building quite impressive applications with what — by today’s standards — would be considered somewhat quaint computing power and natural language methods.

While DARPA had largely reduced the contracts it let in speech recognition in the mid-1970s under Director Heilmeier, some speech recognition work continued in this era that would prove key to programs like BBN’s IRUS system that was deployed in the Battle Management system in the late 1980s. Several key pieces of work going on in the interim were: Baum et al.’s continued work developing hidden Markov models (HMM) at the Institute for Defense Analysis, Jim Baker’s HMM-enabled Dragon speech recognition system built at CMU, and BBN’s applied work on various speech recognition and compression projects for paying customers. On the natural language understanding front, BBNers and researchers related to BBN — such as the Harvard and MIT grad students of professors who worked part-time at BBN — made new discoveries and first-of-their-kind implementations in augmented transition networks and structured inherited networks in this period. Both of these technologies — which seem to have been replaced by more advanced methods by the late 1990s — proved extremely useful in BBN’s IRUS system which proved the feasibility of certain natural language tasks in the context of applications like Battle Management.

Even while DARPA was not giving contracts for speech recognition, believers in the technology at frequent DARPA contractors like MIT, CMU, and BBN that believed in the technology continued to push the field forward in the meantime. By 1986, the IRUS system already had a speech recognition vocabulary of almost 5,000 words and an accuracy rate of over 90% which was enabled by some of BBN’s work on alternative models. The relatively massive size of IRUS’s vocabulary — in comparison to the vocabulary of under 100 words in the PA’s speech recognition system — was largely due to the noisy conditions of Air Force cockpits. This constraint limited the scope of the PA’s vocabulary to simple vocabs and something like fifty commands along the lines of “Fire!.” In contrast, DARPA knew from the beginning that Battle Management would be a more conducive environment for speech recognition systems. So, from the beginning, SC management had ambitions of building a system capable of utilizing a vocabulary in the 1,000 to 10,000-word range in the next five years or so. 

As the battle management application projects were ongoing, there were eight different contractors across nine projects in the broader DARPA ecosystem pursuing various research projects in the area of speech recognition. To help wrangle all of these findings into technology that could be folded into future generations of battle management systems, CMU was given a contract as an “integrator.”

In the SC portfolio, integrators were sometimes given separate contracts and tasked with the primary responsibility of finding ways to integrate the cutting-edge findings from the performers’ work in the broader technology base — who were supposed to be keeping applications in mind anyway — into the final applications. An integrator contract was given in the PA project as well. In that project, five groups of performers who were working on eight different DARPA programs fed relevant results to TI, who was tasked with integrating the new knowledge into subsequent generations of the PA. In theory, with prime contractors like Lockheed and university researchers doing their jobs, this role should not be necessary. But in practice, the integrators could often make a big difference in the level of ambition of the integration work.  

In the end, the speech recognition work went so well that SC management never needed to change the initial, lofty goal of achieving a system with a vocabulary of roughly 10,000 words. And that's saying something; oftentimes, lofty goals are set at the beginning of projects like this with the understanding that, after a period, the PM, relevant performers, and DARPA management will touch base and agree on more reasonable goals. However, revising the goals was not necessary in this case.

The other area of battle management’s language work — NLU — also had integrators that it leaned on. The broader NLU applications work was made up of two integrators and five other teams of performers. The integrators were both teams. The first team, in charge of integrating text understanding, was made up of NYU researchers and the Systems Development Corporation — a frequent DARPA contractor since its systems engineering work on the SAGE air defense system. The second team, in charge of integrating broader work in natural language query processing, was made up of BBN and the Information Sciences Institute (who is the star of the FreakTakes piece on MOSIS). This natural language understanding work also met the initial, non-quantitative goals that SC’s planners set out: the final users were able to provide the system new information and get understandable feedback in return explaining the system's “thought process.”

The Naval Battle Management projects clearly accomplished their short term goals. The primary goal of the program, on paper, had been to help the Navy with supporting and planning in areas where time constraints are extremely critical. And the systems did just that. For some important tasks done at locations like Pearl Harbor, they reduced the time it took to do certain planning tasks from 15 hours to 90 minutes — along with the increase in computational reliability that comes with using a computer. Additionally, these Battle Management systems were in place and working within five years. This not only meant that the projects were a success as quickly as possible, but also that the projects helped DARPA Directors in doing things like justifying budgets and convincing increasingly austere Washington decision-makers in the late 1980s that their work was saving money for the taxpayers even if it often looked somewhat speculative. In many ways, Battle Management was the PR success that the ALV was hoped to be.

Autonomous Land Vehicle Operations

The third application placed at the heart of SC’s founding document was the Autonomous Land Vehicle (ALV). As the SC program wore on, ALV would also come to be one of SC’s biggest disappointments — at least in the short term. The ALV was envisioned as an autonomous recon vehicle that could navigate battlefields — obstacles and all — to conduct reconnaissance missions and report its findings back to soldiers who did not have to leave safer positions to obtain the information. 

From a mechanical perspective, DARPA hoped for a machine that could travel over thirty miles in a single trip with top speeds of almost forty miles per hour. Of course, machinery existed that could mechanically hit these benchmarks. However, DARPA wanted a machine that could do all of this while being autonomously operated. To do this, many computational abilities would need to be developed for the system. The computational tasks required for an ALV to successfully work in battle included: route planning, navigating via visual analysis of physical landmarks, rerouting and avoiding surprising obstacles that were not in the route plan, ingesting useful information on the target, and transmitting that information to a military team. At the time, it was believed that performing these tasks would require an expert system that could run as many as three times more rules per second than current expert systems. Additionally, DARPA believed that the computer required to run the machine would not just need to be small — something like 4 ft x 4ft x 4ft — but would also need to be powerful enough to perform ten to one hundred billion von Neumann instructions per second. This was a tall task, as contemporary von Neumann machines were generally limited to around thirty or forty million instructions per second. This need for a three-order-of-magnitude improvement in computing meant that the ALV project achieving its goals relied even more strongly on improvements in computing power than the PA and Battle Management. 

The project sought to tackle problems in computer vision that modern autonomous vehicle research groups are still working through today. However, Clinton Kelly was confident that enough leg work had been done on the research front for DARPA to begin creating a program as a technology pull to bring DARPA’s vision work to practicality. This confidence derived in no small part from DARPA’s Image Understanding program which had kicked off in the mid-1970s. The Image Understanding (IU) program — sponsored by IPTO and then-Director J.C.R. Licklider — was formally spun up as a cohesive program on the heels of progress in DARPA-funded projects like DENDRAL in the 1960s.

DENDRAL was not computer vision per se — in that it could not actually ingest a photo — but the system could utilize a text representation of some molecular structures and used expert systems to perform analyses on them. Projects like this demonstrated that, once information taken in visually can be coded symbolically, systems were in progress that could make intelligent use of the information. Somewhat naturally, “signal-to-symbol mapping” became a key point of emphasis in the IU program — which would allow visual information from cameras or sensors to be encoded in such a way that systems could act intelligently on the information. Somewhat surprisingly, there were even some practical successes from the early-IU years. One example was DARPA and the Defense Mapping Agency’s (DMA) joint funding of a system — housed at the Stanford Research Institute — to automate certain cartographic feature analysis and assist the DMA in certain mapping and charting functions. However, most of the work done by the performers in IU in this period was quite far from applications largely because the field was young and there were not many clear-cut applications for the work just yet. However, it should be noted that as IU was getting underway in 1975, Licklider and others hoped for the work to progress in such a way that applications were within reach by 1980. 

Even with the ALV project kicking off in 1985, ten years after the start of IU, many veteran vision researchers felt this push for an ALV-type application might be a bit premature. Kelly and Ronald Ohlander — the DARPA PM who oversaw vision-related projects in this era — polled some veteran IU researchers to ask what they felt would be reasonable benchmarks and timelines for the project. Kelly told Roland and Shiman in an interview that he felt the answers were far too conservative and that he “took those numbers…and sort of added twenty percent to them.” When the researchers pushed back, they settled on a  compromise. Ohlander noted to Roland and Shiman that he thought the program to be “a real stretch” — far more than the other applications.

This general sentiment could also be seen with ALV’s lack of early buy-in from its intended customer: the Army. The Army was the intended customer for the ALV, but Army leadership saw no acute need for a robotic vehicle at the time and, thus, did not commit to helping fund its early development. The project, however, was seen by Kelly — and likely Cooper — as a fantastic pull mechanism for DARPA’s image understanding work. DARPA — as it did with other projects in this period such as the ILLIAC IV and Connection Machine — was hoping that providing funding and a contract for a machine with clear goals that required improvements to its component technology could help bring a young area of research to a new level. As Richard Van Atta wrote in one of his technical histories of DARPA:

Before the ALV project, the vision program had centered on defining the terms and building a lexicon to discuss the mechanics of vision and image understanding. The ALV was one of the first real tests of IU techniques in practice.

Even if the technology proved to be out of reach of his generation of researchers, Ohlander acknowledged that the researchers’ work deploying their ideas on a real-world test bed would expose the shortcomings of their approaches and help move the field along faster than it would otherwise.

With the kick-off of SC, funding for all vision-related work in the SC portfolio expanded rapidly. IU’s budget abruptly increased from its usual $2-$3 million a year to $8-$9 million a year. The ALV application program itself was, to some extent, co-managed by Clinton Kelly as well as its official PM William Isler.  They planned for one prime contractor to be in charge of engineering the vehicle and gradually incorporating the new component technology from the technology base into the machine on an ongoing basis. The program managers felt that one of the major aerospace companies from the regular defense contracting pool would be a good fit for the project. While these groups did not have experience in computer vision — almost nobody did — they did have experience overseeing projects on this scale of complexity and had all proven quite adept at large-scale systems integration. Some of these companies, at least, had experience with machines like remotely operated vehicles and basic forms of AI systems. As they were submitting proposals to DARPA’s January 1984 qualified sources sought, these aerospace companies understood that the project would be carried forward with whichever one of them won in partnership with at least one university department. The partnership was intended to help promote the integration of the expanding capabilities of the DARPA technology base into prototypes of the ALV as quickly as possible. 

The ultimate winner was Martin Marietta — which would later become the “Martin” in Lockheed Martin.1  The contract was to last 42 months and was for $10.6 million — with an optional extension of $6 million for an additional 24 months. DARPA chose the University of Maryland (UMD) as the university research partner to assist the company in folding technology improvements from the technology base into the machine in the early stages of the project. UMD was a well-known entity to the AI portion of DARPA and its PMs as it was one of the primary university research centers contributing to IPTO’s Image Understanding work. So, the university’s primary task was to get the practical vision system that they had been working on folded into the machine so the Martin team could begin working towards the goals of its demos as early as possible. 

As implementation work got underway, the ALV quickly adopted an accelerated schedule. Towards the end of 1984, DARPA moved up the date of the machine’s first demonstration to May of 1985 — six months before the previous initial demonstration. The vehicle would have to identify and follow a basic road in a demo with no obstacles of note. As Martin Marietta rushed to quickly assemble its first vehicle, it largely relied on off-the-shelf components that were tried and true. The mechanical parts such as the chassis and engine were standard, commercially available parts. The software and hardware for the machine’s computing were not exactly cutting edge. One of the two types of computers chosen — a VICOM image processor — may have been chosen because one of the University of Maryland Researchers had extensive experience working with the machine and stood the best chance of quickly getting the vision program up and running with familiar equipment. The second type of computer — Intel’s single-board computers, which were more or less personal computers — was used to control the vehicle itself. The camera that fed into the vision system was a color video camera mounted at the top of the vehicle and paired with several charge-coupled devices to transmit the visual information to the computer in a usable format.

The first demo was quite basic. The goal was for the machine to follow the center line of a road at Martin Marietta’s testing facility for about 1,000 meters. To give the reader an idea of how the vision system encoded real-world image data in a usable way, please see the following figure provided in a 1985 Martin Marietta report explaining what a digital scene model would look like for a given road image.

With basic image processing, some engineering cleverness, and standard geometry for calculating things like distance traveled and curve angles for steering, the machine seemed like it might be able to navigate a basic course quite well. The machine was powered by two major categories of visual decision-making software. The first category was the software dedicated to understanding exactly what the ALV’s camera was seeing. This software processed and enhanced the images so it could then use some of UMD’s edge-tracking algorithms to find the various boundaries in the image. Using these boundaries, the software calculated where the center line seemed to be. Another special algorithm would attempt to plot a 3D grid.

This work was passed to the second software system which was meant to perform some basic reasoning tasks using the current 3D grid model as well as the 3D grids from the timestamps immediately preceding the current one. At this stage of the process, the reasoning system largely attempted to assess the reliability of the current image, given what it had seen in the previous images. If the image seemed unreliable, the system attempted to make up a more believable path that fit with the images seen previously. If this switch happened, in that moment the reasoning system was seen as driving the vehicle rather than the vision system — since the vehicle was operating off of a pre-stored model rather than the current, low-confidence model built by the vision system. This backup plan was necessary for achieving the goals set by the demo because, given the slow processing speeds, the vehicle’s speed would often outrun its visual processing. So, this was necessary to keep the vehicle moving without stopping too frequently or going too slowly. 

The first demo — a standard road-following task — went off rather successfully. The vehicle traveled the approximately 1,000 meter course in about 1,000 seconds and did so traveling about 85% of the way under vision control rather than reasoning control. The machine did this while generating over 200 scene models and trajectories to help inform the vehicle along the way. On the heels of this success, the Martin and UMD team were right back to work because the next demonstration was in six months. And this demonstration was a test that was considered less of a layup than the first one had been. The course for the demonstration — which the ALV had to navigate at 10 km/h — required that the ALV navigate a sharp curve at least 3 km/h, progress along several straight stretches of road, and, lastly, stop at a T intersection to turn around and repeat the course. It had to do all of this without the reasoning system overriding the vision system.

To do this, the team would have to add more sensor information than just the single video camera to take in all the required information. Additionally, they had to re-work the vision software to somehow make use of all of this new information. In attempting to navigate this much harder problem, Martin relied on a DARPA-funded laser range finder that had been developed by the Environmental Research Institute of Michigan. The device could be attached to the front of the vehicle and was used to scan — up and down as well as side to side — the terrain in front of the vehicle to identify obstacles and other important features. Those working on the software attempted to find a way to combine this knowledge into a single 3D model of the scene. This knowledge was combined with the reasoning system which now contained a digital map of the track itself which the machine could use as a reference — the Engineering Topographical Laboratory had developed the map. The vehicle, impressively, passed this second test as well in November 1985. At this point, the ALV application seemed to be making extremely satisfactory progress. The technology the ALV used in these early demos — including the laser range finders — was not exactly cutting-edge, but the machine was hitting its practical benchmarks and hopefully could begin subbing in more cutting-edge component technology as the demonstration benchmarks continually raised the bar for the ALV. 

While the work on these early demos was happening at Martin Marietta, Martin and other ALV-related component contractors were also holding meetings at regular intervals — often quarterly or as frequently as monthly — discussing project needs, developments in the technology base, and how to delegate tasks. These conversations with research teams working on broader vision work were particularly important for work on the demos scheduled to happen in later-1986 and beyond — when it seemed like Martin would begin having acute needs to integrate more work from the technology base into the ALV to meet the goals of the demos. Much of what went on at these meetings was not so different from what happens at many DARPA PI meetings — with the standard presentations of progress and teams getting on the same page.

One logistical point of particular note was that, as some of the higher level planning meetings played out, the project structure was almost very heavily shaped based on the arbitrary preferences of the component contractors.  The contractors who found themselves working on the reasoning system vs. the vision system had very different interests and incentives. Most of the reasoning contractors were commercial firms and had the incentive to fight for a broader scope of work and more tasks getting assigned to the reasoning system because that meant better financial returns to the company. Most of the vision contractors, such as the University of Maryland, were academics and largely content to let that happen — or, rather, not as committed to fighting for a broader scope of work as the companies — if it meant they got to focus primarily on specific vision sub-tasks that were more in line with the size and scope of projects academics often undertook. For example, the private companies felt that building up much of the systems mapping capabilities, knowledge base, and systems-level work should fall to them.

CMU and one of its project leaders — Chuck Thorpe — seemed to be the strong voice of dissent on many of these matters from the academic community. Ever since CMU had become a major contractor in DARPA’s Image Understanding work, it had taken a rather systems-level approach to attacking vision problems — working with civil and mechanical engineering professors like Red Whittaker to implement their vision work in robotic vehicles — rather than primarily focusing on the component level. The CMU team far preferred this approach to things like paper studies — in spite of the increased difficulty. So, as the project progressed and these conversations were ongoing, both sides had at least one party fighting to work on component work as well as some amount of integration work prior to Martin’s work integrating components in the last mile — often right before a demo.

While that future planning was taking place in the background, it was becoming clear that Martin Marietta’s pace of actual technical progress was not as positive as the demos had been. Their work implementing basic vision algorithms proved finicky as small changes like the lighting, road conditions, and shadows caused the vehicle to behave very differently from one day to the next. Each new software program or change seemed to cause a new problem which took time to debug. It was messy work. Additionally, to the dismay of Martin with its strict demo timelines, it began to seem like a large number of specific vision algorithms might be required to handle different, niche situations such as even conditions, uneven conditions, etc. This would take significantly more work than if one unified vision approach could cover most situations — as the team had initially hoped. 

Meanwhile, demo timelines began to get tighter and tighter. More demo-like activities kept being added as more PR was brought to the program by Clinton Kelly — news releases were issued, the press was invited to watch some of the vehicle's runs, etc. In the midst of all this demo pressure, the ALV project began to embody more of a negative “demo or die” principle than embodying a pull mechanism to effectively move the technology base forward. As Martin’s actual progress slowed and the demo schedule tightened, the company began to focus only on achieving the benchmarks of the next demonstration at the expense of developing truly useful, new technology. For example, for one of the demonstrations, the obstacle avoidance algorithm — put together by one of the component technology contractor companies — worked in such a narrow set of circumstances that it could not even be used successfully in a parking lot, but it could work on the roads the demonstration took place on. 

As Martin was rushed in these middle years of the program, the ALV was clearly not acting as the test bed for the SCVision projects — the name encompassing much of the vision work in PM Ronald Ohlander’s AI portfolio — that it was meant to be. Much of Martin’s staff time and equipment was dedicated to hitting the next demo and not finding adequate time to serve this more future-looking function. It does seem that the team was truly busy and not just blowing the researchers off. By April of 1986, the 1.5-year-old ALV had logged 100,000 miles and needed to have its engine replaced since it had worn out. In one two-month period of testing prior to a demo, 400 separate tests were conducted using the machine. Preparations for a subsequent demo might begin as much as six months before — often not long after the last demonstration had just finished. In essence, things were becoming very crunched at Martin.

In 1985, in the period between the first and second ALV demonstrations, Ronald Ohlander — a former student of Raj Reddy at CMU — decided it might be necessary to rely on a contractor separate from Martin Marietta to integrate the component technology improvements from the technology base more ambitiously. Ohlander wanted to issue a contract for a scaled-down version of the ALV operating from within the SCVision program. The integration contractor in charge of the machine would be in charge of testing the computer vision algorithms of other research teams that fell under the SCVision umbrella.

Ohlander, in doing this, adeptly observed that the prime contractor on the ALV project might be under pressure to meet its contract milestones and not be sufficiently motivated to fold the most cutting-edge component technology coming out of his SCVision program into the ALV. In addition, he seems to have felt a level of unease relying on a machine, contractor, and program management from a different DARPA office — Isler and Kelly largely operated out of the Engineering Applications Office —  to demonstrate the usefulness of the investments coming out of his own office — IPTO. Ohlander explained the decision to Roland and Shiman in an interview, stating:

It wasn’t to outshine them or anything like that…but we were…used to doing things in universities with very little money and getting sometimes some startling results. So I…had a hankering to just see what a university could do with a lot less money. 

In the end, the contract would go to Carnegie Mellon. CMU had already, unlike most of the component contractors in the SCVision portfolio, been testing its component technology on its own autonomous vehicle: the Terregator. CMU’s used the Terregator — for often slow and halting runs — to test the algorithms developed for its road-following work. The Terregator proved extremely useful to this research, but its use had also helped the CMU team learn enough to outgrow the machine.

In CMU’s ongoing efforts to implement sensor information from a video camera, a laser scanner, and sonar into one workable vehicle, the CMU researchers had come to realize that they needed funding for a new vehicle. CMU needed a vehicle that could not just accommodate the growing amount of sensor equipment, but also one that could carry out its tests with computers and grad students either on board or able to travel alongside the vehicle. This would allow bugs to be fixed and iterations between ideas to happen much faster. The vehicle that the CMU computer vision and robotics community was casting about trying to get funding for would come to be known as the NavLab. Fortunately, what they were seeking money for seemed to be exactly what Ohlander felt the SCVision portfolio needed. The funding for the NavLab began around early 1986. The funding set aside for the first two vehicles was $1.2 million, and it was estimated that any additional NavLab vehicles would cost around $265,000.

As Martin was somewhat frantically taken up with its demo schedule, CMU, with its longer time windows, was  free to focus on untested technology as well as technology that ran quite slowly but seemed promising in terms of building more accurate models of its environment. As the CMU people saw it, if anything proved promising they could upgrade the computing machine used to operate that piece of the system later on. The NavLab team was working closely with CMU’s architecture team that was developing Warp systolic array machines from within Squires’ computer architectures portfolio. The CMU architecture team was developing its equipment with vision applications in mind as an early use case. As a result, they were happy to change the order of functionality they implemented in order to account for the acute needs of the SCVision team. Additionally, if a piece of vision technology proved promising to the NavLab that could not be sufficiently improved with the immediate help of a Warp or some other new-age computing machine, the vision researchers likely figured that a piece of hardware powerful enough would likely exist soon to help push the idea along — even if not in the next year. Lacking the hard metrics of yearly demos with speed requirements loosely matching how fast the vehicle would need to go in the field, the CMU team could afford to think more along the lines of  “how do we build a machine that is as accurate as possible, even if it moves really slowly today?”

As CMU took up the mantle of building SCVision’s true test bed vehicles, it also took over the role of SCVision portfolio’s true “integrator” for Martin’s ALV. Carnegie submitted a proposal to carry out this integration work along with building the NavLab. From a high level — in spite of neither the ALV nor SCVision projects having set aside money for an integration contractor — this made sense. Martin was meant to integrate work from the technology base on its own, but around mid-1985, it was becoming clear that they were not doing an effective job on their own. Academic departments were often close to the evolving frontier of their own field, but often did not have the project management skills and scaffolding to do great integration work. CMU was somewhat exceptional in that it did this sort of thing quite frequently. Clinton Kelly, while apparently not thrilled about the idea since he was a massive supporter of the ALV project, agreed to the arrangement. 

As 1986 went on, Martin Marietta was able to overcome issues with its vision system well enough to continue passing its demos which required the machine to go on larger tracks at faster speeds — such as 20 km/h on a 10km track with intersections and some obstacles. As 1987 approached, plans were being made to upgrade the computing equipment in the machine — some of the new computers were developed in Squires’ architectures portfolio. Additionally, as the demos in the coming years were slated to take a marked step up in terms of the complexity of their missions and terrains, the Martin team’s relations with CMU as the new integration contractor — a bit cold at first — warmed considerably.

Personnel from both orgs kept in regular touch and visited each other’s sites. Martin even went as far as to enroll a member of its staff as a grad student under Chuck Thorpe to serve as a liaison between the two. In mid-1986, CMU began helping Martin install software for map planning, obstacle detection, sensor calibration, and new image recognition functions that it had developed and integrated on its own machines on campus.  This period also saw specialized tools developed so CMU could more easily run its code without alterations on Martin’s machines — which ideally would have been done earlier in the project, but better late than never. The machine was steadily improving. In the summer of 1987, when one of the private sector contractors — Hughes — was at Martin’s Denver testing facility to test its planning and obstacle avoidance software, the ALV made its first extended drive in country conditions on a route it had selected for itself using only map data and sensor inputs. As Roland and Shiman wrote of the rather triumphant run:

The vehicle successfully drove around gullies, bushes, rock outcrops, and steep slopes.

However, that mid-1987 period would prove to be the high water mark of the ALV application program. After the ALV’s November 1987 demo, a panel of DARPA officials and technology base researchers met to discuss phase two of the program. Roland and Shiman described the titanic shift in how the ALV project and the SCVision portfolio operated in the wake of that meeting as follows:

Takeo Kanade of CMU, while lauding Martin’s efforts, criticized the program as “too much demo-driven.” The demonstration requirements were independent of the actual state-of-the-art in the technology base, he argued. “Instead of integrating the technologies developed in the SC tech base, a large portion of Martin Marietta’s effort is spent ‘shopping’ for existing techniques which can be put together just for the sake of a demonstration.” Based on the recommendations of the panel, DARPA quietly abandoned the milestones and ended the ALV’s development program. For phase II, Martin was to maintain the vehicle as a “national test bed” for the vision community, a very expensive hand servant for the researchers.

For all practical purposes, therefore, the ambitious ALV program ended in the fall of 1987, only three years after it had begun. When phase II began the following March, Martin dutifully encouraged the researchers to make use of the test bed but attracted few takers. The researchers, many of whom were not particularly concerned about the “real-world” application of their technology, showed more interest in gathering images from ALV’s sensors that they could take back to their labs.

The cost of the test bed became very difficult to justify. Phase I of the program had cost over $13 million, not a large sum by defense procurement standards, perhaps, but large by the standards of computer research. Even with the reduced level of effort during phase II, the test bed alone would cost $3–4 million per year, while associated planning, vision, and sensor support projects added another $2 million. Perhaps most disappointingly for Kelly, the army showed little interest in the program, not yet having any requirement for robotic combat vehicles. One officer, who completely misunderstood the concept of the ALV program, complained that the vehicle was militarily useless: huge, slow, and painted white, it would be too easy a target on the battlefield. In April 1988, only days after Kelly left the agency, DARPA canceled the program. Martin stopped work that winter.

Autonomous Land Vehicle Results

The Autonomous Land Vehicle hit its agreed upon, quantitative benchmarks in its early demonstrations. It also incorporated many technologies from the SC technology base into successive iterations of the ALV such as BBN’s Butterfly processor, CMU’s Warp, and other vision technologies that CMU and others had helped develop. If the Army had been excited by the far-off sounding application and the ALV’s progress in the early demos, there would likely have been a much higher chance that the program continued on schedule. However, since the Army was not yet interested in autonomous recon vehicles, the ALV’s biggest service to DARPA and its portfolio was as a technology pull mechanism to help move its technology base forward in a coordinated fashion. And since Martin Marietta had proven rather lackluster in the integration department, DARPA’s decision in late 1987 to wind down the ALV project and, instead, put more emphasis on the NavLab makes sense.

Work in the spirit of the ALV largely lived on in CMU’s NavLab efforts. CMU’s somewhat unique setup — running an engineering research program equipped with de-facto program managers from within an academic department — proved extremely effective in carrying out this project. With the organization dedicated to attacking an extremely concrete goal like autonomous driving, the CMU team was able to keep a clarity of vision throughout the project — while maintaining an organized structure that was less strict than a company but more organized than a traditional academic department. CMU would go on to make very noteworthy contributions in the years since the ALV was wound down. In an IEEE oral history of CMU NavLab program manager Chuck Thorpe, Thorpe describes the steady practical progress of the NavLab team, comparing it to his newborn son, saying:

My son was born in ’86, and we had contests to see when NavLab 1 was moving at a crawling speed, my son was moving at a crawling speed. When NavLab 1 picked up speed, my son was walking and learning to run. NavLab 1 got going a little faster, my son got a tricycle. I thought it was going to be a 16-year contest to see who was going to drive the Pennsylvania Turnpike first, because they were each progressing at about the same speed.

Even without the consistent proof of progress demos such as those that drove the ALV project, steady progress was being made by the interdisciplinary, practically-minded, academic CMU team. But the team’s progress very much received one of those unexpected jolts of progress that groups of researchers diving down riskier rabbit holes with some regularity hope to experience. One of CMU’s grad students working on the project, Dean Pomerleau, published a 1988 paper on a system called ALVINN that would be a major step in showing us the future of the field. Thorpe finished the paragraph he started comparing the NavLab and its steady progress to his son, recounting:

...I thought it was going to be a 16-year contest to see who was going to drive the Pennsylvania Turnpike first, because they were each progressing at about the same speed. Unfortunately, this really smart graduate student named Dean Pomerleau came along and built a neural network technique, which managed to be the most efficient use of the computing researchers that we had, and the most innovative use of neural nets at the time, so in 1990, he was ready to go out and drive the Pennsylvania Turnpike.

Thorpe then recounted how, by 1995, a later version of the NavLab running on a newer vision system, RALPH, took things even further. RALPH’s approach — which Pomerleau also came up with —  combined pieces of ALVINN’s neural net approach with a new model-based approach which was similar in spirit to simpler approaches used in the ALV. Using RALPH’s vision system, the NavLab was taken by Pomerleau and another student from D.C. to San Diego, driving 98% of the way autonomously. Things like “new asphalt in Kansas, at night, with no stripes painted on it,” flummoxed the machine. But it was plain for all to see: the technology was taking massive steps.

Lessons Learned (and Caveats) From All Three Programs

The legacy of the ALV project is more complicated than that of PA and Battle Management, but it may have done more good in shaping the technology base than the other two projects combined. All three projects had their bright spots as well as their weaknesses.

In terms of customer relationships, PA and Battle Management had significantly better relationships with their armed services customers than the ALV had with the Army. Battle Management’s practical success in terms of positive impact on its customer, the Navy, seems to have outdone the practical success of the PA. But it is hard to make that out to be solely the fault of the PA or its management team. After all, in many situations in ARPA-like orgs, you simply get the customer you get.

That being said, Battle Management certainly provides a great exemplar of how to — with the right customer — integrate a test bed as close to the production process as possible to minimize the number of annoyances that can arise when translating technology built in lab environments to real-world environments. This strategy from the Battle Management team also helped mitigate the organizational/logistical hold-ups that often result when looking to implement technology from a remote test bed.

The PA, in spite of its lack of success in pushing a truly useful product out of the program, proved to be a somewhat successful pull mechanism for the speech recognition and natural language understanding technology base. Let’s remember, to many, it was not crystal clear that speech recognition and NLU research was ready for translation into products. The PA application and associated funding on related problems in the technology base helped harness the efforts of DARPA’s performers to work on common problems. This, in the end, helped push certain areas of research into practice. In other cases, the PA highlighted areas that the field needed to work on.

The ALV, in contrast to these two projects, never had the customer relationship of PA and Battle Management. Not even close, really. The Army did not want the ALV; the Air Force wanted a working PA and the Navy was extremely enthusiastic about Battle Management. However, those at DARPA understood that the Army was not excited about getting involved early on with this project. So to judge the program as a near-term failure as an application is somewhat fair, but also largely misses the point. The ALV was something different than the other two applications: an extreme pull on the DARPA technology base. And, in that, the program did a much better job. A large sum of money and energy may have been spent non-optimally pursuing the Martin Marietta work when it was the prime contractor. But DARPA was also relatively quick to redirect the portfolio after several years of watching its broad array of contractors work and assessing their approaches and their results. In the end, CMU proved to be better suited to help push the vision area of the portfolio forward than Martin and the ALV. Fortunately, DARPA was light on its feet in making that change in strategy happen.

It is somewhat hard to tell what the legacy of the DARPA’s vision work during SC would have been had it not been for the exceptional setup, management, and overall culture at CMU with its DARPA contracts. The CMU computer science and engineering teams involved with DARPA embodied some of the most positive aspects of academia and some of the most positive aspects of industry. I will explore the setup of the CMU department and how they fulfilled their DARPA contracts further in a coming piece. What matters, for the scope of this piece, is that the CMU department was exceptionally equipped and internally motivated to go after a problem like practical vision applications, and a large amount of DARPA money found them and enabled them to go on to do exceptional work.

Each of these three projects provides much to learn from for a PM orchestrating and balancing a portfolio of investments that is looking to show some level of practical success while also hoping to maintain a high level of ambition. Hopefully this piece proves of some use to them.

Thanks for reading. Subscribe and stay tuned for coming pieces.

Subscribe now

Share

PA Links:

• Pilot's Associate: a cooperative, knowledge-based system application

  • A report from the Air Force program managers for the PA detailing the performance of the PA’s subsystems in its demos. 

• DARPA’s John Retelle meeting, presentation, and Q&A on the PA to the Transportation Research Board

  • Contains a mention of being able to transition the program to parallel computing machines around 1990. 

• Rizza, the Wright PM on the PA project, speaking at the 1991 Space Operations, Applications, and Research meeting

  • In the document, Rizza explains how the systems and principles that helped make the PA something of a success could be applied to NASA’s work with managing the movement and maintenance of deployed satellites — which was very man-power intensive at the time

• Bachman, Gunning, and Burke’s Integrated Artificial Intelligence Systems

  • Interesting throughout, but its primary relevance in this section of the write-up is the authors’ clear description of why exactly the machine was not pursued further from a component technology capability standpoint.

Battle Management Links:

• 1986 document from John Flynn outlining some aspects of the early demos

  • DARPA Naval Battle Management PM and then long-time BBN employee goes into some depth on the specific hardware, expert systems, user experience, and contractors on the projects.

• Computer Effort Falling Behind article from Michael Schrage written for the Washington Post in 1984

  • Certain pieces of the article did not age particularly well when it comes to a program like Battle Management — a Pentagon insider even referred to these programs as “duds.” Within two years of the article being written, battle management was becoming a clear success. 

  • But other pieces of the article — like the tension between Kahn and Conway — were very fair and got at something that was a real complication for the program for years.

  • The article provides a clear view of how programs like these might be covered in the press if things do not hit the ground running. 

•  Chapters 14 and 15 of A Culture of Innovation — a book on BBN’s 

  • From a high level, the book dives into BBN’s culture, structure, and details on many of its most famous projects

  • Chapters 14 and 15 provide great detail on the specifics of the researchers, contracts, and work details surrounding BBN’s speech work in the early-1970s all the way through the early-1990s.

Autonomous Land Vehicle Links

• Chapter 14 of Van Atta’s DARPA Technical Accomplishments. Volume 2. An Historical Review of Selected DARPA Projects 

  • This chapter outlines much of the lead-up to this generation of DARPA’s vision work.  Particularly useful is its focus on the Image Understanding program which kicked off in the mid-1970s but had its roots in work from the prior decade. 

  • The work dives into the details of specific projects as well as what specific performers worked on what research areas in different eras. 

• CMU Strategic Computing Vision Project Report: 1984 to 1985  

  • This report outlines the first year of CMU’s work on the application and DARPA-relevant vision work as a whole.

  • Particularly goes into more depth on the technical details of various CMU vision research approaches in the early years of SCVision in using technologies like sonar, laser vision, stereo cameras, and video cameras.

• Martin Marietta road-following write-up: The Autonomous Land Vehicle (ALV) Preliminary Road-Following Demonstration 

  • Martin Marietta describes the finer technical details of the ALV during its first demo.

  • The write-up also dives into how Martin Marietta hoped the system would progress from demonstration to demonstration as well as their planning in this nascent stage.

• Dean Pomerleau’s 1988 paper, ALVINN: An Autonomous Land Vehicle in a Neural Network

  • The paper dives into Pomerleau’s initial implementation and results using a 3-layer backpropagation neural network trained on approximately 1200 simulated images of roads to train the model used to steer the car. At the point of the paper being written, the performance of this system seemed roughly equivalent to CMU’s more manual methods — often utilizing expert systems-style reasoning given the sensor data and clever edge-finding heuristics — of training models to steer the machine to that point.

• Chuck Thorpe IEEE oral history

  • Explores many of the finer details of how the projects came together and progressed at CMU. It is great fun to read for anybody interested. 

• Misc.

  • Developing Technologies for Army Autonomous Land Vehicles — 1985 document written by army engineers on the program area

  • Implementation and performance of a complex vision system on a systolic array machine

  • Norberg’s oral history of Alan Newell

General Links:

• Strategic Computing: How DARPA Built the Computer Age

• Strategic Computing’s founding document

Pattern Language Tags:

• Encouraging modularity and open interfaces

• Democratizing an area of research via cost reduction

• Promoting a coordination/service mechanism to reduce material costs and increase research feedback cycles

• Facilitating tool/hardware improvements in a key technology area far from its suspected theoretical frontier

Introduction

In the early 1980s, a large amount of money and energy from DARPA and the DoD was shifting towards computing infrastructure and applications. While many involved with those organizations already had one (hopeful) eye on long-term applications such as AI, it was clear that almost all aspects of DARPA’s AI plans, in some way, leaned on ease of access to computer chips for the research community.

So, as a suite of DARPA and DARPA-adjacent projects sprung up — all in the hopes of systematically building up the greater universe of chip capacity and computing applications — DARPA’s MOSIS project was established to ensure that researchers working on computing problems could have faster and cheaper access to computer chips. In this early era of computing research — when the technical capabilities of chips were orders of magnitude more limited than they are today — it was the norm for many researchers and engineers to design bespoke chips for the particular task at hand. MOSIS would provide the middleman service to all researchers who needed their designs fabricated into chips more cheaply, quickly, and discretely than would otherwise be available.

Context

In the late-1970s, printing chips was extremely expensive. For independent researchers paying a separate facility to print their design, the price of a single wafer could have a price tag in the tens of thousands. Since hardware was often specially developed for individual applications or experiments, single research projects often required printing individually designed chips. The cost was a big problem and the process itself was a major annoyance to researchers. Manufacturers who printed chips often had specialized syntax in which they encoded the digital designs into their machines. Going back and forth with a manufacturer was a major tax on researchers’ time and timelines. Also, researchers handing all of their designs to manufacturers was a point of concern. If their project was a success and they attempted to commercialize it, one of the researcher’s now-competitors would have their digital designs in hand. The system of fabricating chips, at the time, was just not set up for convenient access by the community of individual researchers. That is why MOSIS was created.

MOSIS could make things cheaper and easier for these researchers. While individual silicon wafers and runs with manufacturers were expensive, the average chip that needed to be fabricated for a standard research project was quite small — about 1/20th the size of a wafer of silicon. Since the field was new and there was not exactly a mass market of hobbyists who would pay for many chips to be run in parallel, a market/middleman service had not been made for this particular service. Without a solution, new research might be far more expensive, time-consuming, and sparse than it otherwise could be.

In the backdrop of all of this was the belief – encompassed by a 1976 RAND report by Carver Mead, Ivan Sutherland, and Thomas Everhart – that the integrated circuit revolution had only half run its course. Another four orders of magnitude improvement in chip manufacture seemed theoretically possible. Despite this potential, the authors believed that many industry players largely seemed to persist in incremental development. They believed DARPA should take steps to get involved and help speed up ambitious development and research work in the field.

Beginnings

The demand and usefulness of the eventual MOSIS service were first proven through Carver Mead’s chip design course at Caltech — leveraging a more computational, less manual pipeline for designing chips he had developed — and subsequently through courses modeled after Mead’s course at MIT and elsewhere.

Mead wanted to teach his students how to design VLSI (Very Large-Scale Integration) chips. Since there was no single industry standard to teach to because each manufacturer had its own protocols, Mead developed a general set of protocols out of the disparate standards that would allow students to learn the principles in a specific way — but in such a way that the students would still be useful to any arbitrary company they may go work for after graduation. Above all, Mead wanted students to learn to design chips by actually designing their own chips.

To make this goal feasible, printing the chips would have to be cheaper. So Mead figured out a way to put many chip designs onto the same wafer to reduce printing costs. Mead turned to Lynn Conway — Head of LSI System Design at Xerox PARC — and Xerox PARC fulfilled the role of silicon broker in this first iteration of the course. And it was a success. Mead and Conway succeeded in the course's goal of trying to ‘totally isolate the designer from all the trivia that fabrication requires’ to enable students to learn chip design faster.

These learnings and design standards were eventually codified into what became the bible of VLSI design, a book co-authored by Mead and Conway titled Introduction to VLSI Systems. This book and the course's approach were how the course’s teachings began to reach more schools and researchers. The approach demystified chip design and enabled researchers to learn chip design in a course setting — rather than having to consult for a chip manufacturer to learn all the little tips and tricks of the trade as was common at the time. You didn’t need to know everything about chip manufacturing to design workable chips anymore. In 1978, Conway went to MIT to teach MIT’s iteration of the course for the first time. By 1980, things were progressing. In the spring alone, “250 designers from 15 universities and R&D organizations submitted 171 projects for implementation.” These specific chips were fabricated by HP.

With the potential of the service proven at several schools and across multiple manufacturers, Mead, Conway, and others began pushing more strongly for a DARPA project to take over and scale this work. At the time, most DARPA PMs made efforts to be in close contact with as many potential PIs as possible in relevant research areas. In this particular case, Conway — while at Xerox PARC — began pushing Kahn — then head of IPTO — to push this work forward. Although, it is very likely that if Conway had not made this push, Mead could also have reached out to one of the IPTO PMs who frequently worked with CalTech.

The Information Sciences Institute (ISI) — which had an affiliation with USC but was independent of the university in many ways — was granted the role of DARPA “silicon broker.” In the 1980s, they became one of IPTO’s go-to performers to carry out a variety of services like this which required research familiarity as well as the organizational skills of a company — such as orchestrating the Machine Acquisition Program in the adjoining piece. ISI was started by a group of researchers who felt their former employer, RAND, was too geared towards paper studies and did little in the way of directing research and work on technology itself. ISI was an unfunded research arm of USC — paying its own way through (mostly DARPA) grants and contracts. Their regular work included performing services for DARPA, conducting applied research, creating technology transfer programs, and more. ISI had often conducted work somewhere in between applied research and DARPA program administration before, and, thus, was very well-suited to this new role.

Operations

Once the program was established, the user experience was quite straightforward. Those who had access to MOSIS — largely NSF researchers, DARPA grantees, and other individuals whose funds were largely from the government — communicated with the MOSIS central office via the ARPANET. MOSIS put together a user manual explaining how electronic mail via the ARPANET worked — those were the early days of the internet — and how to email designs and queries to the mail system in a structured way so a program could sort them.

Researchers sent their designs — in CIF files — to MOSIS. MOSIS, initially using a system designed by Ron Ayres of ISI, took it from there. The system:

• Checked the CIF file to ensure it conformed to the basic design standards outlined by MOSIS in the user manual.

• Packed sets of projects onto a smaller set of dies.

• Translated each die into MEBES format.

• Made bonding diagrams for manufacturers.

• Most importantly, produced tapes that the foundries used to make masks.

Danny Cohen and George Lewecki of ISI, in a 1981 talk on the early MOSIS system, described how the process continued from there, saying, “The next step in the process is mask fabrication. Mask houses expect two types of things from us: tapes with MEBES files and job decks. MEBES files contained the information that the mask houses used to make bitmaps (which are made into masks). A job deck…contains the specifications for each MEBES file — parity, record size, etc.”

The talk continued, “Fabrication itself is very simple because somebody else does it. Once the masks are made, all we have to do is drive three, four, maybe ten miles in Silicon Valley with the masks to a wafer fabricator. (It is wise to drive slowly to make sure the masks don’t break.) After that, if we’re lucky — and typically we are — we end up with a couple of wafers.” MOSIS completed its job by dicing, bonding, packaging, and shipping chips.

The MOSIS staff also took measures to ensure that they were as ‘lucky’ as possible. And when they weren’t lucky, they made arrangements that ensured as few defective chips as possible were handed to users. They accomplished this in several ways. MOSIS carried out basic probing tests on transistors, inverters, etc. ISI kept track of defect rates for any given type of job from all of their fabricators to 1) ensure that they continued to work with the best fabricators possible and 2) that they ordered enough duplicates of any given part for a researcher to ensure a 90% probability that a given print would yield a working chip for the researchers — helping keep research projects on schedule.

ISI also carried out different forms of applied research to ensure progress in a variety of operationally important areas. Some of these areas included exploring the feasibility of expanding the service into new chip types, working to incorporate new coding language into ISI’s workflow, using new varieties of machines in their work; etc. In general, it was ISI’s job to do any research work required to make the service work as efficiently as possible and take advantage of the rapid technological improvements happening in the field of computing. While the ISI team did not set out to do fundamental research in its own right, the ISI team’s skills as researchers may still have been important to the project’s success. In the hunt to make chips smaller, new technical problems constantly arose in fabrication processes. Courses of research were often required to work through these problems.

One ISI research project involved finding a way to translate the different design languages used by researchers into those used by manufacturers. Another case — discussed in the Cohen and Lewecki talk at CalTech — outlined how the ISI team worked around a difficulty in testing new softwares to see which software was best:

The problem arises in comparing two masks, one produced by the old [software] and the other produced by the new system. Are the patterns really the same? A microscope is supposed to help, but it can’t do a good job. We tried many ways and finally worked out a very strange technique. Suppose you want to compare mask A and mask B. What we did was to overprint A and B bar [the reverse of B], and A bar and B. In this way we discovered all the changes. We did all the printing on one plate so we wouldn’t have to use a special microscope.

One final — often forgotten — aspect of MOSIS’ success was that it provided a layer of security and confidentiality to researchers and manufacturers. Researchers came to trust ISI agents and their system for communicating design instructions to manufacturers. As Roland and Shiman — who wrote an in-depth history on DARPA’s 1980s computing efforts — noted, “Only ISI had the electronic version of their designs; versions on masks or even on the chip itself were prohibitively difficult to decode.” Manufacturers also maintained open lines of communication with ISI as they became convinced that ISI respected their proprietary information. Many recognize that ISI’s service in this project was ill-suited to a traditional university department. However, as this remark from Roland and Shiman also indicates, this particular piece of ISI’s service may have been hard to replicate for certain companies in DARPA’s performer pool as well. 

Results

The MOSIS program is widely considered a success.

The number of wafers MOSIS printed rose steadily throughout the 1980s as demand increased and the field developed. The number of wafers printed grew from 258 in 1981 to 1,790 by 1985. The cost of the average chip for a researcher often came in around 5%-10% of what it would have cost if a researcher had to pay for the entire wafer. In 1988, George Lewecki noted that a single 2µ chip in a MOSIS run could cost as little as $258 (~$650 today). For the most part, DoD and DARPA contractors got the service for free, NSF grantees paid for the service out of their research grants (and eventually received the service free), and any commercial actors had to pay full price.

Throughout the 1980s, the average turnaround for chip times from MOSIS gradually decreased from around 10 weeks to about 8 weeks. The ability to print smaller and smaller features for a given cost and time frame continuously improved by large margins as the field evolved. MOSIS’ somewhat stagnant turnaround time was a conscious decision because allowing more projects to fill the queue helped ISI take further steps to reduce costs.

MOSIS was considered a quite efficient operation. A 12-month period from 1987-1988 saw $9 million flow to the MOSIS program. George Lewecki noted that 74.3% of MOSIS expenses went to fabrication costs, 13.1% to salaries, and 11.4% to ISI overhead.

Lessons Learned (and Caveats)

Some in industry were annoyed that MOSIS might support future competitors of theirs. Private researchers could get free access to resources that TI, Intel, and IBM had to pay for. These researchers were not just looking to expand the knowledge frontier, but spin out companies from their university research if things worked out. Conversely, some within the government were annoyed that the MOSIS service — which was primarily targeted at university researchers — could be used by commercial actors at cost. Essentially, they felt the years of infrastructure and know-how MOSIS built up were not being priced in, and the service was essentially providing an industry subsidy.

To this writer, those criticisms are not to be taken too seriously. But a third area of criticism is worth examining. Some didn’t like that MOSIS seemed to be a never-ending program. MOSIS was providing a service that required some research and also helped the research community, but once it settled into its patterns it was not considered high-risk — even if it was seen to be high-payoff. DARPA tended to focus on higher-risk R&D projects. However, MOSIS also did not seem like a sure-fire commercial bet to spin off into its own firm. While the service was successful, it was not considered a good bet for venture capital by many familiar with the program. It was more of a service and contemporary VCs preferred products. Additionally, 98% of the service's customers — in this young and research-focused area — were from DARPA, NSF, or DoD funding sources. Competing firms had little need for MOSIS' main service of subdividing wafers. 

If left on its own in the private sector, one could not be sure MOSIS would survive. For example, it could end up in some sort of ruinous cost-cutting competition with a competing manufacturer and end up out of business. But, still, the service was too good to terminate because it offered a cost-effective way to support a vital area of research and teaching. So, instead, MOSIS was moved under the umbrella of more traditional defense funding to continue operation indefinitely. Funding for the program by the DoD and NSF ended in 1998. However, the program does still exist today under the ISI umbrella at USC. Its continued existence — in an era of very different technological conditions than those that created the original need for the program — may be considered, by some, a data point in favor of “sunset clauses” for certain projects. But I will not address that point further as it is somewhat out of scope of this series.

All in all, the program is considered a success. The MOSIS program succeeded in lowering the barriers to doing ambitious research in a broad research area whose progress was all-important to DARPA and the DoD’s computing goals; microelectronics research could be done by individual researchers for a fraction of the cost it would have previously cost. Additionally, as Kuan and West write, MOSIS “modularized [decomposing a problem into separate modules] the semiconductor ecosystem” by enabling semiconductor design to be done productively on its own. With MOSIS, DARPA was able to help alter the structure of a field in a way that industry incumbents may not have done on their own — but was likely better for the progress of the field.

Previously, design was generally done by those with the skills, experience, and machines to manufacture semiconductors — large industry players. The MOSIS service was able to help unbundle these two separate tasks and pave the way for the semiconductor industry to subdivide into design-only  “fabless” firms and production-only foundries. With so much of this fabless design research being done by the academic community — rather than existing private sector incumbents — the field itself was able to be built out in the open, with researchers making rapid progress by leveraging one another’s discoveries.

Some might believe that rising tides lift all boats and, thus, any infrastructure program in an area that turns out to be explosively productive — like mid-1980s computing hardware — will look good in retrospect. However, one should keep in mind that not all of DARPA’s Strategic Computing infrastructure projects with similar goals from this same era are considered a success in a similar vein. For more on that point, see the accompanying piece on DARPA’s Strategic Computing Machine Acquisition Program. Some feel there could have been a world where the MOSIS investment’s heavy reliance on Mead-Conway standards artificially incentivized researchers to develop λ proportions at the expense of others. The accompanying piece describes a program that made a somewhat analogous error. However, I believe that MOSIS’ placement as more of a broker than a capital purchaser provided it a greater level of flexibility to work around this category of issue that ailed the Machine Acquisition Program.

Check out the adjoining piece here

• Strategic Computing's Machine Acquisition Program

Thanks for reading! Subscribe to continue following along with the series!

Subscribe now

Share

Specific Links:

• MOSIS User Manual .txt file

  • Particularly useful if the reader wants to understand how MOSIS instructed self-onboarding onto a complicated technical service.

• Danny Cohen and George Lewecki giving a talk at Caltech on the project in 1981

  • Particularly useful if the reader wants to understand: 

    • How MOSIS ran into problems and solved them using research or other qc measures

    • How the MOSIS service operated in more detail

• Chapter 4 of Strategic Computing: How DARPA Built the Computer Age

  • Particularly useful if the reader wants to understand how MOSIS progressed from just an idea, through its early years, and into a mature program. 

  • Also covers the politics involved in maintaining the program. 

  • Additionally, any details about specific chip design measurements and program decisions are covered at moderate length in this chapter.

• Kuan and West’s Interfaces, modularity and ecosystem emergence: How DARPA modularized the semiconductor ecosystem

  • Particularly useful if the reader would like to understand (in more detail) how MOSIS was able to unbundle design from manufacturing capacity and the effects of this on the commercial and research actors in the space.

• Misc.

  • 1983 ISI report on the present and future of MOSIS

  • MOSIS: A Gateway to Silicon

  • MOSIS 1983 Annual Technical Report

  • Ronald Ayres narrative history of MOSIS software

General Links:

• Strategic Computing: How DARPA Built the Computer Age 

• Introduction to VLSI Systems

• Keith Uncapher’s Oral History

  • Dives into the founding of ISI as well as the significance of the MOSIS system.

• Robert Kahn’s 1989 Oral History

  • Briefly dives into the overarching need for the program and industry’s trepidation to provide fab capacity to the program at the start.