Subscribe now

Share

DARPA doesn’t exist in a vacuum. It is enmeshed in both the military ecosystem and the D.C. ecosystem. DARPA also has no labs; it is dependent on the community of performers (contractors) available to it — academic labs, companies, and others. Major changes in the politics, economics, incentives, or preferences of any of the above can substantially change what a DARPA office is at any given time.

It is a testament to DARPA’s resilience that it has been able to withstand so many changes throughout the decades, continuing to serve its mission in a less bureaucratic and more entrepreneurial fashion than the vast majority of the government R&D ecosystem. However, it is still essential that those seeking to understand historic DARPA successes understand the “regime” — a DARPA office’s political environment, performer ecosystem, etc. at a given moment — in which particular projects existed.

Through my research on DARPA history thus far, I have come across many small changes that have impacted what DARPA is at a given moment. But three factors have jumped off the page and proven salient enough that I will not sit down to write up a DARPA project from history without understanding them. They are:

1. The level of organizational oversight in a given period

2. The extent to which project “visions” came from office directors vs. PMs

3. The timeline in which projects were expected to pay off and how exactly a given office interpreted DARPA’s mandate that projects serve the military in some way

Today’s piece will not cover any specific project or contractor from DARPA history. Instead, I will detail how changes in these three factors alter how DARPA operates. I will provide examples from oral histories of long-time DARPA employees and performers as well as DARPA historians describing the specific effects of changes in these factors. Many of the specific examples will draw on DARPA computing history and the Information Processing Technology Office (IPTO) —responsible for many or ARPA’s now-famous early computing projects.

I’ll begin by exploring what might be the most substantial shift DARPA has ever experienced: the increase in bureaucracy and procurement rules following the Vietnam/Watergate era. These shifts fundamentally changed how DARPA PMs were allowed to operate.

1. From a Small World to a Bureaucratic One

The post-Vietnam and Watergate shifts in Washington D.C. mark a clear “before and after” in DARPA history. Regulatory crackdowns and increased oversight became typical in this period. Much of this oversight impacted the entire government or the entire DoD at once. Regardless of whether or not DARPA had done anything wrong in the public’s eye, the government had. The sweeping reforms left DARPA a very different agency — from the perspective of both its PMs and its performers. While DARPA remained a more fast-acting and less bureaucratic place than many other agencies in D.C., it became noticeably more bureaucratic than it once was.

Procedural Changes (e.g. 1984 Competition in Contracting Act)

This general change in political attitudes had begun to steadily impact DARPA operations throughout the 1970s, but among the years of steady changes there were also abrupt shifts. One example was the implementation of the 1984 Competition in Contracting Act. To DARPA PMs and longtime performers who served both before and after the changes, this shift is often seen as demarcating where an old DARPA stopped and a new DARPA began. While it is reasonable for one to believe that the ramping up of oversight in how government funds were spent was, in principle, good. It’s more difficult to argue that the changes did not substantially impact the speed and freedom with which DARPA PMs could source, iterate upon, and fund ideas.

Even the procedural changes that seemed, on the surface, very reasonable had a noticeable impact on the speed and fluidity of DARPA’s operations. One example of this was the new requirement that agencies publish a notice for all procurements above $10,000 in the Commerce Business Daily. Once published, PMs would only have to wait thirty days for bids to be solicited and could then begin negotiations. Roland and Shiman — who wrote the book on DARPA’s Strategic Computing initiative and extensively interviewed many of the major staff and performers of 1980s DARPA IPTO — wrote the following on the impacts of this change:

Paired with the Competition in Contracting Act, this legislation transformed the old IPTO-ARPA style of funding research.

In the salad days of IPTO during the 1960s and 1970s, program managers solicited proposals in the research community, often by direct contact with the researchers. An office director such as J.C.R. Licklider, or a program manager such as Larry Roberts, for example, might call up a researcher to ask if he or she would be interested in doing a project for ARPA. If there was a meeting of the minds, the researcher would submit a proposal that was essentially approved before it came through the door. Program managers achieved results by knowing what was going on in the research community and who could deliver. They also accepted unsolicited proposals coming over the transom if they were good and suited ARPA’s purposes. And they put some contracts up for competition, as they did the ARPAnet contract that BBN won with Robert Kahn’s help. The style, however, was still less formal than at agencies such as NASA or even elsewhere within the DoD.

With the implementation of the Competition in Contracting Act, far more DARPA contracts were now formally competed. As a result, a wave of new federal procurement laws now applied to these contracts. This particularly impacted an organization like DARPA because DARPA staff not only fund work on the technological frontier, but come up with plans to shape it. DARPA PMs had previously relied on their ability to work out what a contract might look like with the performer community before a formal contract was ever written. Now, all the performers were legally competitors; how DARPA PMs could interact with them once a bid was out had changed. Roland and Shiman describe how this change in the legal relationship between PMs and the performer community impacted project creation, writing:

Particularly crippling to DARPA was the requirement in the new law limiting contact between procurement officers and potential contractors. Such contact was the taproot of the PM’s success. It was by staying in touch with the community, by letting the community know what was needed at DARPA, by finding out what the community was capable of doing that the PM was able to map out a reasonable research agenda in his field and sign up the best workers to make it happen. In the new dispensation, such networking could be viewed as collusion. PMs in the future would simply have to be much more careful, much more circumspect, much more correct when dealing with researchers who were potential contractors. This did not make their jobs impossible, but it did make them harder and less enjoyable.

“Harder and less enjoyable” is a succinct way to describe the differences in the lives of a DARPA PM of the 1980s vs. a DARPA PM of the late-1960s. If it were truly critical, there were procurement levers DARPA directors, office directors, and PMs could pull to make DARPA work the way they needed it to work. As one example, directors could use an OTA-like authority to get around particular pieces of red tape if it was truly important. But directors were also aware that this should only be done sparingly. Also, if a technicality got in the way of common-sense project management — like the above regulation which negatively impacted PMs’ ability to brainstorm with performers when putting together contracts — entrepreneurial individuals at DARPA often found ways to at least approximate their old common-sense workflows in legal ways. In fact, this is how DARPA’s Broad Agency Announcements (BAAs) came to exist. To get around the aforementioned issue, Stephen Squires came up with a technique for soliciting proposals — once described as “a cross between an RFP and a fishing expedition” — which took hold at DARPA.1 It eventually evolved into what is now the BAA. While the BAA was a wonderful organizational innovation, it was only necessary because of the aforementioned changes in procedure.

There was now an added step to the game of being a DARPA PM. PMs and office directors could still be proactive and ambitious, but they often had to devise strategies to make their projects work within the confines of the new legal procedures and red tape. By the late 1980s, many longtime DARPA affiliates felt the PMs could no longer do precisely what they wanted exactly when they wanted to do it. These shifts in procurement laws and contract management contributed to DARPA becoming a place in which funding could easily be given to promising ideas in days or weeks, to one where months was increasingly becoming the norm.

Of course, there were also games to be played in the prior DARPA regime. In the case of IPTO, for example, many researchers at computer science departments that did not receive IPTO’s Centers of Excellence grants — which Licklider’s IPTO office had used to foster top-tier computing departments when the field was in its nascent stages — complained that they were unfairly overlooked. To overcome being outside of the network was also a game, to be sure. A researcher might need to go further out of their way to put themselves on a PM’s radar. But the new procedures ushered in a game that was quite different in nature. The following subsections detail some of the performers’ and PMs’ perspectives on this new game and how it had to be played.

The Performer’s Perspective On These Changes

In their oral histories, often recorded once they had retired, long-time DARPA performers often listed this change in procurement bureaucracy as the single factor that most threatened DARPA’s future effectiveness.

In a series of interviews with those who played major roles in BBN’s work on the famous ARPAnet contract, two separate BBNers used the free air the interviewer provided them to put on the record how bad they felt these changes were for DARPA. In his 1990 oral history interview, Frank Heart, BBN’s project leader for the ARPAnet contract, described what he believed to be the increasing silliness of the government’s approach to technology contracting:

[Interviewer]: Okay, that covers the questions I had, unless you have any general comments you would like to add to the record.

HEART: As I say, the only thing I would emphasize is that if people try to look at all the projects that this country does in the computer world, and all the failures there have been, all the large military computer systems that have fallen on their face, or had a factor of ten overruns in time or money...If you look at all those things, and then you ask, "Why do these few make it and be successful, and why do all those not be successful?" That's an important thing, if one could somehow get that lesson across to Congress and others, it would be kind of nice. It's very hard apparently because they keep doing it the wrong way, mostly. What happens is that people who know nothing about the technology write long lists of requirements. Then they go out to industry and ask industry to bid on these silly requirements, and they get back silly bids. Then they pick the low bidder of the silly bids, which often is the one who understands the technology least; and then ten years later they have a disaster. Instead of people who understand the technology very, very well in the government who are able to drive it to the groups that are the right groups, even if it isn't the low-price bid, and who take close control of it, who don't have an enormous list of specs, and then other people who never wrote the specs, but who have a participatory prototyping kind of relationship with the thing where it grows. If one could get across that lesson, it would be kind of nice. But I don't hold much hope for that.

[Interviewer]: Can you give me any examples of what you consider big failures, using this kind of method?

HEART: Well, gee, there are probably... No, I probably can't do that, just because I wouldn't want to be quoted. But, I mean, they're legion. The number of defense systems where they've tried to build computer systems, and they've been horrible, there are probably lists twenty or so long. I probably wouldn't want to be quoted on naming them, but it's many, many, many subsystems that have been ten years late at ten times the cost — things like that. The ARPANET went from a contract award to an installed operating system in nine months. It went from a contract award to equipment being delivered, on site and running, in nine months.

Heart also believed that working with DARPA, in 1990, was still quite a positive experience and not nearly the headache it had become to work with other pieces of the government that supported research. But the following quote from Alexander McKenzie’s 1990 oral history — Mackenzie was BBN’s so-called “ARPAnet generalist” — makes it clear that these procurement changes had pronounced negative effects on DARPA performers, compared to how things used to be. McKenzie’s interview reads:

[Interviewer]: We have covered what I wanted to ask you about. I want to open it up and ask if there's anything you want to add, any general comments you would like to make about the ARPANET, or your involvement in it, or ARPA in general.

MCKENZIE: I think that the example of the way ARPA worked in those days, in the 1970s — not only in communication but in a lot of other programs — was pretty effective in terms of cost and return on investment. That the model of some really bright people in the ARPA office, with some particular goal in mind, something they thought would help the country, or the DoD, being given the authority to go out and find the smartest people they could find who were interested in working on that problem, and then giving them relatively free rein to do some research or some development, produced pretty remarkable results. And I know that there has been fraud and abuse in government and in contracting and so forth, but it seems to me that the kind of rules and regulations that there are now, that are attempting to prevent that, really make it very difficult for the government to get the same kind of power out of its research dollars these days as it was able to then. I know it's hard to find a balance between accountability and free rein, and these days the government approach seems to be more on the side of accountability and less on the side of free rein. But I think that a lot is being lost.

Mckenzie continues by sharing a quite specific example that he felt characterized the kind of thing DARPA eventually began to worry about that it hadn’t in the 1960s:

I remember a discussion that I had with Jon Postel back in, I don't know, the early 1970s. We were talking about travel to a Network Working Group [which helped come up with the early internet protocols] meeting. And he was saying that he really would like it — maybe it was a subgroup meeting — if it would be some place rather than some other place, because the government accountants were getting university accountants to not want them to spend so much on travel. And his comment was, "I know that in my contract there's only a fixed amount of money. I want to use that money the most effective way possible. I don't know why they think that, when my salary is paid out of this, that I'm going to waste money on unnecessary travel. I want the money for my salary; I want to keep having a job, but sometimes the travel is necessary. They don't seem to understand that that's the case. They think it's just a boondoggle." I think there's a lot to that attitude. That, yes, we spent a lot of time in Network Working Group meetings hammering out ideas, but something really good came out of it.

Mckenzie closes by explaining how, in his eyes, the previous DARPA regime was ideally set up to keep costs down in its own way, while staying ruthlessly focused on outcomes and not process:

BBN wasn't required to file monthly fund expenditures reports and justify every decision on the basis of cost effectiveness. We could pick the Honeywell 516 because it was OK — it was an OK processor for this. It would do the job, and it wasn't, maybe, the most cost effective and we didn't have a subcontract bidding plan in effect to give everybody the opportunity to bid, and thereby put another three months of delay into the schedule - we could just decide. I think that was pretty valuable. If you're talking about multi-million dollar projects, it's probably more important to pay attention to those things than it is in tens-of-thousand or hundreds-of-thousand dollar projects. But, even there, there's probably things where better results would come to the government if they could find a way to shift back a little bit more towards accountability for the results, rather than accountability for the pennies. I think that DARPA in the 1970s did a really good job for the country in that way. It was a joy to be associated with the ARPANET project. It was fun. It was challenging. And I think it was good for the country. It's not so easy to find that mix now, and I think regulation is a big part of it.2

The negative opinions on these bureaucratic changes at DARPA were not limited to its performers.

The PM’s Perspective

As these procedural changes began to take effect, the life of a PM became “harder and less enjoyable” in many ways. Opinions varied on how annoying the changes were. Many PMs thought the changes were horrendous for the pace and cost-efficiency of projects. Some, such as IPTO’s Ronald Ohlander, expressed less polarizingly negative opinions. Ohlander — whose budget more than tripled from $14 million to $55 million as DARPA’s computing applications budget grew — felt this increase in resources granted him many more opportunities than the increased paperwork and bureaucracy took away. But even after taking moderate views like Ohlander’s into account, it is clear that the DARPA of the mid-1980s operated with the increased computing budget much differently than the DARPA of 1970 would have.

PM Paul Losleben’s thoughts on the new mix of resources and bureaucracy were clear:

I…had twice the budget that I had had before…and I was accomplishing less because I had lost the freedom to build the program, to interact, to define direction, to work with the community.3

Ohlander, on the other hand, merely complained that he had to hire more staff than he otherwise would have to keep up with his budget increase. But, with that being said, some of the difficulties he experienced managing the ill-fated Autonomous Land Vehicle (ALV) project may have been mitigated if he could operate more like a ~1970 ARPA PM. The original ALV project had to be scrapped largely because of the mismanaged expectations and misaligned incentives of both Martin Marietta and the university vision researchers. As I covered in a prior FreakTakes piece, Martin Marietta turned out to be a prime systems integration contractor with little interest in implementing cutting-edge component technology. Martin pressed DARPA to specifically define intermediate benchmarks for the technology. Martin would then find ways to hit these benchmarks using minimal cutting-edge technology, not trying to satisfy the spirit of the contract. Meanwhile, many of the university-based computer vision researchers cared more about developing algorithms that were interesting in a basic research sense than they did about developing algorithms to make the vehicle work. Their algorithms often did not perform when plugged directly into machines like the NavLab at CMU.

Whether or not these failures would have happened absent the new procedures, at the minimum the new regulations exacerbated these issues. Roland and Shiman describe how these procurement changes negatively impacted Ohlander’s ALV project:

The new methods of procurement…made Ohlander’s task doubly difficult. In a previous era, he would have held informal discussions with the researchers at an early stage in the planning process, so that contractors understood what was expected of them and what they would contribute. Such a casual approach was no longer possible. The potential contractors were all competitors now. Ohlander could gather the researchers together to help write the [project] plan, but he could not discuss specifics with them about who would do what. The researchers themselves would have been very reluctant to talk, especially with their potential competitors. Ohlander could not be certain that the successful competitors would work well together. He had never worked at all with some of them, especially those in industry, such as GE and ADS. All he could do was to specify in their contracts that they were required to participate in open discussions and exchange information freely with the other contractors. They [DARPA] would have to trust in his [Ohlander’s] ability, as the source of funds, to twist arms if necessary. Running SCVision in two phases made this easier, as it allowed DARPA to cut off recalcitrant or underperforming contractors after only two years.

The multi-phase structure of the project proved to be essential. At the end of the first phase, DARPA would have to fire the prime contractor, Martin Marietta. While many of the academics continued to receive their basic research funding, they gradually became only indirect contributors to DARPA’s main autonomous vehicle project. Their research was often not as applied as DARPA needed it to be. The spirit of the ALV project primarily lived on in the autonomous vehicle teams Chuck Thorpe and others were building at the CMU Robotics Institute. In the CMU Robotics Institute, Raj Reddy had built up a research operation that valued what DARPA wanted it to value: cutting-edge research alongside practical systems engineering work. In CMU, DARPA’s autonomous vehicle staff eventually found a one-stop shop that could manage the practical aspects of the project and many of the research portions. That was great. But more time and money likely had to be spent on the wrong performers than may have been necessary under DARPA’s old procedures.

In 1984, as the Competition in Contracting Act was being implemented, a Science reporter named Mitchell Waldrop — now of Dream Machine fame — wrote an article that characterized the feeling of DARPA PMs and their performers as “confusion and heated tempers.” One side was frustrated as they figured out how to write bids for government contracts while the other tried to figure out how to evaluate them in a legal and auditable fashion. ARPA’s office staff and performers had begun to spend far more time checking compliance boxes than they ever had before. Bureaucratic obstacles stood between the PMs and their beloved performer communities. Things had changed.

To learn from any project from ARPA history, it is crucial to understand whether the project took place before or after these procedural shift and how that affected a PM’s project management.

2. Office Director’s Visions vs. PM’s Visions

DARPA has become quite famous for the freedom of its PMs. Many advocates of the ARPA model believe a PM’s freedom to pursue their own unique vision is an emphasis that DARPA has always had. However, in DARPA’s early decades, in particular, the freedom to decide which project areas to pursue often fell to its office directors, not individual PMs. But even in cases when specific goals were dictated to PMs by office directors, the PMs still generally had the freedom to decide exactly how to go after those goals.

There are pros and cons to leaving the job of “visionary” to office directors vs. the distributed cohort of PMs. The former may provide an office with fewer unique opinions but ensure that the whole office is working in a coordinated fashion towards a few primary goals. The latter ensures that a given cohort of PMs is free to pursue as many unique ideas as possible. Many offices from DARPA history tend to fall somewhere in between the two extreme ends of the spectrum.

A prominent example of a relatively director-driven office was IPTO in its early decades. Early IPTO office directors were picked because they could both provide a strong vision for the office as well as effective leadership of the PMs. Roland and Shiman describe the different directions in which early IPTO directors steered the office, writing:

J. C. R. Licklider revolutionized computing by aggressively supporting his twin hobby horses of time-sharing and man-machine symbiosis, and by proselytizing for his vision of an “intergalactic computer network.” He recruited Ivan Sutherland to do the same with graphics. Strong successors such as Taylor and Roberts ensured that artificial intelligence (AI) received the support it needed well into the 1970s. The payoff from AI was slower to come than in time-sharing, graphics, and networking, but IPTO kept faith with the field nonetheless.

These IPTO directors often fostered beneath them a few academically-decorated PMs with visions of their own which they were green-lit to pursue. But, oftentimes, there were even more PMs stewarding the visions of others than were encouraged/expected to have visions of their own.

Three examples of projects that were dreamt up by someone other than the PMs who ran them were the Pilot’s Associate, Battle Management, and Autonomous Land Vehicle projects — all covered in a prior FreakTakes piece. Several individuals across IPTO, in consultation with the armed forces, had a hand in dreaming up these projects. The most important of these individuals was probably Clinton Kelly. Despite this, Kelly became the PM of none of them. He helped guide them, in a way, but was not the official PM. Despite the fact PMs like John Retelle (Pilot’s Associate), John Flynn (ship-level Battle Management), Al Brandensteein (fleet-level Battle Management), and William Isler (ALV) did not come up with the ideas themselves, they still had relatively long leashes to execute their projects as they saw fit. Despite not having the freedom to come up with projects from scratch, their freedom was more similar to that of a SEAL team commander who might not get to choose which objective to take, but often has the freedom to decide how to take the objective. In fact, PMs from the armed forces found themselves in the position of stewarding others’ visions relatively frequently.

A major upside of office directors coming up with a vision and several PMs being expected to execute that one vision is that PMs could be assigned to projects that they might not 100% believe in. This can be useful. There exist cases that might draw appropriate skepticism from all individual PMs, but which it is optimal for the office portfolio to pursue nonetheless. I’ll provide an example of a program which was roughly this shape — one in which the right course of action required PMs to pursue all possibilities, however unlikely.

In the early 1980s, IPTO director and chief visionary Robert Kahn thought that computers needed about a three order of magnitude increase in performance before they could truly approach AI-type applications. There were three paths that he felt could make this happen. To be thorough, he felt it made sense to fund all of them — even though some seemed far more likely to work than others. The goal was important enough that he felt no stone should be left unturned. Option 1 was to improve chip performance, funded through the VLSI program. Option 2 was to speculatively design new software approaches that could be run on later generations of machines more suited to the approaches than present machines. Option 3 was considered the most promising option. Option 3 was to design and build parallel architectures. With the office largely following the vision of one or a few people, and not every single PM, Kahn was easily able to ensure each of these approaches was pursued thoroughly — even if options like speculative software design felt risky and unlikely to pay off.4

In IPTOs early decades, such as in the 1980s, it was also not uncommon for PMs to be called in to manage projects that were the vision of another PM. PM Ronald Ohlander’s portfolio of AI projects was one example of this strategy. In the following excerpt, Roland and Shiman describe what was expected of the non-visionary PMs brought in to help Ohlander manage his then-swelling AI portfolio:

All three men were typical of the active-duty military officers who did one or more tours at DARPA on assignment from their services. They served beside civilian program managers who came to DARPA from industry or academia…The civilians usually came with more powerful credentials in their technical fields; this often meant that they had technical agendas of their own and attempted to shape the field while at DARPA. The military officers came with a better understanding of the clientele for DARPA’s products…They could translate service needs for the computer science community and translate technical capabilities to serving officers. Sears played the latter role. He took over programs for which he had little formal training, but he educated himself on the job and relied on his common sense and management experience to keep the programs on track. Most importantly, he had to recognize good excursions from the plan, to know when a project was making better-than-expected progress and needed more resources to exploit the opportunity. Just as importantly, he had to know when a project was failing and had to be invigorated, redirected, or abandoned.

While many seem to intuitively understand the pros of PMs’ freedom to have visions of their own throughout DARPA history, it’s important to understand that this is not something that has been true of every office throughout DARPA history. For example, DARPA’s golden goose, early IPTO, did not always work this way. The individual freedom of PMs has huge upsides, but so does the entirety of an office’s PMs working in a coordinated fashion to accomplish a few goals. There is room for both approaches in any ARPA-like organization; the state of a field, an office’s goals, the personnel available, and the set of institutional circumstances an office finds itself in will determine which approach is optimal.

3. Payoff Timelines and DARPA’s Military-Focused Mandate

The final section details a variable that heavily dictates which projects are in scope for a PM and how a project is pursued: a DARPA office's attitudes towards payoff timelines and dual-use technology. This is often dictated by things like the preferences of DARPA directors and the political moment an office finds itself in.

Changes in acceptable payoff timelines for a project can come about for a variety of reasons. For example, wartime or a change in political winds can lead administrations to push for more “fiscal responsibility.” A common response to this is for DARPA directors to emphasize project plans that can produce applications quickly. On the flip side, certain directors and presidential administrations have been known to openly embrace dual-use technologies and provide generous payoff timelines. DARPA’s exceptional track record often earns the agency a longer leash than is typical. There is also a lot of middle ground between these two extremes. It is not uncommon for certain offices within DARPA to maintain quite short leashes and keep a clear focus on military practicality while, simultaneously, other offices exercise long leashes and focus on more dual-use technology with unclear time horizons for payoffs. This was the case in the 1970s in which the applied Engineering Applications Office (EAO) coexisted with the risk-embracing IPTO.

Early IPTO’s Dual-Use Ethos, the Mansfield Amendment, and George Heilmeier

Obvious shifts in timelines and attitudes towards dual-use technology might abruptly follow a change in the President and/or DARPA director. One example of an abrupt shift in how military-centric DARPA was expected to be came in the early 1970s. In 1970, with the Vietnam War waging and amid a generally troubled political environment, the Mansfield Amendment was passed as an addendum to the Defense Appropriation Act of 1970. The amendment proposed that the DoD should not be allowed to conduct any basic research. While the amendment was repealed within the year, the point Congress intended to make was clear. In response, ARPA soon changed its name to DARPA. The D in DARPA was no longer silent.5

Throughout the 1960s, Licklider and prominent IPTO PMs, such as Larry Roberts, had openly funded the most promising and useful research they could find, often without too much thought to how military-focused it was. In general, the IPTO office visionaries were true believers in the technology and its eventual widespread usefulness in all areas of society. So, they were generally happy to grant funds to any researchers who would push the computing field forward. The general, dual-use ethos that drove these visionaries was something along the lines of, “What’s good for the country will be good for the military.”6

As we now know, this attitude proved to be spot on in the case of computing. When the field of computing was in its nascent stages and much still needed to be worked out, IPTO gave massive block grants to computing “centers of excellence” like MIT, Stanford, CMU, University of Utah, and others. While many somewhat applied projects and direct asks were made of the centers of excellence which were often funded by the block grants, there was a lot of excess funding from the grants which the departments were free to use to pursue their own interests. Notably, the ARPAnet contract itself was pursued by Larry Roberts largely with general use cases in mind. The driving force behind the project was not exactly a military-specific need, but the idea that a broader communication breakthrough like the ARPAnet must improve military operations somehow. Despite the success IPTO had with this approach, this style of operating soon became more difficult.

While the Mansfield Amendment was repealed, it succeeded in forcing changes on DARPA beyond making the D in the name no longer silent. At the minimum, it was now clear to DARPA PMs and office directors that they were being watched. They were now careful to justify projects with specific military applications in mind and find more development projects. Larry Roberts described some of the changes as follows:

The Mansfield Amendment…forced us to generate considerable paperwork and to have to defend things on a different basis. It made us have more development work compared to the research work in order to get a mix such that we could defend it…The formal submissions to Congress for AI were written so that the possible impact was emphasized, not the theoretical considerations.7

However, Roberts did note in his oral history that while the mix of funds did shift to become more applied, for his personal budget DARPA found additional money for applications and that his basic research budget was not cut. While this applied shift felt abrupt to PMs at the time, a modern PM would probably classify this era of DARPA history as relatively exploratory.

This applied shift, which came in the wake of the Masnfield Amendment, ratcheted up a level in 1975. This is when DARPA appointed its most well-known director: George Heilmeier. Heilmeier would become famous for exercising a very heavy hand in deciding which projects would be pursued at DARPA along with his famous Heilmeier Catechism. Office directors and PMs now needed to have satisfying answers to Heilmeier’s six questions related to things like current technological limitations, applications, and intermediate measures of success if they wanted funding for a project. While none of the principles in the Heilmeier Catechism were new to DARPA, the expectation that each of the six questions had a thorough answer was much stricter. Roland and Shiman attempted to describe many PMs’ views on the change, writing:

Heilmeier’s catechism was not unreasonable, but he reportedly applied it with a draconian enthusiasm that reflected a chilling political climate settling over Washington in the first half of the 1970s.

While Heilmeir’s DARPA may not look too strict to modern eyes, it felt very strict compared to prior ARPA regimes. Heilmeier believed DARPA should look much more like mission-driven agencies, such as NASA, than a basic research funder like the NSF. He thought DARPA had unreasonably broadened its scope. He even mocked what had come of DARPA, derisively calling it “NSF West.” Under Heilmeier, PMs were pushed harder to get their exploratory research out into the field as applications. Funding exploratory research with only a hazy idea of what the applications could look like became a taller task than before. For those in D.C. who felt DARPA could be perceived as spending recklessly, Heilmeier was just the man to rein it in.

But Heilmeier’s tenure at DARPA should not be viewed as an event that marked some monotonic decrease in DARPA’s embrace of exploratory, dual-use technology. While the federal government norm is often for things to grow only more onerous over time, in DARPA history it is quite common for operating rules mandating a certain level of strictness to abruptly become less onerous.8 The period coinciding with the departure of Heilmeier as DARPA director and the prospective appointment of Robert Kahn as IPTO Director brought about one of these shifts on an office level.

Under Heilmeier, basic research funding from DARPA had become insufficient by the standards of Kahn and the computing community. New DARPA Director Robert Fossum was sympathetic to this viewpoint and was happy to bring Kahn on as IPTO director and honor his one condition to take the job: that DARPA increase basic research funding in computing. Roland and Shiman described how this shift in how applications-focused IPTO was occurred, writing:

In congressional testimony in 1981, Fossum reported that DARPA funding in constant dollars had fallen steadily from 1970 to 1976; it had not yet recovered its 1970 level when Kahn became IPTO director in 1979. The post-Vietnam heat of the mid-1970s was abating, Fossum did not share Heilmeier’s catechism or his pessimism about artificial intelligence, and Fossum wanted Kahn to take the job. By 1982 the IPTO budget had climbed from its 1979 low back up to a respectable $82 million.

These changes in an office’s focus, which are often the result of non-technological factors, not only impact which projects enter the DARPA funnel and continue to receive funding, but also the life cycle that a given project takes. In the case of DARPA’s 1980 Strategic Computing Initiative and the underlying pressure to have clear applications for the program, this pressure likely had both positive and negative impacts. When assessing DARPA’s impact on the development of autonomous vehicles, the pressure may have been for the best. While the Autonomous Land Vehicle project did come up short, the push to get vision applications into the real vehicles quickly allowed CMU to prove itself to be DARPA’s great systems integration contractor in the field of autonomous vehicles. On the flip side, in the case of the Pilot’s Associate program, the PM in charge of the program thought the premature push for a mature prototype took a program that was making solid technological leaps and forced it to abruptly implement a system that was not as cutting-edge or intelligent as it could have been otherwise.

In very obvious ways, variations in DARPA’s attitudes towards dual-use technology and the pursuit of projects with long-run or unclear payoffs impact which projects get chosen and how they are pursued. But one would be incorrect to view some eras of DARPA as simply better or worse places for the most ambitious research. For example, while some (too crudely) might view Tony Tether’s wartime DARPA agency in the 2000s as a more difficult place to pursue ambitious programs — given its use of procedures like budget wedges and emphasis on programs that might pay off in the current war — an entrepreneurial PM like Geoff Ling proved this was an ideal time to pursue one of DARPA’s most ambitious programs ever: the Revolutionizing Prosthetics Program. Ling’s program made major breakthroughs in neurally integrated upper limb prosthetics and brain computer interfaces (BCIs) that helped lay the base for some of the ambitious BCI projects of today.

While the impact of an office’s attitude on dual-use technology and payoff timelines is somewhat intuitive, one important wrinkle to consider is whether the customer (the military) must actually agree an application is desirable before a project is funded. Whether or not an office can pursue a project despite the military being lukewarm or cold on an idea has a massive impact on what sorts of projects get pursued.

A military-focused office can often say “no” to the military

Military-focused project selection and listening to the military are not always the same thing. Different DARPA regimes and offices can have varying abilities to say “no” to the armed services. In the case of the Pilot’s Associate, the Air Force did want it and indicated they‘d be willing to actively participate in its development. In the case of the ALV, the Army did not express much desire for an autonomous recon vehicle — at least not strong enough to warrant DARPA’s spending on the vehicle — but IPTO staff had a strong belief that the Army would want the ALV once they saw it in action. So, DARPA was comfortable making the ALV application the centerpiece of its SCVision portfolio. This ability to say no to the army in the 1980s DARPA computing portfolio stands in stark contrast to the accounts of some PMs who worked under directors like Tether. Tether often mandated that PMs obtain a budget wedge — a formal promise by an armed service to continue funding a particular project if the R&D benchmarks reached a certain point — before beginning to fund many projects.

In Tether’s case, the recent 9/11 attacks and the country transitioning into wartime allegedly had a strong impact on this decision. Ling’s prosthetics program was the right kind of ambitious for the moment; meanwhile, overriding the no of the military to build seemingly futurist technology the Army said it did not want likely would not have been. One cannot truly understand and apply the lessons of any project’s success or failure without understanding the office’s attitudes towards dual-use technology, payoff timelines, and saying no to the customer in a given era.

Concluding Thoughts

DARPA is no longer a young government agency. In a difficult environment, it has managed to remain a place where bright minds take risks, act (relatively) nimbly, and get things done. The agency continues to specialize in pursuing projects that are big-if-true, fall through the cracks of the standard R&D ecosystem, or might succumb to the valley of death without DARPA’s intervention. The agency’s history is one we would be foolish not to learn from.

To adequately learn from this history, it’s crucial we understand what changes and what stays the same about the agency over time. Certain key factors have remained constant throughout (D)ARPA’s life. The twin constants that, to me, define the ARPA model over time are the use of a clear customer relationship as the agency’s North Star and the use visionary PMs (or office directors) with agency instead of passive funding panels. But it is possible that there are even more key variables across DARPA history than there are constants. Today’s piece outlines what I believe to be three of the most important ones those. Those looking to learn applied lessons from the agency’s history should keep them in mind.

Thanks for reading:) As always, I’d be eager to discuss/assist PMs or PM-like individuals in any way I can. Just reach out to me on Twitter (eric_is_weird) or via email (egillia3@alumni.stanford.edu) to chat.

Announcements

An Announcement on RenPhil’s AI For Math Fund. Talented and curious readers should eagerly apply for grant funding from RenPhil and XTX Markets’ AI for Math Fund. The funders are particularly interested in those working on software, datasets, field building, and big-if-true projects to build AI technologies useful to mathematicians.

A Coming Piece Will Cover Constants Across DARPA History. While the focus of this piece was on how DARPA has changed over time, there are also important constants throughout DARPA history. I will say more about these in a brief follow-up piece on FreakTakes. For now, I’ll simply say that the two most notable constants are probably:

1. DARPA’s continued commitment to replacing the panels typical in government R&D grant-making with PMs with agency

2. Its use of a clear customer relationship as a strong North Star for its projects

Subscribe now

Share

Appendix: A Note on Other Headwinds In The 1980s

Regulations are not solely to blame for the initial negative results of the ALV project, and projects like it. In the mid-1980s, Ohlander was fighting against two other sweeping trends that surely made the life of a PM for a project like the ALV harder. Many groups that would once have been more aligned with DARPA’s incentives were gradually drifting in scope.

The first of these headwinds was the gradual drift of university labs away from applied research contracts. With post-war federal basic research funding continuing its explosion, universities were leaning more and more into basic research funding and further away from the applied research contracts that were a key source of funding in prior decades. University labs were gradually growing less well-suited to DARPA-style contracts that required a very applied technological component — like operating a working vehicle in which to plug academic vision algorithms.

The second headwind PMs like Ohlander were fighting against in the 1980s was an increasingly myopic sense of what projects corporate R&D should pursue. A 1970s recession, the 1980s “financialization” of management in engineering organizations, and likely other causes saw corporate research drift in its ambition. My prior FreakTakes piece on Lockheed’s Skunk Works provides an interesting case study of what these pressures looked like from within a defense prime. It was even difficult for Kelly Johnson and the famous Skunk Works to fight against them as the decades wore on.

Both of these trends happening simultaneously contributed to an organization like BBN’s ability to recruit top academic engineers from places like Lincoln Labs so convincingly. BBN’s recruits simultaneously felt that Lincoln Labs did not work on truly useful technology, but also that their minds would be wasted at standard corporate R&D departments like Honeywell’s. BBN was one of a small number of organizations that still existed to work on truly novel, truly useful applied research contracts. These two simultaneous trends likely had their own negative impacts on DARPA on top of the negative impacts brought on by the new procurement rules. But, at the minimum, the growing apart of industry and academic research made it even more vital that DARPA keep a strong hand in coordinating — and when needed, twisting the arms of — performers on an ongoing basis. The new procedures made this much more difficult.

General Links

Beyond the specific citations above, much of the general knowledge in the piece is indebted to the great books and oral histories on early IPTO conducted/written by Judy O’Neill, Arthur Norberg, William Aspray, Alex Roland, and Philip Shiman:

• University of Minnesota IPTO oral histories can be found here

• Transforming Computer Technology: Information Processing for the Pentagon, 1962-1986

• Strategic Computing: DARPA and the Quest for Machine Intelligence, 1983-1993

Subscribe now

What the Skunk Works does is secret. How it does it is not. I have been trying to convince others to use our principles and practices for years. The basic concept as well as specific rules have been provided many times. Very seldom has the formula been followed. — Clarence “Kelly” Johnson

Introduction

This piece will round out a trio of pieces covering exceptional contractors from ARPA’s early decades. The first two pieces covered BBN and CMU’s autonomous vehicle teams. The former was a research firm unusually focused on novelty; the latter was a group of university researchers unusually focused on shipping real technology and incorporating firm-like management structures. Unlike those organizations, Skunk Works appears at first glance more similar to a traditional defense contractor than a research lab. But the novelty of its outputs demonstrates that Skunk Works is an all-time great R&D lab in its own right. Skunk Works got much more funding from the armed forces and CIA than it did ARPA. But it still deserves to be covered in the ARPA playbook because, to many, the name “Skunk Works” has become synonymous with the topic of defense innovation.

Today, many view a big bureaucracy like a defense prime as a terrible place to build ambitiously. This is more true today than it was in the 1950s, but it was still often true back then. Yet, Kelly Johnson was able to build an applied research powerhouse within Lockheed that designed arguably the three most impressive spy planes and stealth bombers in American history. To top it off, many of Skunk Works’ novel planes were built on budget and in only 180 days. This was all made possible by Johnson’s clear vision of what was important when running a lab and the trust Lockheed management and DC project funders had in him.

But before I detail how Kelly built Skunk Works, I must first dive into how Kelly Johnson earned his stripes as a young aircraft engineer. This is key to understanding his management of the Skunk Works. In many ways, his management practices sought to maintain an environment similar to that of the young aviation industry he came up in.

The Education of Kelly Johnson

To many decades-long Skunk Works employees, the essence of the lab was inseparable from its founder and boss: Kelly Johnson. So, to begin to understand the lab and how it operated, we first need to understand Johnson and his background.

As a boy, Kelly came to the decision that he wanted to learn how to build airplanes. He read every book on the only two-decade-old subject he could get his hands on at the local Carnegie Library. When he got to the University of Michigan, he quickly found a job helping a professor operate the school’s wind tunnel. The professors at the University of Michigan, like many engineering schools at the time, did much of their research on industry contracts. This gave Kelly the opportunity to help improve real vehicles for paying customers. In his 1985 autobiography, More Than My Share of It All, Kelly describes several contracts he was working on with a Professor Pawlowski, writing:

He, like some of the other professors, had contracts outside the college. In the wind tunnel, I worked for him on design of the Union Pacific streamlined train, on a smoke-removal project for the city of Chicago, and on one of the very early proposals for generating energy with a wind machine.

In addition to assisting professors with their projects, Kelly asked for permission to take on his own contracts. They agreed and said that he could use the wind tunnel when it was idle. He simply had to pay the university an hourly fee to rent it. The first customer he sought out was the Studebaker Motor Company. Kelly was confident he could help the company. The wind tunnel was a relatively young scientific instrument. Many companies did not have one of their own. He described his attitude going into the contract, writing:

It was obvious that the wind tunnel could be very useful in designing streamlined automobiles. We got an assignment to test the Pierce Silver Arrow, which was to become one of the early “totally-streamlined” cars. We knew all the tricks on how to reduce drag caused by air resistance. We found, for instance, that the big ugly headlamps on Studebaker cars were eating up 16 percent of the power the engine developed at 65 miles an hour. We managed to get them shaped into the fenders. We worked on a lot of other problems and ran many, many tests.

In 1932, Johnson graduated with his bachelor’s degree. It was a horrible job market, but he and a friend borrowed a professor’s car and drove to the West Coast to introduce themselves to aircraft companies to see if they had any work available. One of the companies was a small outfit named Lockheed which had recently been purchased for $40,000 (~$850,000 today). They told Kelly that they did not have a job for him, but that if he came back next year they might have something for him. So, he returned to Michigan and enrolled in a master's program. In this period, his consulting and research assistant projects became even more impressive. The outputs of one of his personal contracting endeavors even made a splash in the local newspaper:

Five of the qualifying cars which will race at Indianapolis Memorial Day have bodies designed by two university graduate students, C.L. Johnson and E.D. Palmer. All of the cars are semi-stock Studebakers and all qualified for the race at speeds ranging between 110 and 116 miles an hour.

Kelly referred to the experience earned on these contracts as “a liberal education for me in the practical application of aerodynamic theory.”

During this year, one of the airplane models sent to Kelly’s professor for wind tunnel testing came from Lockheed. The model, which would become the Lockheed Electra, was in a new business area for the small firm. Board Chairman Robert Gross had decided that the recently re-organized company’s future was no longer in single-engine wooden aircraft, but in twin-engine all-metal designs. These designs could carry more passengers. At the end of testing, Kelly and his professor markedly disagreed on the readiness of Lockheed’s model. Johnson noted:

It developed some very serious problems, I thought, from what I then knew of aerodynamics. It had very bad longitudinal stability and directional-control problems. But most aircraft of that day had similar failings. Professor Stalker, in consultation with Lloyd Stearman, already a recognized top-notch designer at age 33 and first president of the company, decided the figures were acceptable.

The report was sent to Lockheed without Kelly’s opinions included.

After completing his additional year at Michigan, Lockheed hired Johnson to work under Chief Engineer Hall Hibbard — a young man himself, only seven years older than Johnson. Kelly was going to be making far less than he was running his consulting projects at Michigan. Johnson recounted his initial hiring arrangement, writing, “I was to receive $83 a month (~$2,000 today) to start in tool design until they could assign me as an engineer. There were five engineers at the time.” He may have received a small pay bump when he got his title changed to engineer. But this was not about the money for Johnson. This was about doing what he always wanted to do: designing new planes. He reflected on why leaving his lucrative wind tunnel practice and joining Lockheed was an easy decision, writing:

At the university we certainly weren’t going to design aircraft — and that was my goal. But I didn’t make that much money again until ten years later.

Johnson was outspoken from his first moment at the company. He was not one to sit quietly if he thought mistakes were being made. Johnson recalled his first days at the company, writing:

Practically the first thing I told Chappellet and Hibbard was that their plane was unstable and that I did not agree with the university’s wind-tunnel report.

Hibbard was not insulted. He was, instead, quite open-minded and would not forget that Johnson had said this — even if he did not scramble to fix the problem at that moment. Hibbard recounted years later in an interview that he had wanted to get some “new young blood…fresh out of school with newer ideas” in his engineering department. Kelly Johnson proved to be that and more.

But, initially, the young Johnson was assigned to work in the tooling department. There, he received a kind of hands-on education that he felt most airplane designers couldn’t find anywhere in the post-war years. He took quite well to the hands-on nature of the work. He had, after all, spent much of his youth working in construction. In the tooling department, he worked with an old hand named Bill Mylan and learned the airplane production business from the workman’s end of things. While in the tooling department, Kelly learned to actually build planes, designed a furnace to heat-treat a metal that was new to Lockheed’s production, and made beginner’s mistakes passing design instructions to workmen. In no small part, Kelly credited these early experiences with helping him understand how to design airplanes that were buildable.

Johnson’s Growing Reputation

After a few months of tooling work, Hibbard called Kelly into his office to give him a new assignment. Hibbard was going to put the young would-be engineer’s ideas to the test. The assignment would prove that Johnson’s wind tunnel days had left him with a remarkably deep intuition for aerodynamics. Johnson recalls the meeting with Hibbard and getting sent back to his old wind tunnel, writing:

“Kelly, you’ve criticized this wind-tunnel report on the Electra signed by two very knowledgeable people. Why don’t you go back and see if you can do any better with the airplane?”

Hibbard sent me back to the University of Michigan wind tunnel with the Electra model in the back of my car. It took 72 tunnel runs before I found the answer to the problem…It was a process of evolution. On the seventy-second test, I came up with the idea of putting controllable plates on the horizontal tail to increase its effectiveness and get more directional stability. That worked very well, particularly when we removed the wing fillets, or fairings onto the fuselage — put on apparently because they were coming into style and being used successfully on such airplanes as the Douglas DC-1. And we avoided the trouble others had with them when not used properly…We then added a double vertical tail because the single rudder did not provide enough control if one engine went out. That was so effective we removed the main center tail. And there you had the final design of the Electra. The distinctive twin tails on all of the early Lockheed metal airplanes, and the triple tail of the familiar Constellation airliner of the mid-’40s and ’50s, were the result of these tunnel tests.

Hibbard was impressed and sent Johnson a note telling him how great of a job he had done. Johnson very proudly noted that:

When I returned to the plant. I was a full-fledged member of the engineering department. I was number six.

As a member of the small engineering department, he did crucial work on all parts of the Electra. In the process of learning to wear so many different hats for the Electra and other projects, Kelly felt that he became an “honest-to-god aircraft engineer.”

I worked not only on the aerodynamics of the airplane, but on stress analysis, weight and balance, anything and everything they threw at me. And, of course, more wind-tunnel testing. From that, I became the logical choice to be flight test engineer on the airplane when it was ready to fly.

Kelly noted that the Electra, with its iconic twin-tail, became the fastest multi-engine transport in the world at the time, with cruising speeds above 200 miles per hour. Lockheed delivered the first aircraft in mid-1934. At the time, the small company had six engineers and about 200 factory crew.

Over the next few years, Johnson would do all sorts of work on several derivatives of the Electra. For one of these planes, the Model 14, Kelly ran extensive wind tunnel tests to help design a new airplane part called wing flaps. These flaps allowed the wing to remain large during takeoff to aid in control and then become smaller in flight and increase speed. This Lockheed-Fowler Flap earned Johnson the Lawrence Sperry Award — the first of countless aviation awards he would win in his storied career.

But this achievement was nothing compared to Kelly’s 1938 efforts. With World War II ramping up, the British knew they needed an anti-submarine patrol plane. Getting word of this, Lockheed quickly threw together a proposal for an anti-submarine version of their new Model 14 to show the British. The British representatives were so impressed at how quickly the small company incorporated the recommendations from the group’s first meeting — in a single weekend — that they invited Lockheed to meet with their technical staff in England. Lockheed sent a four-person team on the trip. Kelly was the sole representative for the engineering team.

In the first meeting in England, the British requested several adjustments that required an almost complete re-design by the Lockheed team. Upon leaving the meeting, the team acquired a drawing board and drafting equipment. Kelly then proceeded to lock himself in a hotel room over the three-day weekend. Johnson recalled the event, writing:

I had to fit in all this new equipment, re-arrange copilot and radio operator positions, make weight and structural analysis, figure contract pricing, and guarantee that the design would meet certain performance requirements.

It was a three-day holiday weekend — Whit Sunday, Whit Monday. I worked a solid 72 hours on this redesign — not taking time for sleep, just catnapping briefly when absolutely necessary. I was a rumpled figure…When finally I fell into bed for some very sound sleep — in the room I shared with Courtlandt to save on expenses — it was the first time I had removed my clothes in 72 hours.

Johnson’s efforts were well worth it. The team at the Air Ministry was extremely impressed. Amid an ongoing war, Lockheed’s ability to work quickly was reassuring. Lockheed closed the deal in England that week. Before giving Lockheed a contract to build at least 200 these Hudson airplanes, the British only had one point of concern for Lockheed. The impressive designer who re-modeled the plane over the weekend was extremely young — Johnson was 28. In fact, the entire Lockheed team was young. The leader of the attaché, Courtlandt Gross, was only 36. Gross recounted Air Marshal Sir Arthur Varnay expressing his one concern as follows:

Mr. Gross, we like your proposal very much, and we very much like to deal with Lockheed. On the other hand, you must understand that we’re very unused in this country to dealing — particularly on transactions of such magnitude — on the technical say-so of a man as young as Mr. Johnson. And, therefore, I’ll have to have your assurance, and guarantee, in fact, that if we do go forward, the aircraft resulting from the purchase will in every way live up to Mr. Johnson’s specifications.”

Gross assured Varnay that he and his brother, Lockheed’s Chairman, had every confidence in the young Johnson. With this assurance, a deal was struck. It was, according to Kelly, “the largest aircraft production order placed up to that time in the United States.”

Not long after to returning from the trip, Johnson was named Lockheed’s Chief Research Engineer. The young man now had a big title and a fast-growing reputation.

Skunk Works is Born

As the war progressed and Lockheed’s production lines rolled off Hudson’s vital to the war effort, the firm found its way into more and more contracts. Before the US entered the war, Lockheed had proposed building a prototype jet engine-powered plane for the Army Air Corps. The Air Corps was initially cool on the idea, but eventually agreed as they saw the British working on a similar aircraft. Kelly agreed to produce an experimental model of the proposed aircraft in a timeline that would become quite standard for his Skunk Works crew: 180 days.

The Army was quick to get Kelly his contract. Kelly recalled how quickly General Frank Carrol went from deciding this was a good idea to producing a contract, writing:

“We’ll give you a contract for the airplane, Kelly, and for Nate’s engine as well,” said Gen. Frank Carrol, commanding officer of Wright Field. “But you’ll have to use the British engine in the first airplane because we need it — and all the jet fighters you can build — as soon as possible to use against the Me-262. Your new engine couldn’t possibly be ready for service in time.”

Since I had promised to build a jet airplane within 180 days, I asked, “When will we get a contract? When will the time start?” “You will have a Letter of Intent this afternoon by 1:30 p.m.” he replied. “There is a plane leaving Dayton for Burbank at two o’clock. Your time starts then.”

And it did. The date was June 8, 1943. Gen. H. H. “Hap” Arnold, himself, had approved the contract.

Upon returning to Lockheed’s Burbank headquarters with the good news, Robert Gross was quick to tell Kelly that Lockheed had few people, few machines, and almost no room for such a project. They were already up to their eyeballs producing around 28 airplanes per day — P-38s, B-17s, and Hudsons, and others. Gross was also quick to tell Kelly that he thought nothing would come of the contract. But none of that meant Johnson wouldn’t be allowed to pursue the project. Kelly wrote:

But he [Gross] and Hibbard always were open to new ideas and backed me in many critical times…“You brought this on yourself, Kelly,” Gross stated. “Go ahead and do it. But you’ve got to rake up your own engineering department and your own production people and figure out where to put this project.”

This was an opportunity much bigger than just this one contract for Johnson. The Lockheed company had been rapidly expanding throughout his tenure there, and he felt this was making the design of new, interesting planes more difficult. He reflected on these difficulties, writing:

For some time I had been pestering Gross and Hibbard to let me set up an experimental department where the designers and shop artisans could work together closely in development of airplanes without the delays and complications of intermediate departments to handle administration, purchasing, and all the other support functions. I wanted a direct relationship between design engineer and mechanic and manufacturing. I decided to handle this new project just that way.

Upon receiving Gross’ green light, Kelly set up some crates adjacent to the new Lockheed wind tunnel and found a circus tent to put over the top of them. With that, the Skunk Works was born.

To acquire more tools, Kelly got permission to buy out a small local machine shop. He scrounged together 22 other engineers from around the company who he had worked with before and trusted. Johnson had high standards, and he believed they were all very good. He also set the group up with its own purchasing department. To Kelly, it was vital to Skunk Works’ success that “every function…needed to operate independently of the main plant.”

The Air Corps moved almost as fast as Johnson did. Nine days after the project was green-lit, Skunk Works hosted colonel and a major at Skunk Works for a mockup conference that went smoothly.

Many complications arose in completing the project. Kelly emphasized that “it was a complete new world of flying and testing” with these jet-engine aircraft. The plane ran into the same compressibility problem when it approached Mach 1 that Lockheed had encountered with its P-38 development. The team was able to overcome the airplane’s aileron buzzing by installing shock absorbers in just the right fashion. In addition, common problems present in new aircraft designs popped up. Certain parts of the plane proved precarious to weld together. Extensive re-designing and vacuum testing were required to overcome these issues. But whatever problems came up, the small team of particularly talented engineers and workmen worked through them quickly. The small operation seemed to be as effective at producing experimental aircraft as Kelly had hoped.

And the Air Corps was also an excellent project partner. This was vital, in Kelly’s estimation. He noted that:

There were only six people from the military involved and two or three of us from Lockheed. We had approval to proceed that night. Six days later we had our government furnished equipment — guns, radio, wheels and tires, etc. At every stage of the work, we had excellent cooperation from Wright Field and the officers involved with the project. The job could not have been completed on such a tight schedule without it.

Skunk Works delivered the XP-80 plane to the Air Corps on Day 143 of the contract — more than a month ahead of schedule. The XP-80 took its first flight on January 8, 1944. Almost immediately, the Air Corps gave Skunk Works another contract to produce a modified version of the plane with an airframe almost twice as large — to make use of GE’s new engines. The two copies of the aircraft the Air Corps requested were delivered 132 days later. This design, the YP-80, was the precursor to the famous P-80 and two-seat T-33 training vehicle. According to Kelly, there went on to be more than 6,000 built in total.

The plane would not go on to see major use in World War II given the ending of the conflict. But it would later prove itself — first in the North Korean vs. South Korean conflict in 1950 — and also set several records. The most notable of which is probably the F-80 flying the route from Long Beach to LaGuardia in a record 4 hours and 13 minutes, averaging 584 miles per hour.

These first Skunk Works projects were a smashing success. Producing a prototype of a truly novel aircraft will never be completely smooth. But with these projects the team at Skunk Works was beginning to make it clear that they could routinely overcome these obstacles on tight schedules, even if not easily.

The First Iconic Skunk Works Plane: the U-2 “Spy Plane”

In 1953 the group began work on their most ambitious airplane yet. Around this time, defense contractors were made aware of the country’s desperate need for aircraft that could fly recon missions over the USSR to acquire information on Soviet military capabilities. They needed equipment that could do this without being shot down by the Soviet’s advanced anti-aircraft weapons. Always game for a challenge, the Skunk Works put together a proposal for a plane that could fly approximately 70,000 feet, have a range of 4,000 miles, and would be a steady platform for advanced photographic equipment. Without a plane that could reach that height, the CIA and Air Force would have to resort to low-flying, high-casualty tactics to acquire information1

The Air Force initially did not think Skunk Works’ proposal was realistic. Other proposals to achieve this altitude were somewhat primitive technology like weather balloons.2 But the folks in DC decided to let the Skunk Works go ahead with the project anyway. Many in DC had exceptional faith in Kelly Johnson, just as the Gross brothers and Hall Hibbard did back in Burbank. Johnson describes the events that led to Skunk Works winning the contract as follows:

Our first presentation was to the Air Force, where it was turned down as too optimistic. They questioned that any engine even would operate at the altitude we were proposing. They were correct in that there was not proof at that time that this was possible. And the Air Force already had an airplane in development with the Martin company — an airplane with two engines which was preferred to our single-engine design.

But our proposal reached Trevor Gardner, then Assistant Secretary of the Air Force for Research and Development, and a brilliant engineer in his own right. Late in 1953, he invited me to come to Washington to discuss it. He had assembled a committee of scientists and engineers, and for three days they put me through a grilling as I had not had since college exams. They covered every phase of the aircraft design and performance — stability, control, power plants, fuels — everything.

Later I met and lunched with Air Force Secretary Harold Talbott, CIA Director Allen Dulles, and his right-hand man, Larry Houston, among a distinguished group. When I was asked why I thought Lockheed could do what I proposed — build 20 airplanes with spares for roughly $22 million and have the first one flying within eight months, Gen. Donald Putt graciously volunteered, “He has proven it three times already—on the F-80, F-80A, and F-104.”

The CIA took the lead in funding the project and Richard Bissel — Special Assistant to CIA Director Dulles and a fixture of post-war CIA leadership — became Skunk Works’ project liaison.

Kelly selected 24 other engineers to work on the project with him. The total project staff would eventually grow to a headcount of 81. Given the secrecy of the project, Skunk Works was not allowed to fly the plane at its Burbank headquarters. So, Kelly sent a test pilot around to find a secure location on which to build a runway and basic testing facilities. He named the remote strip of land that the pilot found in the Nevada Dessert “Paradise Ranch.” Today it is known as Area 51. Once selected, Lockheed hired contractors to begin constructing some roads, hangars, offices, living accommodations, etc. on the land.3 All of this was done while Skunk Works was working on the novel aircraft.

Skunk Works had to keep an extreme focus on ruthlessly minimizing weight throughout the design and construction of the aircraft. Many staple building materials had to be made thinner or replaced altogether. The entire team was constantly scouring for lighter materials and designs for each part of the aircraft. Each would have to be extensively tested and modified until the engineers were comfortable using it. All of this led to a quite interesting aircraft. Ben Rich explains how strange the aircraft seemed to other pilots and engineers, writing:

We designed and built that airplane for lightness. The wings, for example, weighed only four pounds per square foot, one-third the weight of conventional jet aircraft wings. For taxiing and takeoffs, jettisonable twin-wheeled “pogos” were fitted beneath the enormous fuel-loaded wings and kept them from sagging onto the runway while taking off. The pogos dropped away as the U-2 became airborne.

The fuselage was fifty feet long, built of wafer-thin aluminum. One day on the assembly floor, I saw a worker accidentally bang his toolbox against the airplane and cause a four-inch dent! We looked at each other and shared the same unspoken thought: was this airplane too damned fragile to fly?…Pilots were scared to death flying those big flapping wings into bad weather situations — afraid the wings would snap off…Adding to the sense of the airplane’s fragility was that the razor-thin tail would be attached to the fuselage by just three five-eighth-inch bolts.

(The plane and its enormous, drooping wings that almost sagged onto the runway, proved more resilient than the pilots or designers ever imagined.)

The engine was also a massive obstacle. This was partially a problem for the Skunk Works team but primarily fell to a long-time Skunk Works contractor. Rich described the difficulty in building an engine to operate at these high altitudes, writing:

My principal work was on the engine’s air intake, which had to be designed and constructed with absolute precision to maximize delivery of the thin-altitude air into the compressor face. Up where the U-2 aimed to cruise, just south of the Pearly Gates, the air was so thin that an oxygen molecule was about as precious as a raindrop on the Mojave desert. So the intakes had to be extremely efficient to suck in the maximum amount of oxygen-starved air for compression and burning.

In the end, the engine problem was overcome by the work of Skunk Works and two very impressive contractors. The most important contributor was probably Pratt & Whitney, a long-time Skunk Works partner. The P&W team modified an existing high-altitude engine they made that was considered top-of-the-line. The head of Pratt & Whitney himself put his best people on the job and they found a way to modify the engine so it could function in the extreme conditions. A major contribution was also made by Shell’s research team. Kelly Johnson got Shell’s R&D team to develop “a special low-vapor kerosene for high altitudes” that would not boil or freeze in the extreme conditions in which the U-2 operated. In many Skunk Works projects, key contributions from trusted contractors were vital.

The U-2 was completed and shipped to Paradise Ranch in late July — less than nine months after the contract was first issued. In early August, it accidentally took flight for the first time while conducting a routine taxi test. As test pilot Tony LeVier recalled, “Who’d have guessed an airplane could take off only going 70 knots? That’s how light it was.” The plane took its first official test flight a week later. The high-flying plane was a marvel, “able to glide 250 miles from 70,000 feet.” Within nine months of the first test flight, the CIA U-2s became operational.

On top of the U-2s achieving Kelly’s lofty technical goals and obtaining high-priority intelligence on the USSR, many consider this contract one of the greatest bargains in defense procurement history. Johnson and the Skunk Works came in far under budget. They returned $2 million of the original $22 million budget and were able to construct six extra airplanes from the spare parts the team didn’t need. Rich said they were able to build with these spares simply “because the U-2 functioned so well.”

What did Kelly Johnson think was Skunk Works’ special sauce?

Skunk Works’ success on the U-2 was not an isolated event. The Skunk Works would also go on to produce two more planes considered all-time greats — the Blackbird models and the F-117 stealth bomber — in addition to dozens of other successful planes. This, of course, raises the question: how did Skunk Works staff build so many novel planes on such a routine basis? Did they have a kind of special sauce?

Kelly Johnson believed they did. Kelly famously had a list of 14 rules he used to run the Skunk Works. However, upon reading his biography, it becomes clear that the beliefs and experiences that led to those rules could be distilled down to something like three key principles. The three principles are, roughly:.

1. Reduce the bureaucracy to almost zero. Ideally, one person should have almost complete authority over day-to-day decision-making.

2. Keep the team ruthlessly small.

3. Whenever possible, only take on contracts where there is enough mutual trust with funders and subcontractors to work with them with a minimum amount of bureaucracy. If funder decisions cannot be made swiftly, the project is probably not worth pursuing.

The three beliefs above can be seen at work all over the Skunk Works operation. The following subsections will give the reader a clear idea of how these beliefs manifested themselves in the practical management of the Skunk Works.

Luckily, in More than My Share of It All, Kelly Johnson dedicates an entire chapter to explaining the simplicity of his special sauce. In the chapter, he makes it clear that he can’t believe how few true imitators Skunk Works had — given that so many companies claimed to have their own Skunk Works. Kelly opens the chapter, a bit incredulously, writing:

What the Skunk Works does is secret. How it does it is not. I have been trying to convince others to use our principles and practices for years. The basic concept as well as specific rules have been provided many times. Very seldom has the formula been followed.

As this section will detail, the lack of true Skunk Works imitators will not be shocking to any who are familiar with bureaucracies. The key management principles that helped Skunk Works thrive are simply non-starters in most bureaucracies. This is precisely why so many of Kelly’s rules-of-thumb are obsessed with warding off bureaucratic processes; this is also why Ben Rich lived in such fear of bureaucracies smothering Kelly’s great band of engineers from all sides when he took over the Skunk Works.

Principle #1: Minimize Bureaucracy, Centralize Power

For a variety of reasons, Kelly believed bureaucracies killed progress in designing and building new aircraft. Kelly believed one person should have complete control over day-to-day decision-making. An engineer should be able to come to him with a problem, and he needed to have the authority to tell them “yes” or “no” on the spot.

This type of approach may not have been as rare in his youth, but as his career wore on it became almost extinct. He lived to see the onset of slow decision-making processes begin to expand in the industry. Lamenting this, he writes:

I fear that the way I like to design and build airplanes one day may no longer be possible. It may be impossible even for the Skunk Works to operate according to its proven rules at some point in the future. I see the strong authority that is absolutely essential to this kind of operation slowly being eroded by committee and conference control from within and without. The ability to make immediate decisions and put them into rapid effect is basic to our successful operation.

In the 1950s, even the comparatively small Lockheed had grown bureaucratic enough to stifle ambitious work. Before being recruited to the Skunk Works in the late 1950s, Ben Rich worked in the main portion of the Lockheed company. He said the following about his time there and how the company operated:

Lockheed was very regimented and bureaucratic, and by my fourth year on the payroll I felt stymied and creatively frustrated. I had a wife and a new baby son to support, and my father-in-law, who admired my moxie, was pushing me to take over his bakery-delicatessen, which earned the family a very comfortable living.

The two decades between Kelly Johnson and Ben Rich joining the company had seen Lockheed cease to be the kind of place where an exceptionally driven individual or a genius had the chance to spread their wings. Hall Hibbard used to say that Kelly Johnson could “see air.” The then-Chief Engineer kept assigning the young Johnson more and more responsibility until, by the time he was made Chief Research Engineer at 28, he’d been largely responsible for many key aspects of Lockheed’s designs anyway. Rich — who was eventually Kelly’s hand-picked protegé — on the other hand, was considering leaving Lockheed to run a deli!

Johnson wrote the management chapter in his book hoping that individuals would find it who were brave enough to run a truly ambitious R&D operation. While the aircraft industry did not take his advice, readers of this Substack would be wise to. The same reasons that large organizations resisted properly copying Skunk Works back then are similar to the excuses they would use today — many aspects of bureaucracies and their emergent behavior are evergreen. Johnson outlined the primary reason truly copying Skunk Works was a non-starter in most companies, writing:

Most companies, while desiring the benefits, will not pay the price in revised methods and procedures for setting up a Skunk Works-type of operation. They will not delegate the authority to one individual, as Lockheed did in my case from the very first Skunk Works. It requires management confidence and considerable courage. Without the authority assigned to the Skunk Works by our military customers and the Lockheed corporation, we would not have been able to accomplish many of the things we have done, things about which I felt we could take a risk — and did.

Skunk Works may never have been green-lit in the first place had Hibbard and Gross not had an exceptional level of trust in Kelly Johnson. As Rich recalled, there was a general sentiment at Skunk Works when he joined in the late 1950s: “The open secret in our company was that the chief engineer [Kelly] walked on water in the adoring eyes of CEO Robert Gross.” And Johnson took full advantage of his well-earned long leash.

Kelly set up Skunk Works with a belief in the supreme power of individual visions. He did not believe that committees would design broken planes. But he did believe that they would produce uninspired and unoriginal planes. And there was absolutely no room for processes like that when the goal of his team was to design truly novel, experimental aircraft. He reflected on the impacts of bureaucracy on airplanes themselves, writing:

There is a tendency today, which I hate to see, toward design by committee — reviews and recommendations, conferences and consultations, by those not directly doing the job. Nothing very stupid will result, but nothing brilliant either. And it’s in the brilliant concept that a major advance is achieved.

Development of some of this country’s most spectacular projects — the atom bomb, the Sidewinder missile, the nuclear-powered submarine — all were accomplished by methods other than the conventional way of doing business outside the system.

Johnson’s efforts to minimize bureaucracy and belief in individual visions allowed Skunk Works to both produce ambitious plane designs and build them on short timelines. Kelly recruited some of the best Lockheed had to offer. He implemented a system where a small number of simple drawings were used and things could be changed on the fly. He put all the engineers, designers, and workmen in close quarters. And he did everything he could to enable them to test ideas as fast as they had them.

A genius would feel challenged by the environment and Johnson’s no-bullshit technical grillings, not stifled by bureaucracy. In his book, Ben Rich referred to a 24-year-old aerodynamicist working on a project with him, the late Dave Campbell, as “destined for greatness.” Skunk Works was an environment where those qualities could both show and be taken advantage of. The primary portion of Lockheed was no longer like that. But in Skunk Works, Kelly kept a culture alive similar to the company he first arrived at in the 1930s. In that young Lockheed company, with only five engineers, Kelly was not just an aerodynamicist, but an aerodynamicist, stress analyst, weight/balance engineer, and flight test engineer all in one. He was challenged and largely unencumbered by rules and processes. In that role, his genius could truly be put to work.

When Kelly Johnson was at the helm of the Skunk Works, the individual visions overriding the projects were often his own. Of course, during his reign there was plenty of room for Skunk Works staff to demonstrate their genius on their own components of the plane, helping others problem solve, or via influencing Johnson with their ideas. Johnson even kept quarters on him to dole out if anyone ever proved him wrong. However, the fact of the matter is that in his decades-long run as the lab’s leader, he lost very few quarters. Rich said the following on Johnson’s wide-ranging, freakish understanding of aircraft design:

Nothing got by the boss. Nothing. And that was my sharpest impression of him, one that never changed over the years: I had never known anyone so expert at every aspect of airplane design and building. He was a great structures man, a great designer, a great aerodynamicist, a great weights man. He was so sharp and instinctive that he often took my breath away. I’d say to him, “Kelly, the shock wave coming off this spike will hit the tail.” He would nod. “Yeah, the temperature there will be six hundred degrees.” I’d go back to my desk and spend two hours with a calculator and come up with a figure of 614 degrees. Truly amazing. Or, I’d remark, “Kelly, the structure load here will be…” And he would interrupt and say, “About six point two p.s.i.” And I’d go back and do some complicated drudge work and half an hour later reach a figure of 6.3.

With someone like Kelly around, it makes sense that his vision was the individual vision elevated to the fore. But the lab’s success was not dependent on his vision. Arguably the lab’s greatest military contribution, the F-117 stealth bomber, came in its first project after Johnson retired.

Kelly picked Ben Rich as his successor because he thought Ben was innovative, practical, and an absolute terrier in working through some of Skunk Works’ most difficult problems. Those traits were what was essential. But Rich knew he was no visionary like Kelly Johnson. Rich’s stealth project was extremely unique and gutsy. This was the case because he elevated the ambitious vision of a single individual to the fore of the project. But it was not his vision. He describes the project’s inception, writing:

I wish I could claim to have had a sudden two a.m. revelation that made me bolt upright in bed and shout “Eureka!”…The truth is that an exceptional thirty-six-year-old Skunk Works mathematician and radar specialist named Denys Overholser decided to drop by my office one April afternoon and presented me with the Rosetta Stone breakthrough for stealth technology…

Denys had discovered this nugget deep inside a long, dense technical paper on radar written by one of Russia’s leading experts and published in Moscow nine years earlier…As Denys admitted, the paper was so obtuse and impenetrable that only a nerd’s nerd would have waded through it all…As he explained it, [Pyotr] Ufimtsev had revisited a century-old set of formulas derived by Scottish physicist James Clerk Maxwell and later refined by the German electromagnetics expert Arnold Johannes Sommerfeld. These calculations predicted the manner in which a given geometric configuration would reflect electromagnetic radiation. Ufimtsev had taken this early work a step further. “Ben, this guy has shown us how to accurately calculate radar cross sections across the surface of the wing and at the edge of the wing and put together these two calculations for an accurate total.”

Rich then explains how radar technology works and how existing planes often show up on radar as large as houses. He then continues:

We desperately needed new answers, and Ufimtsev had provided us with an “industrial-strength” theory that now made it possible to accurately calculate the lowest possible radar cross section and achieve levels of stealthiness never before imagined. “Ufimtsev has shown us how to create computer software to accurately calculate the radar cross section of a given configuration, as long as it’s in two dimensions,” Denys told me. “We can break down an airplane into thousands of flat triangular shapes, add up their individual radar signatures, and get a precise total of the radar cross section.”

So, Rich had Overholser and a labmate put together a program to take advantage of this insight and design a model plane out of 2D surfaces that had a minuscule radar cross section — computing power was not strong enough to build the model out of 3D surfaces. The final design’s radar cross section appeared to be smaller than existing planes by several orders of magnitude. Instead of the plane being the size of a home or a barn, the question was now, “Which of these ball bearings in Ben Rich’s pocket would show up bigger than the plane, and which wouldn’t?”

Many Skunk Works veterans thought the design was horrible. It looked like an ugly, crude, origami rendition of a plane. The more a lab member knew about aerodynamics, the uglier the model looked. Many Skunk Works old-timers pushed back. Ben Rich wanted his team — which specialized in building planes that flew exceptionally — to focus on building the least aerodynamically sound plane they’d ever seen. Additionally, many in DC thought Rich must be mistaken about what the true radar cross section would actually be. Kelly Johnson held both of these gripes against Rich. He referred to the radar cross section computations as “theoretical claptrap.” Rich won a rare quarter off Johnson when the wooden model did register a vanishingly small radar cross section. Johnson, still skeptical, flipped him the quarter and said, “Don’t spend it until you see the damned thing fly.”

Luckily, it did. It was a tall task. But with the help of recent developments in computing power, the plane was able to fly using constant computer adjustments to help counteract issues with its chaotic aerodynamics. Kelly was wrong to not believe in the plane. But he was visionary in building a lab that elevated individual design visions and a single, strong boss above the gripes of a retired consultant like him. And he was right to entrust the lab to someone who was a terrier in assessing a technical idea’s merits, even if the results looked very unorthodox.

The plane broke no speed records, but that didn’t matter much since it showed up no bigger than a bird on enemy radar screens. The stealth bomber project is a strong data point in favor of the idea that minimizing bureaucracy and entrusting a bright individual with power was more essential to Skunk Works’s success than any stroke of genius from Kelly Johnson. Kelly Johnson was a genius. But, more importantly, Skunk Works was a place built to facilitate genius ideas into built reality.

Kelly built an operation that left enough leeway for the common sense and practical wisdom of his team to take over. From one project to the next, many of the operational specifics changed. Johnson noted:

The theory of the Skunk Works is to learn how to do things quickly and cheaply and to tailor the systems to the degree of risk. There is no one good way to build all airplanes.

At the start of each project, the Skunk Works leader was free to sit down with his best people and plan out exactly how to build the new plane in the way that made the most sense. From one plane to the next, the process might look extremely different. To make this work, the team needed to be kept small enough that new operating procedures could be implemented without chaos ensuing. And they needed to do this without countless pre-arranged meetings. That wasn’t the Skunk Works way.

Principle #2: Keep the Team Viciously Small

Skunk Works’ small team emphasis allowed team members to seamlessly stay updated on the entirety of the project. It kept things small enough that even the machinists could contribute knowledge to the early design stages of the project, not just have ideas dictated down to them.

Johnson emphasized the high-level importance of maintaining a small team, writing:

Working with a limited number of especially capable and responsible people is another requirement. Reducing reports and other paperwork to a minimum, and including the entire force in the project, stage by stage, for an overall high morale are other basics. With small groups of good people you can work quickly and keep close control over every aspect of the project.

Ben Rich’s book describes precisely how this small team emphasis made a difference in Skunk Works’ processes. He writes:

We had our own unique method for building an airplane. Our organizational chart consisted of an engineering branch, a manufacturing branch, an inspection and quality assurance branch, and a flight testing branch. Engineering designed and developed the Have Blue [stealth prototype] aircraft and turned it over to the shop to build. Our engineers were expected on the shop floor the moment their blueprints were approved. Designers lived with their designs through fabrication, assembly, and testing. Engineers couldn’t just throw their drawings at the shop people on a take-it-or-leave-it basis and walk away.

Our senior shop people were tough, experienced SOBs and not shy about confronting a designer on a particular drawing and letting him know why it wouldn’t work. Our designers spent at least a third of their day right on the shop floor; at the same time, there were usually two or three shop workers up in the design room conferring on a particular problem. That was how we kept everybody involved and integrated on a project. My weights man talked to my structures man, and my structures man talked to my designer, and my designer conferred with my flight test guy, and they all sat two feet apart, conferring and kibitzing every step of the way. We trusted our people and gave them the kind of authority that was unique in aerospace manufacturing. Above all, I didn’t second-guess them.

Even the procurement staff were a part of this. Johnson described why it was vital to keep them in the loop in a similar fashion, writing:

We maintain a very close liaison from me to the designer, to the purchasing agent — so that he understands the urgency in acquiring materials; to the tooling people; and to the people who actually will build any part of the plane.

For those who read the previous FreakTakes piece on ARPAnet contractor BBN, this approach should sound remarkably familiar. In it, I shared the following quote from ARPAnet project leader Frank Heart, reflecting on how he managed his small team of bright engineers:

I think mostly I tend to believe important things get done by small groups of people who all know all about the whole project. That is, in those days all the software people knew something about hardware, and all the hardware people programmed. It wasn't a group of unconnected people. It was a set of people who all knew a lot about the whole project. I consider that pretty important in anything very big. So I suppose if you call it a management style, that would be something I'd state. I think also that they were a very, very unusually talented group. I think things tend to get done best by small groups of very, very good people — if you can possibly manage that. You can't always manage it. So if you again want to call it a management style, it is to get the very, very best people and in small numbers, so they can all know what they're all doing.

In that piece, I then expanded upon Heart’s point, writing:

Heart was extremely averse to the team growing so large that communication had to be done on paper. He preferred their style of “very, very frequent interaction on problems.” As Walden remembered, “I don’t remember such a thing as we had a weekly progress meeting. We probably were more in tune with the progress than that; we probably did it hourly.” Heart and the entire, small team did their best to stay technically involved with each other’s work. Heart, as a rule, insisted on technically understanding every bit of the project. The individual team members attempted to do the same, which they felt was quite easy to do. The team all had offices and work stations immediately in the vicinity of each other. As things came up, they’d simply go talk with the relevant individual or call an impromptu meeting.

BBN’s Frank Heart and Kelly Johnson shared extremely similar beliefs on how to manage engineering teams building extremely novel equipment.

The Skunk Works project teams were about ten times larger than BBN’s project teams, given the more industrial nature of the operation. But these were still very small teams by aviation standards. Kelly estimated that Skunk Works’ teams were only 10%-25% of the usual size when “compared to the so-called normal systems” of his day. Ben Rich, after some time at the Skunk Works, helped lead a staff of three on the Blackbird project. He was quick to note that “by Skunk Works standards that was almost an empire.”

Even an exceptionally difficult project like the SR-71 Blackbird did not balloon in size. That’s saying something. Nearly every material used and part designed for the airplane needed to be rethought due to the extreme cruising speeds (~2,000 mph) and altitude (~85,000 feet) the plane would reach. Kelly made it clear at the very beginning of the project that the team would need to “start from scratch as if we are building the first airplane, just like the Wright brothers.”

In the face of this overwhelming project, Johnson doubled down on his own people and systems. He did not resort to an inordinate increase in headcount. The following excerpt from Ben Rich confirms the still-modest scale of things on the massive project. Rich writes:

Developing this air-inlet control system was the most exhausting, difficult, and nerve-racking work of my professional life. The design phase took more than a year. I borrowed a few people from the main plant, but my little team and I did most of the work. In fact the entire Skunk Works design group for the Blackbird totaled seventy-five, which was amazing.

Writing his book in 1994, Ben Rich was acutely aware of how much the industry had changed. He closes the above quote with the following quip:

Nowadays, there would be more than twice that number just pushing papers around on any typical aerospace project.

Principle #3: Maintain Similar Expectations of Project Partners

Kelly’s belief in minimizing bureaucracy and keeping team size ruthlessly small did not stop with his own Skunk Works team. He was adamant that, if at all possible, projects should only be undertaken when a project’s funder could promise similar, strong project leadership. This was the second of his famous fourteen rules:

Strong but small project offices must be provided both by the military and industry.

Johnson likely felt there was little use in putting his team into overdrive just for a customer to hold up progress and changes every step of the way. So, he wanted quick “yes” or “no” decisions from funders as well.

Throughout his book, Johnson was quick to lavish praise on those individuals and areas of the armed services that, once they decided to move forward, were excellent project partners. When he had it his way, Johnson’s projects often had something like one project liaison and ~6 people whose opinions mattered in key decisions. Kelly provided many specific cases of what ideal relationships with funders looked like. Sharing one example, he writes:

Operating at its best on our Air Force programs, the Skunk Works could get an almost immediate decision on any problem. I could telephone Wright Field, Dayton, for example, talk to my counterpart who headed the small project office there for the military — and who was allowed to stay with it to conclusion — and get a decision that same morning. Now, that’s just not possible in standard operating procedure. It’s a difficult concept to sell for the first time, though, since it means abandoning the system.

This is what an ideal, standard case looks like. But it should also be noted that Rich and Johnson mention several cases of project liaisons going above and beyond in almost unbelievable ways. In one case, Rich and Johnson noted that an exceptional CIA project partner set up shell companies and a kind of false supply chain to to acquire a large quantity of titanium from the Soviet Union for an aircraft. But, of course, that level of proactivity from the funder was not the usual.

But working with a customer that knew how to make decisions quickly was essential. It was so essential that Ben Rich noted that Kelly had an unwritten fifteenth rule:

Starve before doing business with the damned Navy. They don’t know what in the hell they want and will drive you up a wall before they break either your heart or a more exposed part of your anatomy.

On one occasion, Ben Rich did not heed this warning and sought to build a stealth boat for the Navy. He planned to do it using the same design software that they were using to build the F-117 bomber. Pursuing this project was a decision Rich came to sorely regret.

The project actually broke multiple Kelly Johnson rules. But the first misstep was forced onto Ben Rich due to internal Lockheed politics. Lockheed’s Ocean Division was in financial trouble. Meanwhile business was booming at the Skunk Works even without the stealth ship contract. So, Rich handed off the development contract for the ship to Lockheed’s Ocean Division. But Rich also sent several core team members over to the division to help lead the project. The Skunk Works group group was led by Ugo Coty. Before sending Coty and co., Ben had told the Ocean Division executives that it would be essential to let the mathematically inclined team members have the final say over everything in the ship’s design.

To the executives, this was a seemingly off-the-wall way to build an aircraft or ship. But this is exactly what Rich had decided to do with the F-117 project. It was not at all the normal procedure, but it’s what made the most sense. The radar cross section of the ship could only be as low as the most visible of its parts. Even one periscope being slightly misdesigned could make the ship appear bigger than a barn. And the mathematics of radar cross sections was unintuitive to almost all of the ship engineers.

The Oceans Division proved incapable of following Ben’s advice. And the Navy brought so many cooks into the kitchen that it probably wouldn’t have mattered even if that had not happened. Rich describes the mess caused by the Lockheed and Navy bureaucracies on the project, writing:

“We need Ugo to keep those damned shipbuilders from going off on a tangent,” I told Roy Anderson. “This is one project where the method of shipbuilding is much less important than the stealth technology,” I told Roy. “They’ll want to sacrifice the stealth if it gets in the way of the ship’s performance, but Ugo will force them to stay focused. All Dr. Perry wants to prove out is the stealth. That’s key to this test. If the ship merely floats that’s good enough.” As it happened, my fears about the conflicting agendas between professional shipbuilders and experts on stealth technology, like Dr. Perry, were realized almost from the first day that the Ocean Division took over the project.

Ugo Coty did his best, but he ran into heavy weather. His original six-man operation quickly was shunted aside by eighty-five bureaucrats and paper-pushers running the program for the division. Then the Navy marched in, adding its supervision and bureaucracy into the mix with a fifty-man team of overseers, who stood around or sat around creating reams of unread paperwork. No ship ever went to sea — not even a top-secret prototype — without intensive naval supervision to ensure that all ironclad naval rules and regulations were strictly enforced before the keel was ever laid.

“Where is the paint locker?” a Navy commander demanded of Ugo, rattling the blueprint plans. Since the days of John Paul Jones, every naval ship afloat has a damned paint locker on board. Sea Shadow would definitely not be the only exception since the Revolutionary War.

To hammer home the point about the “Navy way,” Rich goes on to tell a story of Skunk Works modifying its Jetstar for the Navy under Kelly Johnson. The original Jetstar project itself took 55 engineers eight months. The Navy project, which was a little more complicated, took 27 months. Rich noted that “one hint as to the reason why” is that Jetstar’s mockup conference had six visitors on hand. For a similar conference, the Navy sent 300. To him, that was the “Navy way.”

Before moving on, it is important to note that Johnson also had a strong preference for only working with contractors who didn’t require increased bureaucracy to be added to the project. In an ideal case, this would look like the U-2 project in which key technical project problems were offloaded to Pratt & Whitney and Shell researchers to work their way through. In another case, Skunk Works took a crucial tire problem to B.F. Goodrich “which developed a special rubber mixed with aluminum particles that gave our wheels a distinctive silver color and provided radiant cooling.” This was just the solution Skunk Works needed. However, in more standard cases what this means is simply sticking with trustworthy suppliers and extending them the same respect Skunk Works liked to have itself. As Kelly Johnson put it:

Suppliers and others associated with a project must be extended the same kind of rules and permissions that are given us for the entire program. This cuts red tape and costs and allows all participants to concentrate on the product instead of a system. It is so simple.

This was more feasible in some cases than in others. But these were the relationships Johnson strove to maintain.

While all of this sounds quite simple, replicating Skunk Works has proven anything but simple for most would-be copycats.

Why So Many Skunk Works Copycats Failed

In the early 1970s, shortly before Kelly Johnson’s retirement, Northrop made an offer to poach Ben Rich — then one of Johnson’s top lieutenants. Ben Rich recalls Kelly’s frank warning when he confronted Johnson with his offer, writing:

I laid out Northrop’s offer, and he closed his eyes and solemnly shook his head. “Goddam it, Ben, I don’t believe a word that guy said to you. I’ll bet my ranch against Northrop starting its own Skunk Works. Companies give it lip service because we’ve been so successful running ours. The bottom line is that most managements don’t trust the idea of an independent operation, where they hardly know what in hell is going on and are kept in the dark because of security. Don’t kid yourself, a few among our own people resent the hell out of me and our independence. And even those in aerospace who respect our work know damned well that the fewer people working on a project, the less profit from big government contracts and cost overruns. And keeping things small cuts down on raises and promotions. Hell, in the main plant they give raises on the basis of the more people being supervised; I give raises to the guy who supervises least. That means he’s doing more and taking more responsibility. But most executives don’t think like that at all. Northrop’s senior guys are no different from all of the rest in this business: they’re all empire builders, because that’s how they’ve been trained and conditioned. Those guys are all experts at covering their asses by taking votes on what to do next. They’ll never sit still for a secret operation that cuts them out entirely. Control is the name of the game and if a Skunk Works really operates right, control is exactly what they won’t get…Mark my words, you’ll be reporting to a dozen management types and they won’t let you out of their sight for one minute.”

Ben Rich didn’t leave. And the Northrop operation went on to operate essentially how Kelly said it would. This prediction was not Kelly Johnson demonstrating any kind of remarkable prediction abilities. He was simply recounting what he continually saw happen when companies said they wanted a Skunk Works.

The sorts of problems Kelly figured would happen to Northrop and those that had happened in the Naval stealth project arose repeatedly. The following excerpt describes similar problems that afflicted the Army’s Cheyenne rigid-rotor helicopter program. Kelly himself attempted to train Jack Real and six of his top supervisors who would be running the program. Kelly had them shadow his Skunk Works operation for six months. But his efforts were in vain. Kelly writes:

Real and his team began with great enthusiasm to apply our operating methods to meet the Army’s design specifications. But within six months, the satellite Skunk Works had a purchasing department larger than my entire engineering department working on seven projects. They had become buried in the usual paperwork already.

Despite the best of intentions, the Army had at the time ten different test centers and bases involved in the procurement of new weapon systems. And when you have that many representatives involved in design and development, with no single person in charge to represent the customer, the Skunk Works concept cannot work.

It is absolutely imperative that the customer have a small, highly-concentrated project office as a counterpart to the Skunk Works manager and his team. It is not a concept easily adopted after years of working within the system. There has to be an all-out commitment, or the method will not work.

When there was not an “all-out commitment” to work within the basic system, the differences in results were startling.

At the time the Cheyenne contract was cancelled, 145 Army personnel were involved in the program. In contrast, the total at the Skunk Works for both CIA and Air Force representatives in our U-2 and SR-71 programs did not exceed six people.

It should also be noted that Kelly himself was able to come into at least one operation outside the Skunk Works and implement his system successfully. At one point, he was called into the Lockheed Missiles and Space Company to help turn around an ailing project. Kelly came in and rolled out the same playbook I’ve described to you in this piece. He describes his work on the project, writing:

When Skunk Works principles really are applied, they work. An example of their successful application was development of the Agena-D launch vehicle…The satellite that was to become this country’s workhorse in space was in trouble in terms of design and cost but especially in reliability, which stood at an incredibly low 13.6 percent. I was drafted, in effect, to go up to the Lockheed Missiles and Space Company and fix it. We set up a Skunk Works operation with the company’s design project engineer, Fred O’Green, as head…

It proved again our axiom: If you have a good man and let him go, he’ll really perform. In terms of today’s world, that axiom should apply to women as well. When I first reviewed the Agena project, I discovered that 1,206 people were employed in quality control alone, achieving only that 13 percent reliability factor! It should have been the world’s most reliable vehicle just using the inspection department. That was enough people to design and build the thing.

At the Baird Atomic Company, which made the vehicle’s horizon sensor, Lockheed had 40 people inspecting, coordinating, and reporting. Yet Baird had only 35 people building the instrument. We resolved that situation by returning responsibility for the product to the vendor. For example, I telephoned Walter Baird personally since he and I had worked together on a number of other Skunk Works projects. He immediately agreed to pick up his end of the log.

Kelly estimated that the changes saved the government at least $50 million in costs and led to completing in nine months what had been scheduled for 18. 350 drawings had been made instead of the projected 3,900. And quality control personnel had been “slashed” to 69. The operation was now able to output design drawings in a day rather than a month.

Ben Rich had this to say about Kelly’s turnaround of the Agena operation:

The president was already spending a billion dollars in covert funds on the Agena rocket that would boost our first spy satellite into orbit. Bissell was in charge of that program, too, and the first twelve test firings had all been failures. Lockheed’s Missiles and Space Company in Sunnyvale, California, had that contract, and Bissell asked Kelly to evaluate and reorganize their operation. Kelly set up a mini Skunk Works and, coincidentally or not, the thirteenth test shot was a success.

Many claim to have copied Skunk Works in its heyday. But few ever truly did.

A Different World

In many ways, times have changed. Generals can no longer have an aircraft development contract written up and sent out in a few hours. Lowest bidder subcontracts might be required before an outfit like Skunk Works has a new kind of fuel made. Many think the management of public companies is much more short-sighted than it was in the 1960s. The list goes on.

The writing of Ben Rich himself emphatically demonstrates how much these changes haunted him. Throughout his book, he routinely mentions things like nightmares regarding corporate bean counters. In one breath he tells us of the best idea his Skunk Works would ever produce. In the next, he mentions how the Lockheed bureaucracy might threaten the idea being realized. He writes:

I wish I could claim to have had a sudden two a.m. revelation that made me bolt upright in bed and shout “Eureka!” But most of my dreams involved being chased through a maze of blind alleys by a horde of hostile accountants wielding axes and pitchforks.

And these were not small, nagging fears. Ben Rich’s tenure as head of Skunk Works saw the Lockheed and DC bureaucracies creeping in from both sides. At one point, he lobbied Lockheed management to take on a relatively risky production contract for the F-117 sooner rather than later. The idea was only green-lit with much skepticism. Rich writes:

“They’ll want at least one hundred bombers, and we’ll be looking at tens of billions in business. So what’s this risk compared to what we can gain later on? Peanuts.”

It was not a very happy meeting, and the conclusion reached was reluctant and not unanimous. The corporate bean counters insisted we install a fail-safe monitoring and review procedure that would sound the alarm the moment we fell behind or hit any snags. “Above all, no nasty surprises, Ben,” Larry Kitchen warned me.”

The Gross brothers and Hall Hibbard were gone. Kelly Johnson had been a living legend who routinely turned down Lockheed’s CEO position. His freedom and insistence on the same for his Skunk Works was tolerated by subsequent generations of executives throughout his career. But by the end of his career, many in DC and Burbank had little patience for him. They would not be so lenient with his successor.

Ben Rich’s Skunk Works — despite proving itself with its inspired F-117 project design — was being watched. He recounted:

An independent engineering review team, composed entirely of civil servants from Wright Field in Ohio, flew to Burbank to inspect and evaluate our entire program. They had nothing but praise for our effort and progress, but I was extremely put out by their visit. Never before in the entire history of the Skunk Works had we been so closely supervised and directed by the customer. “Why in hell do we have to prove to a government team that we knew what we were doing?” I argued in vain to Jack Twigg, our assigned Air Force program manager. This was an insult to our cherished way of doing things. But all of us sensed that the old Skunk Works valued independence was doomed to become a nostalgic memory of yesteryear, like a dime cup of coffee.”

He was largely right.

Johnson shared an anecdote in his 1985 book, noting just how much the defense procurement world had changed in a few short decades. He noted:

Our Air Corps project officer for the XP-38 was a pilot, a young lieutenant, Benjamin S. Kelsey. He was excellent. In those days (1938) a project officer with that rank had more authority than many four-star generals do today (1985). If we asked Ben for a decision, we got it — on the spot.

Conclusion

Times have surely changed in defense procurement. But the reader’s takeaway should not be that Skunk Works’s management playbook was great, but no longer feasible. It is true that many pools of government capital that used to be risk-tolerant and trusting of individual visions are the opposite nowadays. However, there still exist fantastic funders with a deep belief in bright people with ambitious visions. And many of these funders require only common-sense oversight. With capital from phenomenal funders like these, the Skunk Works management playbook can still prove exceptionally effective. Kelly Johnson’s rules regarding limiting bureaucracy, keeping his team ruthlessly small, and doing whatever possible to allow individual genius to flourish helped create a historically great lab.

The heart of what made the lab exceptional is also strikingly similar to what made BBN and CMU’s autonomous vehicle units so exceptional. All three maintained operating structures that were operated like small companies and were aggressively focused on novelty. The CMU group found a way to make a novelty-focused university group operate in a more firm-like fashion. BBN set up a private firm staffed with MIT professors that could only be rivaled in its novelty focus by the most daring of university labs. In Skunk Works, Kelly Johnson found a way to run a research outfit within a corporate bureaucracy like a small firm. He also insisted that the lab spend close to all of its time building planes that were novel and ambitious. When he stepped down he chose Ben Rich as his successor largely because he knew Ben wouldn’t play it safe with the lab. Without that ambition, the lab would be effective but not ambitious. It wouldn’t be Skunk Works.

Ex-Lockheed CEO Roy Anderson recalls Kelly Johnson saying the following when recommending Rich as successor:

Roy, I raised Ben in my own image. He loves the cutting edge as much as I do, but he knows the value of a buck and he’s as practical as a goddam screwdriver.

To Kelly Johnson, you likely couldn’t run a true Skunk Works without both.

Thanks for reading:)

Subscribe now

Share

Pattern Language Tags:

• Utilizing a contractor made up of individuals with research-style goals and training working within a ‘firm’ structure.

• Facilitating tool/hardware improvements in a key technology area far from its suspected theoretical frontier.

• Selecting multiple industry contractors for a trial period before choosing the final contractor. (F-117)

This piece is a part of a FreakTakes series. The goal is to put together a series of administrative histories on specific DARPA projects just as I have done for many industrial R&D labs and other research orgs on FreakTakes. The goal — once I have covered ~20-30 projects — is to put together a larger ‘ARPA Playbook’ which helps individuals such as PMs in ARPA-like orgs navigate the growing catalog of pieces in a way that helps them find what they need to make the best decisions possible. In service of that, I am including in each post a bulleted list of ‘pattern language tags’ that encompass some categories of DARPA project strategies that describe the approaches contained in the piece — which will later be used to organize the ARPA Playbook document. These tags and the piece itself should all be considered in draft form until around the Spring of 2024. In the meantime, please feel free to reach out to me on Twitter or email (egillia3 | at | alumni | dot | stanford | dot | edu) to recommend additions/changes to the tags or the pieces. Also, if you have any ideas for projects from ARPA history — good, bad, or complicated — that would be interesting for me to dive into, please feel free to share them!

General Links:

• More Than My Share of It All by Kelly Johnson

• Skunk Works: A Personal Memoir of My Years of Lockheed by Ben Rich and Leo Janos

As a preface to the interview, I wrote the following ‘Introduction’ section to highlight why CMU is a singularly important department in autonomous vehicle history. I leave the reader with three options as to how to consume the rest of the piece after reading the introduction. For the bookworms, I have provided an abridged transcript of my meeting with Chuck Thorpe — which picks up after the introduction. For the audiophiles, the full audio of the interview is available — both on Substack and on Spotify. And for the YouTube afficionados, I share a YouTube link of the Zoom interview.

Subscribe now

Share

Introduction

The first piece in the all-time great DARPA contractors trilogy explored the importance of BBN’s unique structure, embodying the “middle ground between academia and the commercial world.” BBN did so well in this role that it earned itself the monicker “Third Great University of Cambridge.” BBN, as a firm, demonstrated what was possible if a firm staffed itself with the very best engineering researchers and cared deeply about technical novelty more than profit maximizing. The flipside of the coin is CMU and its AI work in the later-1900s. Much of CMU’s AI work in this period answers the question, “What if an academic engineering department made its comparative advantage novel systems contracts?” CMU was an academic department eagerly embracing a style of work more common to the defense primes in the DARPA ecosystem.

In DARPA’s Autonomous Land Vehicle project — covered in the ALV section of the ARPA playbook — CMU stood out as the lone contractor involved who was excited to take on both the burden of novelty and the burden of systems integration work. The academics and the project’s main commercial firm had very different incentives when it came to these issues. As I wrote in the earlier ALV piece:

The contractors who found themselves working on the reasoning system vs. the vision system had very different interests and incentives. Most of the reasoning contractors were commercial firms and had the incentive to fight for a broader scope of work and more tasks getting assigned to the reasoning system because that meant better financial returns to the company. Most of the vision contractors, such as the University of Maryland, were academics and largely content to let that happen…if it meant they got to focus primarily on specific vision sub-tasks that were more in line with the size and scope of projects academics often undertook. For example, the private companies felt that building up much of the systems mapping capabilities, knowledge base, and systems-level work should fall to them.

CMU and one of its project leaders — Chuck Thorpe — seemed to be the strong voice of dissent on many of these matters from the academic community. Ever since CMU had become a major contractor in DARPA’s Image Understanding work, it had taken a rather systems-level approach to attacking vision problems — working with civil and mechanical engineering professors like Red Whittaker to implement their vision work in robotic vehicles — rather than primarily focusing on the component level. The CMU team far preferred this approach to things like paper studies — in spite of the increased difficulty.

As the ALV project wore on, Martin Marietta — the lead systems contractor for the ALV — gradually succumbed to its ‘demo or die’ pressure. As the frequency of demos increased, Martin became much more concerned with technically hitting the benchmarks DARPA specified than pushing forward truly ambitious technological changes. Additionally, many of the academics focusing on vision research concerned themselves more with using their DARPA research funds to improve their performance in somewhat simplified environments that helped them write research papers than to do research that would directly plug into the ALV. With this equilibrium becoming clear, Ronald Ohlander — a former student of Raj Reddy at CMU and the DARPA PM responsible for funding vision research — designated CMU as the systems integrator for his DARPA SCVision portfolio. The Martin Marietta team clearly did not care enough about integrating and testing cutting-edge algorithms. So, he tapped CMU and gave them the funds to properly integrate insights from the research ecosystem.

In the prior piece, I described the situation as follows:

CMU had already, unlike most of the component contractors in the SCVision portfolio, been testing its component technology on its own autonomous vehicle: the Terregator. CMU’s used the Terregator — for often slow and halting runs — to test the algorithms developed for its road-following work. The Terregator proved extremely useful to this research, but its use had also helped the CMU team learn enough to outgrow the machine.

In CMU’s ongoing efforts to implement sensor information from a video camera, a laser scanner, and sonar into one workable vehicle, the CMU researchers had come to realize that they needed funding for a new vehicle. CMU needed a vehicle that could not just accommodate the growing amount of sensor equipment, but also one that could carry out its tests with computers and grad students either on board or able to travel alongside the vehicle. This would allow bugs to be fixed and iterations between ideas to happen much faster. The vehicle that the CMU computer vision and robotics community was casting about trying to get funding for would come to be known as the NavLab. Fortunately, what they were seeking money for seemed to be exactly what Ohlander felt the SCVision portfolio needed. The funding for the NavLab began around early 1986. The funding set aside for the first two vehicles was $1.2 million, and it was estimated that any additional NavLab vehicles would cost around $265,000.

As Martin was somewhat frantically taken up with its demo schedule, CMU, with its longer time windows, was free to focus on untested technology as well as technology that ran quite slowly but seemed promising in terms of building more accurate models of its environment. As the CMU people saw it, if anything proved promising they could upgrade the computing machine used to operate that piece of the system later on…the vision researchers likely figured that a piece of hardware powerful enough would likely exist soon to help push the idea along — even if not in the next year. Lacking the hard metrics of yearly demos with speed requirements loosely matching how fast the vehicle would need to go in the field, the CMU team could afford to think more along the lines of “how do we build a machine that is as accurate as possible, even if it moves really slowly today?”

As CMU took up the mantle of building SCVision’s true test bed vehicles, it also took over the role of SCVision portfolio’s true “integrator.”

Things continued like this for a while. Then, after a successful ALV demo in 1987, Martin’s role as the prime DARPA autonomous vehicle contractor suddenly ended. This abrupt shift happened after a panel of DARPA officials and technology base researchers met to discuss phase two of the program. Roland and Shiman wrote in their book on this era of DARPA computing:

Takeo Kanade of CMU, while lauding Martin’s efforts, criticized the program as “too much demo-driven.” The demonstration requirements were independent of the actual state-of-the-art in the technology base, he argued. “Instead of integrating the technologies developed in the SC tech base, a large portion of Martin Marietta’s effort is spent ‘shopping’ for existing techniques which can be put together just for the sake of a demonstration.” Based on the recommendations of the panel, DARPA quietly abandoned the milestones and ended the ALV’s development program.

The CMU researchers and the NavLab — which were remarkably cheap compared to Martin Marietta — continued to be funded. In this way, CMU took its place as DARPA’s top systems integration contractor for driverless vehicles. And it would stay on top of the heap for several decades.

The CMU driverless vehicles were responsible for many — some might even say most — of the practical driverless vehicle breakthroughs in the field's early years. Much of CMU's earliest AI work on these vehicles revolved around the use of things like radar and laser sensors, path planning, and finding ways to integrate inputs from different models into a single set of instructions. But, in the late 1980s, CMU's work abruptly became quite recognizable to the modern eye. In 1988, then-CMU grad student Dean Pomerleau successfully integrated the first neural net-based steering system into a vehicle — his ALVINN system. In 1995, Dean and fellow grad student Todd Jochem drove the successor to Dean’s ALVINN — the more complex RALPH — across the country on their “No Hands Across America” tour. The RALPH system employed neural nets but also put much more effort into model building and sensor processing. Holding Dean and Todd, the upgraded NavLab was able to autonomously drive 98.5% of the way on its successful cross-country journey. In a sense, much of CMU’s work culminated in the CMU vehicles' performances in the second DARPA Grand Challenge, held in 2005. Many only remember that Stanford's team won this landmark challenge. But CMU's two vehicles finished a close second and third to Stanford’s vehicle.

At the helm of Stanford’s team was Sebastian Thrun. Thrun had, the year before, left his CMU research position — which he had held for almost ten years — before joining Stanford and leading their team. In the early years of the field, an astonishing proportion of the best researchers were either trained at CMU or were long-time professors there. In a 2010 oral history, Thorpe described Sebastian Thrun and many of the early Google driverless car team’s CMU ties, saying:

Now, it’s the Google group, and we’re very grateful to Google, but we think of it as the Carnegie Mellon West group. Sebastian, of course, used to be here, and then went to Stanford and Google. Chris [Urmson] is a Carnegie Mellon person on leave. James Kuffner is a Carnegie Mellon faculty member on leave. If you go down through the list, they – everybody except for one on that group either is a Carnegie Mellon person or has a Carnegie Mellon heritage, so that’s our boys. [laughs] And we’re delighted that they’re off with Google, and Google’s given them the researchers to do really cool stuff.

Carnegie Mellon has its fingerprints all over the breakthroughs and researchers from the early decades of autonomous vehicles. It would be hard to argue that CMU’s efforts didn’t move forward the timelines of this field by some number of years. It also seems clear, looking at the history, that the incentives and management structures CMU used to manage projects like this played a major role in setting the university apart.

The university clearly had a comparative advantage in this style of work. So, I decided to ask Chuck Thorpe some questions about how exactly CMU did what it did.

Chuck Thorpe Interview

In the following transcript, I made some cuts to make reading the text version of the interview more efficient. The audio and video files contain the interview in its entirety — which is about 15% longer.

Spotify podcast link

Finding his way to CMU and the early years of the Robotics Institute

Thorpe opened the interview by detailing how exactly he found his way to CMU as a grad student and how CMU’s approach to computing research set it apart from many other universities at the time.

Thorpe: So, I showed up wanting to work on robots. They gave me a book and the robots were all these robot arms and I thought, “Well, that's not what I want to do.” I want to do mobile robots. The reason I came to Carnegie is I was backpacking with my professors from North Park.

One of them said, “What do you want to do?” I said, “Well, artificial intelligence.” “Oh. You ought to go to Carnegie Mellon. Herb Simon just won the Nobel Prize for inventing AI.” I'll tell you the truth. I had to look up Pittsburgh on a map. I had to go to the library and find out what Carnegie Mellon was. And Carnegie Mellon in the seventies was reinventing itself.

It was a very good Western Pennsylvania engineering school that looked at the demographics of Western Pennsylvania and said, “We're going to be a very small, very good Western Pennsylvania engineering school,” and decided that they needed to go national. Now “national” meant that in 1972, once a year, they would go to Chicago to try to recruit students. And every other year they would go to DC.

So, Dick Cyert, the president, bet his presidency on turning Carnegie into a national brand and really expanded it and grew it. He also, Cyert, was a professor of business and his doctrine was comparative advantage. Find out where you can be one of the top two or three in the world and go for it — and don't try to compete broadly.

And a mid-sized university like Carnegie, that makes a lot of sense. So Raj Reddy was talking with Cyert and convinced Cyert that it was his idea, Cyert's, not Raj's, that we could be top two or three in robotics. And Raj said, “But I'm not interested in playing catch-up small ball. If we're going to do it, let's go big.”

So Cyert asked Raj to run the Institute. He got a million-dollar grant from the Navy and he got a million bucks from Westinghouse. And that was the origins of the Robotics Institute.

Gilliam: And how important was it for him to get a big industry contract and a big public sector contract? Was that specifically the goal? To make sure they had one from each?

Thorpe: Specifically the goal was to do both, really crack into ONR (Office of Naval Research) and then into DARPA because they had a lot of money to do very interesting things. But, also, Carnegie has these deep roots in Western Pennsylvania. And if we could work with Western Pennsylvania industry and really help Westinghouse succeed that was a win all the way around.

This was at a time when the steel industry was visibly dying and trying to figure out what was going to save Western Pennsylvania. They thought it was Westinghouse. It turned out that was not completely the story. Westinghouse had its own financial problems. But that notion of trying to see what we could do locally, what we could do with industry, as well as tapping into the US Government was a big deal.

I, then, interjected to provide some historical context and ask a question.

Gilliam: It's interesting to hear you talk about the comparative advantage, because even Karl Compton, who ran the Princeton physics department, was MIT President for a while…he, at one point, wrote a letter in Science in the 1940s. Essentially, he was trying to be polite as he wrote it, but he said (roughly), “We see the talent going into some of the industrial R& D labs from the physics departments. We feel like, to the man, we have higher average talent, but we are not really touching what they're outputting.”

He thought that the firm structure and the concept of comparative advantage, organizing teams in a way that made sense, is what departments should do. It's what made sense. The concept of 20 different budgets [one for each professor] and 20 different fiefdoms [lab groups] to do small research was silly. It made no sense. It's for pure theoretician-type people. But he later became an MIT president and could not succeed at that.

But it seems like Carnegie in running a lot of this work — it seems like Alan Newell was calling it project style work, the Raj Reddy style of work — it seems like Carnegie succeeded in a lot of that ethos. Would you say that's fair?

Thorpe: So, a lot of this starts with Herb Simon saying, “What can we do really well?”

Herb was a founder of the business school, was a founder of the computer science department, was a founder of cognitive science (the psychology department). Herb then brought in Newell. And the two of them saying, “How do people think and how can we encode that in a computer?”…that sort of set the framework for so much of what has made Carnegie successful.

That whole AI and cognitive psych paradigm…if you look at the growth of the computer science department and then school, there's very good theoretical computer science, but the rest of it is all “AI+”. Yeah, AI + linguistics and you get machine translation. AI + learning and you get computational learning. AI + human interaction and now you get the Human Computer Interaction Center. AI + mechanical engineering and you get robotics.

So, it was the combination of leadership from people like Cyert saying, “I'm going to invest a little bit of money here.” And the intellectual leadership of Simon and Newell saying, “This is going to be a growth area.” Simon and Newell, in turn, hired Raj Reddy, poached him from Stanford. That group of them hired [Takeo] Kanade, poached him from Kyoto. And that sort of set us off and running.

The importance of Raj Reddy, what it means to be ‘project-oriented’, and how CMU utilized its grad students

Gilliam: Could I read you an excerpt from Alan Newell's oral history where he talks about Carnegie Mellon's project-oriented style and Raj Reddy?

Because he uses some terms that I'm sure you're going to hear and you're going to go, “I know exactly what he means!” But to me…I knew what I thought it meant — or maybe it's what I wanted it to mean — so I'd love to run it by you. So I'll read you two remarks and then I'll lead you in with the question.

Alright, so the first remark is:

NORBERG [Interviewer] : You had the money along with Green and Perlis [talking about an extra injection of DARPA funds]. What did you people decide to do with it?

NEWELL: I don't know... support people... just spend it. We didn't decide to do anything with it. One of the features of this environment was that it was decidedly un-entrepreneurial. That seems, in one respect, like a contradiction in terms, but we never took these funds and decided we were going to go out and do big things with these funds. That was an attitude typified the operation around here up until Raj [Reddy] shows up.

And remark two is in reference to the 1970s with DARPA. There were a lot of changing political tides, as you’re aware, and they’re talking about the changing goals of DARPA when DARPA is giving out funds. Newell says:

Newell: ...And so, therefore, we have been pushed towards being applied. A response I have, which is not an attempt to change that, that is exactly what's happened. CMU has gone along with that in spades. We started going along with that in the 1970s. I can remember having conversations with Howard Lackler as we were sitting there under the fixed funds with inflation and still trying to get an extra PDP-10. We were allowing that environment to shape us up so that we were not in fact anything like? And we became project-oriented all in that period. We were not project-oriented in the 1960s at all.

…
Newell:..The corresponding thing is the cats that want to play that game -- the Raj Reddys, the Canatis, and so forth, the Coons -- [find] the game they play is how to do that and do basic science at the same time within the same budget.

So I guess my initial question with that is, as Newell’s talking about ‘project-oriented,’ what does it mean to be project-oriented? How does that differ from business-as-usual AI research on a DARPA grant, the way you might see in a Marvin Minsky lab or something like that?

Thorpe: Oh, so Minsky is a very interesting contrast. Minsky wrote this book, The Society of Mind, where you could read…it was, I don't know, 50 two-page chapters. And you could read it in any order and made just as much sense in any order as in any other order.

That's kind of the MIT notion [presumably Minsky’s MIT AI group] of how ‘mind’ works, and their notion of how research works. It’s that you've got 50 independent projects and, together, something may emerge from that.

Carnegie shows up with people like Raj, who says, “I want to conquer speech. I'm going to put together a big team, and these people are going to be working on the phoneme level, and these people are going to be working on the lexeme level, and these people are going to be working on the syllable level, and these people are going to be working on the word level, and these people are going to be working on the sentence level. And there's going to be an architecture which connects them up and down. And we're going to have to have specialized hardware for all of this to work on.”

And so, Raj says, “Let's build this, this project.” And it was a project that had 16 different dimensions, all of which came together. Now, Raj was also flexible enough to say…when somebody showed up and said, “You know, I've got this idea for a completely different method. Let's not bother with all of the structure. Let's just have a little system where everything sort of learns and it figures out all of those mechanisms.” So that was Hearsay (demo), and that was Harpy (demo), and that was the two different systems.

That competition between big structured AI and machine learning, data-intensive AI has been going on at Carnegie for the last 50 years.

Gilliam: To get very in the weeds here…you talk about putting together a big team. And there's a lot of grad students involved in that. And it seems like on the one hand, ‘project-oriented’…it almost sounds like more of a firm structure than an academic team. A lot of the grad students are the employees. But there seems to be some kind of balance where the grad students are the employees — for example, with the ALV or NavLab where you had “on board grad students” — but also the grad students need a thesis.

With a thesis like yours [working to improve the operating speed of a sensor already in use on a test vehicle]…I could see how your thesis would have folded into a big team with that project's goals quite well. Because you were working on a specific sensor technology, and with Blackboard you could fold all that in. But then there's somebody like Dean Pomerleau [whose 1988 ALVINN system implemented neural net steering on CMU’s vehicle] on the other end of things. It's unclear how his project can mesh with everything.

Can you talk about how those management decisions are made? Because I guess the big goal of what I'm trying to do is to help people understand what the CMU approach is on a management level. So this would be great.

Thorpe: So let me back up and say a couple more things about the big CMU picture, and then I'll talk about Dean and how we organized the NavLab, in particular.

The big CMU picture was fundamentally changed when DARPA sent a telegram —this tells you how long ago it was — saying, “We've got a million bucks to spend. If you can send us a 350-word proposal by the end of the week, we'll send you a million bucks.” (~$5-$10 million today)

And that was the grant in basic research for computer science. And that grant lasted for, I don't know, 15 years or something like that. And even after that grant disappeared, that set the ethos of the computer science department. There's this one big chunk of money that supports all of the graduate students. Therefore, the graduate students are jointly sponsored by the entire department.

Do you know about Black Fridays?

Gilliam: No.

Thorpe: Twice a year, the entire computer science faculty sits down and talks about every graduate student. And the student has written a letter. And they [the faculty] read the letter. And they turn to the advisor and say, “What has your student been doing?” And if the advisor says, “Well, the student is making good progress and they did this and this and this,” and it matches what the letter has said, good. Then the student gets a letter saying satisfactory progress.

If the faculty member says, “You know, I've been trying to work with this guy and he's just so stubborn, he won't do what I tell him to do,” then the faculty write a letter saying, “You must, by next Black Friday, do this and this and this.”

If the student is co-advised and the one advisor says, “Well, I thought that Eric was working on it,” and Eric says, “Oh, I thought Chuck was working on it.” Now the faculty turn on the advisors and say, “You bozos, you have to get your act together and do a better job of advising this guy!” But it prevents one student from being abused by their advisor. If you're not happy with your advisor, because you're sponsored by the department you can change your advisors. But it also prevents one soft-touch advisor from saying, “Aw, poor student, he had a rough semester.” No, the entire department gets together and writes something. And there are these very carefully crafted letters of satisfactory progress, unsatisfactory progress, the dreaded ‘n minus one’ letter, and then being kicked out of the department.

So, the ethos is that we're all in this together; that we've got some joint resources; that the graduate students are joint resources; that we need to properly take care of these students and properly supervise them. So that gives the graduate student a large amount of flexibility. But this responsibility to not just their advisor, but to the entire department.

So that's great because students can find interesting projects. There is the marriage process where you walk in day one and each advisor says, “Here's what I'm up to. Here's how many students I'm looking for.” And then the students get to pick who they would like to work for. And the marriage process matches advisors who have interesting projects with students who are interested in working on them…

That's very different from almost any department that I've seen any place. Almost any place you're an indentured servant to the faculty member who happens to have graduate support. And if you don't get along with your advisor, you have the burden of seeing if there's somebody else who has money who will support you and seeing if you can switch advisors.

So that has set this attitude that we're all in this together. We'll have individual little projects, but we'll also have some of these big projects — like the speech projects — that a lot of people will work on.

Newell himself, at his height, had both his human-computer interaction/user interface stuff, but also his SOAR project — which was a pretty good-sized project with a number of graduate students working on it — trying to figure out how they could all fit together.

Newell said that, “You can do science in lots of different ways. One of the ways you can do it is just by doing something that's an order of magnitude bigger, or an order of magnitude faster. If you can process ten times the number of rules, then you're in a fundamentally different space, and you're doing something different, and the science is different. If you can run your computer ten times as fast, then the nature of the problems that you're dealing with changes in ways that you can't even predict.”

So, doing something 10 times bigger is great. So, he was very supportive of that kind of an attitude.

The Origins of the NavLab

Thorpe: Now, how did we get going on the NavLab? So I was sponsored on the original ONR project to do cool stuff. And one of the cool things that we did was to build Takeo’s direct drive robot arm. Another thing we did was to build this stereo vision for navigating across a room.

Side notion. We were sitting there working one day — graduate students, T-shirts, empty pizza boxes. And we heard the door open and we looked over and there was Raj and President Cyert arguing over who could open the door. And in walks, this Navy admiral in full dress whites with all of the…Nobody had told us the head of ONR was coming to visit…

Larry Matthews, who's now Head of Computer Vision at JPL, Hans Moravec, myself…with robots and pieces. And in walks the head of ONR to see what he's been spending money on. Like, “Guys, if you told us we would have at least cleaned up the lab and probably had a demo ready to run.”

Gilliam: Or put on your good T-shirt.

Thorpe: And put on my good T-shirt! But he was impressed. And many years later I ran into him and he remembered that visit. And so he was happy to see robots.

When I was finishing my thesis, we had a meeting and Takeo asked each of us, “What do you want to do next?” I said, you know, we've been doing a good job indoors. I want to go outdoors and really build driving cars. And when we had gone around and said what we wanted to do next, Takeo came back and said, “Good, because DARPA has this money for Strategic Computing. They're going to be building the world's best computers, and then they need ways of showing it off.”

And, being DARPA, they have to have a way for the Air Force, and a way for the Army, and a way for the Navy… (Thorpe then goes on to explain the Battle Management and Pilot’s Associate systems covered in a prior piece.)…And the Army wants to build a robot scout. “So Chuck, you're in luck because what you want to do next is exactly what DARPA is going to have money for. So let's write a proposal for it.”

We had just hired a guy named Duane Adams. Duane had been Deputy Director of one of the DARPA offices — IPTO. And he came to be the Associate Director of Robotics and to help us write proposals. So, Duane said, “Chuck, start writing.” And we wrote a proposal for what it would take to do road following. And that was the beginning of the NavLab project.

So we submitted the proposal, one for road following and one for the architecture of what the system would have to look like. I went back, finished my thesis, and defended my thesis. I thought I would take a couple weeks off to figure out what I wanted to do next. Raj came up and said, “Congratulations, you passed your thesis. There's a meeting in my office to talk about how we start the NavLab project.” And that was my gap between my thesis and the NavLab project, walking from the defense room into Raj's office to start this meeting, to start planning what we were going to do.

Departmental decisions that made big systems projects feasible

Gilliam: To stop you for a quick second to add to the queue of operational questions. So I have the question of 1) grad students as ‘employees vs. novel thesis generators.’ And now 2) in one of the DARPA year-end reports for the NAVLAB from CMU, I saw them refer to you as a ‘project manager.’ Which is interesting…that's not an academic designation. That sounds very industry. So I would love for you to talk a little about that, too. If I should read into that or what that means to you.

Thorpe: Shouldn't read much into that. But what you should read into is when Newell, Simon, and Reddy really got computer science going — and before that Perlis and Shaw, etc…They said, “If we're going to do big projects,” — and we did big projects like the Andrew Project, that turned into the Andrew File System and things like that. They said, “We need to have more faculty than we have courses to teach. We can support them on soft money, so we need to have research scientists who have full faculty privileges.”

So if you're at MIT and you're not tenure track, you're nobody. Carnegie said, “No, we want to have non-tenure track people, supported on soft money, but who have advising privileges and PI privileges and all of the privileges that faculty have. They just may or may not be teaching courses at this time.” So, unique to computer science, there's this whole research track which is fully equivalent to the tenure track. And that was my underlying title. When I defended my thesis in October, Raj said, “I can't make you a faculty offer. You're going to be a postdoc until July 1 and then I'll make you a research scientist.”

So I came up through the research ranks after having spent that eight months or so as a postdoc. And it's partly that business of trying to figure out what research scientists were that caused computer science to first break out of the Mellon College of Science and be a free-floating department. And then form themselves into a school so they could really have this parallel track.

‘Project manager,’ that was just sort of a title because that's what I was doing on the NavLab project. Raj took me aside. He said, “I don't care how many papers you write. I don't care how many awards you win. That vehicle has to move down the road. If you do that, you're good. I'll defend you. I'll support you, I'll promote you.”

Because I was a graduate student writing this proposal, I couldn't be a PI. So Takeo was a PI and I think Hans Moravec was a PI. And then as I became a faculty, I became a co-PI and then eventually the PI. But it was pretty clear that Takeo was interested in the project because it was a cool project. He was also interested if he could have a few people working on individual projects. But he didn't want to worry about scheduling stuff, and who got time on the machine, and writing reports. So, he was very happy to turn all of that over to me and let me run it the way that I wanted to run it.

Gilliam: And how was that? Can you describe your approach? Let's say, describe your approach to…somebody who knows how an academic lab works usually, but doesn't do anything like this. They output papers, they play the h-index game a little bit, etc.

Thorpe: Well, you pointed out a very key feature here, that the whole system has to work and that each student has to have something that they can claim as their own and that they can write up as their own thesis topic.

So, we got ready to put the system together and I said, “How are we going to see the road? Jill [Crismann], how do you want to do it?” Oh, well, she wanted to work on electrical engineering, signal processing, color classification scheme. “Great, you go do that. Here's what your interface needs to look like. You need to tell me where the road is in the image. If you tell me that. Someone will take that and will steer the vehicle.”

“Karl [Kluge], do you have any ideas?” “I think that we ought to be able to track edges and track color features. You ought to be able to track yellow lines and track white lines.” “Oh, that sounds like a good idea.” So Karl went off to do that. Great. “If you can tell me where the road is in the image, then I'll have someone figure out how to steer it.”

“Tony [Stentz], what do you want to do?” “Well, I really want to steer the vehicle.” “Good. You can do the path planning for that kind of thing.”

“Martial [Hebert],” he was a postdoc who came over from France, “what do you want to do?” Well, he was a 3D vision guy. “Cool! They're sending us a laser scanner. Do you think you could use that to determine where obstacles are?” “Oh, great. If you can find out where obstacles are, then feed that to me.”

“Jay [Gowdy] what do you want to do? I think you're a good systems guy. If someone tells us where obstacles are and someone tells us where the road is, could you figure out how to put the whole system together and give them useful user interfaces?”

So, really trying to build the system in a modular way, so that each person had something that they could do and be proud of. And if it worked, great! Then we plug it into the system and it runs. And if it doesn't work, well, I've got two other people working on other approaches to that same thing.

Gilliam: And if somebody like Dean [Pomerleau] came in with neural nets — wanting to apply Geoff Hinton's somewhat theoretical methods to this — how do you fold them in? Do they sit off a little further to the side in the lab? What does that look like?

Thorpe: Well, first of all, we said, “Neural nets? Oh, come on, nobody's ever built a neural net that does anything. Haven't they proved that neural nets aren't going to work? Um, you want to try it? Sure, try it. Let's see what happens. What would you like? You want some images? We'll collect you some images. Come on in. We'll see.”

And then talking to our people at DARPA, because there was a different and competing office at DARPA doing neural nets, and saying, “Look, I know you're skeptical about neural nets. We've got this neural net guy, separately funded. I want to give him a shot on this. You don't mind if he uses our robot?” “No, that's fine. We'll disprove neural nets once and for all. So come on in and try it out.”

Dean turns out to be an extraordinarily bright person. And you wouldn't know it to meet him, but he's extraordinarily competitive. And he's willing to get his fingers into everything. So, mostly what he wanted was this recording of images and steering wheel position. And then he went off and started to work on that and built a system with 100 hidden units and it seemed like it could work, but really slowly. And so he cut it down to 50 hidden units and it seemed like it could work as well, and a lot faster. He cut it down to 25 hidden units. And he eventually got down to four hidden units, and now he was working in a believable amount of time and it still worked pretty well. He cut it down to three hidden units and it didn't work anymore. So, okay, four hidden units.

Back to this theme of Strategic Computing and DARPA building supercomputers. One of the supercomputers they built was at Carnegie, and it was a thing called the Warp machine. The Warp machine had ten pipelined units, each of which could do ten million adds and multiplies at once. (The Warp machine and BBN’s butterfly processor were briefly covered in an earlier piece in the ARPA series.)

H. T. Kung was working on systolic computing, where things just sort of chunk through the pipeline. And we said, “Here's this big box. Because we're sponsored by Strategic Computing, we'll put it on the NavLab. Can anybody figure out what to do with it?” “Well, it's a really complicated thing and really hard to process.” But Dean could figure out what to do with it. Partly because Dean is a really, really smart guy.

Partly because the inner structure of a neural net is, “Multiply, add, multiply, add.” You take these inputs, you multiply them by a weight, you add them together, you sum it up, run it through your sigmoid function. So, Dean was, as far as I know, the only person… well, that's maybe a bit of an exaggeration. Some of the speech people were able to use the Warp also, because they were doing similar kinds of things…But Dean got the full hundred megaflops of processing out of the Warp machine! And then he put it onto the NavLab and we got the whole thing to work. So, after that, Jill got her thesis for doing her thing. Carl got his thesis for doing his thing. But Dean's neural nets seemed to work in most situations better than either Jill's or Carl's one. And so that's the one that got all the attention.

Going from ALVINN to RALPH and CMU’s complementary systems projects

Gilliam: What did you consider a bigger practical breakthrough? There's obviously the ALVINN system, which to a modern machine learning engineer’s eye, would give them a little tear. “Wow, it's so minimalistic It can do all this (with mostly just neural nets).” And then RALPH comes around. It’s a little more manual. Did you consider RALPH a much bigger practical step? Or once ALVINN came around, did it seem somewhat clear to you that something like RALPH would be possible and better?

So that was part of it, but the other part of it is..how do you take something like ALVINN…which at its core is just turning steering wheels. It's not doing any representation of the world…how do you take that and merge that with obstacle detection and avoidance? How do you take that and merge that with a map? And so, layering architectures on top of this that can take something which shouldn't really have a geometric representation and extract some sort of a geometric representation so we could do other things with it. That was really the fun part of it.

Gilliam: Is that when it became a proper, all hands on deck, CMU-type project — going from ALVINN to RALPH? It sounds like ALVINN was not a one-man job, but smaller. But then maybe RALPH became a proper CMU systems effort.

Thorpe: So Todd [Jochem] came along and worked with Dean. And part of what Todd did was to say, “How do you get some geometry out of this so that you can train multiple different neural nets and switch between them.” So that's even separate from RALPH. That was still in the ALVINN stuff. How do you train this thing up and have things work together? But if you look at the NavLab '90 videotape where we're going from my house to Keith's house, that was the obstacle detection and stopping. That was landmark recognition. That was the annotated map saying when we're supposed to do what. That was high-accuracy dead reckoning when we were doing a sharp turn and ALVINN couldn't work. Getting all of those system pieces to work together was a lot of fun. And this is on top of an experimental vehicle where you never knew what things were going to break.

Gilliam: I’d like to put something on the record, and you let me know if I'm wrong. So I think if some of my readers read what you said, they would say, “Oh, how fortunate that CMU had this systolic array machine going on and that it happened to be able to — obviously some computer structures were somewhat specialized back then — that it happened to be able to work with the natural language project and the vision project.”

But my reading of the history is that this doesn’t seem like circumstance. Now, this was a period when a lot of people weren't pursuing building machines at universities anymore. Obviously, this is not long after Slotnick [and the ILLIAC IV machine] at U of I. And Slotnick's gone on the record some years before saying “The era when you could build big, good machines at a university is over. Like, “this is for industry now, it's not for universities to do.” It seems like CMU’s systems approach helped prolong the era where you could build a good workable machine at a university.

Also I read that the Warp folks made sure that their machine could work with the other ongoing CMU systems efforts like vision and like natural language. Is that fair? Or is that me giving too much credit to the CMU structure?

Thorpe: So the CMU structure, when I got there, they had just finished building CM*, and before that C.MMP. CM* was a powerful array of 50 PDP-11s. But the difference between C.MMP and CM* was they were playing with…if you have multi-processor systems, do you connect a big crossbar switch with every memory connected to every CPU? Or do you connect them in some sort of a hierarchy? And so this notion of building big systems, big hardware systems, was there.

And of course, computer vision has always been a hog for CPU cycles. So in the early days, they built these systems and then they said, “Computer vision people and speech people, have at it!” Because the computer vision people and the speech people knew that we needed the cycles. And we were willing to put up with a fairly klutzy interface in order to see if we could take advantage of these sorts of things.

So, that ethos was there. The names C.MMP and CM* come out of the Newell and Siewiorek book. So here's Al Newell, a pioneer of AI, writing a textbook on computer architecture because he understood that AI needed multiprocessors in order to work. So that ethos was there.

We had a guy in the computer vision group named Rick Rashid, who was working on how you make the hardware more efficient, and how you make the software more efficient, and how you build microkernels. And he's the guy who went out and started Microsoft Research. So there's this long history of computer vision people having to have the highest possible processing power and being willing to put up with a fair amount of klutzy-ness in order to take a wire-wrapped prototype and really use it for for high performance.

Concluding Thoughts: CMU’s applications-focused culture, its origins, and the legacy of its grad students

Gilliam: This has all been fantastic. Is there anything else you think people should understand about how CMU managed its projects? It was a quite exceptional contractor there for a while…To me, CMU in this era seems like one of the all-time great DARPA contractors. Right up there with BBN in the ARPAnet era and Skunk Works when it built STEALTH. In many ways, CMU seems quite similar to BBN.

(Here, Chuck and I went on a 5-10 minute tangent discussing some of his friends from BBN, DARPA’s Clint Kelly, and others from the DARPA Strategic Computing era. Check that out on the audio if you’re interested. But it’s been skipped in this abridged version.)

Thorpe: …Back to Carnegie. Carnegie had a couple of aspects of the culture. One is that doing big things and doing practical things makes a difference. So if you look at what Carnegie has done…Duane Adams, his first proposal he wrote was for the NavLab. The second proposal he wrote was for the Software Engineering Institute. Running an SEI, that's not a typical academic thing.

But there are places like Berkeley. Berkeley runs the Berkeley labs and the Lawrence labs. So running an FFRDC (federally funded research and development center) is something that a handful of universities do, but not everybody. You don't get a lot of published papers out of that, but what you do is you provide a venue to take the basic research you've done and to turn it into practical things which change society.

So the SEI does big software engineering projects for DOD, but they also do a lot of training. They also run the CERT (Computer Emergency Response Team.) So, when some new virus breaks out, SEI jumps in and tells the world what the virus is and how to fix it. And that's the kind of thing that is a little bit unusual for a university to do. But Carnegie had the culture to say, “Making a real impact in society is important. And this is one way you can do it. Let's do it separately, off campus, so we don't dilute the graduate students and writing the papers, etc. But let's have this pipeline.”

Within robotics, NASA came to us, I don't know, 30 years ago and said, “NASA is spending a lot of money on robotics. We're not getting good PR out of it. Partly because our robots are, they go off to places like Mars. Could you take the technology that NASA is developing and work with people like John Deere?” So we formed the Robotics Engineering Consortium. It sits off campus. It employs more staff and fewer graduate students. It can do confidential projects. And it takes basic research done on campus and turns it into practical projects that Deere can take and turn it into products.

So that whole notion of a halfway house to take basic research and turn it into something bigger, something more practical, something applied…that this is a good thing for a university to do…that's not a universal university ethos.

Gilliam: Two-part question. The first is: why do you think CMU was able to maintain that ethos for so long? And the second part is interesting because, now, you’re a university president. You have the benefit of all sorts of institutional knowledge. Why do you think schools struggle to manage projects this way, if it seems, in many ways, to be a natural default approach for technical subjects?

Well, let me give you two reasons why Carnegie succeeded in doing that. The first one, I think, is really more Newell than Simon. Simon wanted to do great intellectual projects. Newell wanted to build things. So Newell caught on that building big stuff and seeing how it worked in the real world was important. And then Raj pushed that even further, starting out with speech. But then also the hardware projects etc. The other reason why Carnegie did this…if you go back in the history, there was the Carnegie Institute of Technology, and there was the Mellon Institute of Science.

The Mellon Institute of Science was the contract research shop. When the Mellons owned Gulf Oil, an aluminum corporation, and big chunks of various steel companies, they needed a chemistry research shop. And so the Mellon Institute was this chemistry research shop, and it existed to do contract research. If you got to produce some papers, that was good, but it was 50 Ph.D. chemists doing industrial research. In the great merger of ‘67, which formed Carnegie Mellon, we absorbed the Mellon Institute and had to figure out what to do with it. And partly what we did with it was to keep it running as a contract research shop.

So, there was already this notion of an off-campus, sponsored by soft money, more practical research shop that we didn't quite know what to do with. But in an ideal world, you would take basic research from the main campus and then figure out how to apply it, etc. So that was a little bit in the DNA of Carnegie since the merger in ‘67.

Gilliam: It's not dissimilar to MIT circa 1920. It had a program called the Technology Plan. For context, when MIT was young — from 1860 to 1920 — they were perpetually poor and living on shoestring budgets. Back then, more than now, MIT really lived on the ethos of, “We serve industry above all else. That’s what we’re here to do”. So, the Technology Plan came about because, at a certain point, they said, “The most honorable thing an institute of technology can do is provide a great service for a fair price.” So they let their applied researchers loose to work a lot of these contracts to the point where, at a certain point, their chemical engineering/applied chemistry department was 75-80 percent industry funded.

So this is very interesting.

Thorpe: Sure. And MIT has this history of spinning out the Draper Labs and the Lincoln Lab. And again, that's stuff that's too applied to be done in a university lab by graduate students, but that should have some connection to the university. Less connection in the case of Draper Labs because of some firewalls. But MIT has some of that same same ethos. Yeah, Carnegie had this very practical streak to it.

Paul Christiano was the provost for a while, and his dad was a Carnegie Tech graduate in civil engineering back when the civil engineering curriculum had courses like bricklaying. This is really hands-on, get out there and get your hands dirty.

Gilliam: Yeah. Back when they used to be able to teach industry machinists at night and things like that, at places like MIT or Carnegie. Different world we live in now.

Thorpe: So, when I was director of the Robotics Institute, I said, “We have won the battle to be the biggest robotics research group. We're very good at producing smart robots. We're very good at producing smart graduate students. We're not very good at producing smart industry. How do we build up a robotics industry in the city of Pittsburgh?” And we started working with the big foundations there. We started working with the little spinoff companies. There were a bunch of them, but none of them were very big. And we started working with the university management to say, how do we pull all of this together?

And when I say the foundations…like the Hillman Foundation, which is one of the richest foundations in the city…John Denny from L. C. Hillman's office came and sat with us and said, “How are we going to build a robotics industry?”And that led to the Robotics Foundry, and that led to Pittsburgh becoming RoboBurgh, which eventually led to these bigger clusters and some federal manufacturing research centers and so forth.

Gilliam: So in that case, do you take a lot of pride in the following? In your one previous oral history, you referred to the early Google group as “Carnegie Mellon West.” You said this talking about how Sebastian [Thrun], who of course, built his DARPA Grand Challenge vehicle at Stanford, but he came up at CMU. And you pointed out how everybody but one in that early Google autonomous driving group kind of came up or had a deep affiliation with CMU. And it seems like that's held even recently. There are a lot of departments that do pretty good autonomous car work, but Uber came to CMU, specifically, to essentially attempt to buy up a whole swath of the university and its projects wholesale to fold into itself.

Do you take a lot of pride in the projects and people that CMU has been outputting being considered applied enough that they could be a direct input into an industrial process? Rather than what the current applied researcher’s credo seems to be at a lot of universities, namely, “Somebody could pick this up and within a handful of years, it can then be applied.”

You all are kind of directly plugging in.

Thorpe: So, Matt Mason — who was on my thesis committee and was my successor as Director of the Robotics Institute — he's a much more theoretical roboticist. And he says, “You know, if all of the applied people left Carnegie Mellon, we would still be a famous robotics institute for all of the more theoretical stuff that we do.”

But they had to kind of live in the shadow because, face it, it's much easier to do a videotape of a robot flying down the road and to explain to the Pittsburgh Post Gazette what it is that we're up to than it is for the theoretical people to say, “Ooh, and look what I can do with a quaternion, I can represent this thing…and this is a fundamental, advance that people are going to be using for the next century!” And they're right! And we are grateful to them. They just have a harder time getting the publicity out of it.

So the DARPA urban challenge was just a family feud. Sebastian and I have a best paper award together. Sebastian and Red [Whittaker, the engineering lead on the DARPA Grand Challenge teams] have a best paper award together. Chris Urmson, who is Sebastian's number two, was Red's graduate student. So this was just a family feud. Those guys happen to be out at Stanford, but they’re still buddies of mine.

(We then talk for a bit about semi-related topics before I ask him if he’s also proud of the number of NavLab/CMU grad students that have gone on to be technical leads at companies or found companies.)

Thorpe: Yeah, you know, I had expected more of my alumni to go off into academia. If you're an academic yourself, you sort of think, “Shouldn't everybody want to do what I do?” And, no! They've gone off and started their own companies. Jill Crismann was the AI guru for the Department of Defense and now has gone back into private industry. Jenny [Kay] went into academia and still teaches. John Hancock went off into the computer game industry, which is a good thing. Hardware keeps breaking for him. So we thought that actually working on real robots was not as good as working on simulated robots.

Parag [Batavia] started his own company. Raul [Valdes-Perez] went off to Google. So it's been fun watching these guys. They have this attitude that, “We can do it! Here's a cool problem. Let me figure out how to make something work. Oh, cool. That works. Uh, now that it works, let me see if I can figure out how it works and write a paper out of it.”

And that's just a fun attitude and fun to watch them go and succeed.

Gilliam: “And if it works, how do I make it usable? How can it be productized?”

Thorpe: The next thing, and the next thing, and the next thing. Some of them have figured out, “How do I take this and turn it into money?” Others have said, “How do I take this and turn it into robots that run on the surface of Mars?”

I cut off the recording there. Over the course of the interview, I came to admire Carnegie’s applied-minded approach to engineering work even more than I did coming into the interview.

Hopefully, you enjoyed this exploration into how CMU managed its exceptional autonomous vehicle research work from the mid-1980s through the early 2000s. CMU’s strategies are, in practice, quite logical. However, comparing their methods to academic norms, it is clear that CMU continually made decisions to go against the academic norm. Doing this, they maintained their major comparative advantage. The result, in the field of computer science and robotics, was a systems research powerhouse that helped shape the field of autonomous driving as we know it today. In many ways, this period of CMU’s history is the closest university example I have seen to the fabled early years of MIT.

Thanks for reading:) See you next time!

Subscribe now

Share

Pattern Language Tags:

• Utilizing a contractor made up of individuals with research-style goals and training working within a ‘firm’ structure.

• Building out an experimental test bed for a research field.

• Putting out contractors for specific “integrators” in addition to prime contractors and basic researchers

This piece is a part of a FreakTakes series. The goal is to put together a series of administrative histories on specific DARPA projects just as I have done for many industrial R&D labs and other research orgs on FreakTakes. The goal — once I have covered ~20-30 projects — is to put together a larger ‘ARPA Playbook’ which helps individuals such as PMs in ARPA-like orgs navigate the growing catalog of pieces in a way that helps them find what they need to make the best decisions possible. In service of that, I am including in each post a bulleted list of ‘pattern language tags’ that encompass some categories of DARPA project strategies that describe the approaches contained in the piece — which will later be used to organize the ARPA Playbook document. These tags and the piece itself should all be considered in draft form until around the Spring of 2024. In the meantime, please feel free to reach out to me on Twitter or email (egillia3 | at | alumni | dot | stanford | dot | edu) to recommend additions/changes to the tags or the pieces. Also, if you have any ideas for projects from ARPA history — good, bad, or complicated — that would be interesting for me to dive into, please feel free to share them!

Specific Links:

• Chuck Thorpe’s IEEE oral history (video)(transcript)

• Red Whittaker’s IEEE Oral History (video)(transcript)

• Raj Reddy’s Computer History Museum Oral History (Parts One and Two)

• CMU Strategic Computing Vision Project Report: 1984 to 1985 

• Dean Pomerleau’s 1988 paper, ALVINN: An Autonomous Land Vehicle in a Neural Network

• RALPH paper

General Links:

• Strategic Computing: How DARPA Built the Computer Age

With that, let’s get into it.

Subscribe now

Introduction

DARPA has no lab benches, research facilities, or staff scientists doing science. Instead, it has contractors. DARPA uses contractor ecosystem to execute any ideas the office chooses to fund. Throughout this series I will generally write project histories from the POV of DARPA’s PMs and office directors. This is a natural choice because, depending on the era of the organization, either the project managers or the office directors are usually tasked with driving projects forward.

However, in the coming trilogy of pieces I’ll tell stories from the POV of some of DARPA’s best contractors ever. Projects are selected, funded, and often steered by the DARPA office; but ideas are entirely executed by contractors.

In many cases, which contractor wins a contract might only have a moderate impact on the cost efficiency, speed, and overall results of a project. In those cases, it might be hard for a contractor — whether it be a university research lab, private company, or a non-profit institute like RAND — to prove it is truly top-notch.

One the other hand, there are cases where many in a field do not even bid on a contract because they do not find the project specs feasible. In these cases, a level of excellence can be required to deliver on a proposal. This is the category the now-famous ARPAnet fell into. As most readers know, the ARPAnet was the precursor to the internet. It was as the main contractor on this project that the MIT faculty spin-off research firm Bolt, Beranek, & Newman (BBN) would cement its name in the history books.

The coming trilogy of pieces exploring DARPA contracts from the contractors’ POV provide an essential perspective to this series. Contractors’ work on projects like BBN with the ARPAnet, CMU’s driverless vehicle projects, and Lockheed Skunk Work’s STEALTH project have many management insights in common — despite the obvious differences between those projects and contractors.

Today’s piece dives into the contractor primarily responsible for DARPA’s most famous project: BBN.

What was BBN?

BBN’s stellar results on the ARPAnet project did not shock many in the computing community. In the late 1960s, the firm was known to be special. It was an interesting kind of research firm, of a sort we don’t see much of today. BBN’s staff found themselves natural members of the elite Cambridge, Massachusetts research community despite the firm’s for-profit status. The firm was home to some of the best theoretical researchers in Cambridge — formerly professors at Harvard and MIT — as well as top-tier engineering researchers who often defected from the large computing projects at MIT’s Lincoln Lab.

In general, many researchers defected because they believed BBN was a better place to work on big problems in more exciting ways than at Lincoln. Theoretically inclined researchers, like Robert Kahn, found this a great place to do proper research in an applied context. Great engineers who cared less for theory, like the project’s lead engineer Frank Heart, found the firm an ideal structure to work on implementing real technology that still had an extreme level of novelty.

To be sure, BBN was a company that found ways to make a profit and grow at a considerable rate in its early decades — 26% year over year. However, it was also a firm that went out of its way to ensure that its researchers were happy. It wanted the best problems — as long as it could cover the costs and salaries of the project with some kind of funding — not necessarily the most profitable problems.

Robert Kahn and J.C.R. Licklider are two familiar names that called BBN home for a time. From his view as IPTO chief and minor celebrity in the computing research community in his day, Robert Kahn had a deep understanding of what went on at every notable computer research center in the world. So it should be taken as no small thing when, in an oral history, Kahn had this extreme praise for the firm in this era:

BBN was a kind of hybrid version of Harvard and MIT in the sense that most of the people there were either faculty or former faculty at either Harvard or MIT. If you've ever spent any time at either of those places, you would know what a unique kind of organization BBN was. A lot of the students at those places spent time at BBN. It was kind of like a super hyped-up version of the union of the two, except that you didn't have to worry about classes and teaching. You could just focus on research. It was sort of the cognac of the research business, very distilled. The culture at BBN at the time was to do interesting things and move on to the next interesting thing. There was more incentive to come up with interesting ideas and explore them than to try to capitalize on them once they had been developed.

As we’ll explore, some at the firm would often try to capitalize on big ideas. But this was not the focus of the firm.

From a legal point of view, BBN was legally structured as a partnership for its early decades — kind of like a law firm — before going public in 1966. Teams at BBN were not exactly permanent. Individuals were often recruited simply because BBNers felt they could contribute to some coming project, or simply “raise the average level of competence of the firm.”

The firm’s culture was far more similar to a university lab than an incumbent firm like Honeywell. If a talented researcher found an area of exploration interesting, the firm did what it could to find a payer for that research. For example, this was the case with a young J.C.R. Licklider looking to further explore the world of interactive personal computing on a very expensive machine. Bolt and Beranek did not say no. Rather, they said yes, bought a machine, and found a way to fund the project through a grant from the Ford Foundation. The long research report — later turned into a book — by Licklider and other BBNers titled, “Libraries of the Future” was far more than it seems. The report did more than just carry out a forward-looking analysis by information researchers and engineers assessing what digitized libraries could look like for a library non-profit. The report contained much of the early technical exploration that would grow to become Licklider’s legacy.

Beyond BBN’s whole-hearted embrace of the “research” in “research firm,” its structure as a “firm” allowed it to carry out extended, elaborate implementation projects like the ARPAnet project that were ill-suited to an academic lab. The firm’s expertise in early computing systems — built up by contract research projects as well as the team’s pre-existing experience at places like Lincoln Lab — allowed it to push technology areas forward in a self-funded way. For example, in the 1960s the firm designed, implemented, and maintained one of the first commercial time-sharing systems for non-engineers. The project gave Massachusetts General Hospital a time-sharing system. The system allowed the hospital to handle patient administration in a modern way and automate some of the data analysis in their research. This, of course, was not the first time-sharing system. But projects like this allowed BBN engineers to continue honing expertise with young technology and earn private sector wages in doing so.

The ARPAnet project was another implementation project optimally suited to BBN’s somewhat unique structure and makeup. But, to gain a full understanding of how the firm came to be what it was, let’s take a step back and start at the beginning. BBN was not started as a computing research firm. Bolt and Beranek were not computer scientists. They were acoustics professors who, steeped in the applied research tradition of early MIT, began to become overwhelmed by their contract research work in the 1940s.

BBN’s early years as an acoustics firm

BBN began in a fashion not unlike many early-20th-century MIT spin-off companies. A team of professors and graduate students worked extensively on consulting and contract work. As time went on and word spread, the demand for their contracting services grew. For a while, renting an additional room in their lab building and hiring more graduate students was enough to keep up with demand. Eventually, the young company had to get its own quarters and hire its own full-time employees.

In the case of BBN, the original professors involved were involved in the relatively young area of psycho-acoustics. In the wake of World War II, Richard Bolt was set to become the director of a new acoustics laboratory at the Institute to continue several courses of research that were largely funded by the Navy. Faculty and staff with backgrounds in physics, electrical engineering, architecture, mechanical engineering, aeronautical engineering, and psychology would all contribute to the contracts. In these post-war years, Leo Beranek was recruited to join the lab as its Technical Director from Harvard’s electro-acoustic laboratory. He was also given a professorship.

Since the inception of MIT’s Technology Plan in 1920, the Institute had made a habit of actively facilitating research contracts to its faculty. Professor’s contract salaries covered only four days a week during the academic year and left weekends, holidays, and summers entirely free to the professors for contract work. Professors could wholeheartedly spend their time on contract work with close to zero red tape or permission required from MIT. Unlike places like Harvard — and most universities — the Institute, in this era, encouraged this double life for its professors. Negligible amounts of paperwork and permission-asking were involved. MIT saw no more suitable activity for faculty at a professionally-focused institute of technology than to ply their craft for paying customers. In doing so, professors simultaneously kept up with industry trends, solved industry’s problems, learned how to keep their university research industry-relevant, and brought money into the Institute.

In the heyday of this arrangement, requests from industry regularly came into MIT’s office of the president for help in various problems, such as acoustics. When acoustics queries came in, they were routed to Richard Bolt. In 1946, one of these requests came from an architecture firm responsible for building the UN Headquarters in New York City. Bolt submitted a bid and won the commission. In 1948, the drawings arrived from the firm with the request for Bolt to begin his acoustics consulting work. Given the massive size of the drawings, Bolt realized the project was not a one-man job. He asked Leo Beranek if he’d like to work on the contract as a team. The two had papers drawn up, and the partnership of Bolt & Beranek came into existence.

For this first contract, Bolt and Robert Newman — one of four part-time graduate students hired to work on the project, who was brought into the partnership upon graduation — were responsible for figuring out acoustical treatments for the UN building. Beranek was in charge of the more troublesome task of sound system design and equipment selection for the unorthodox building. Despite the difficulty, the project turned out to be a very public success. Beranek wrote that the firm’s “name was known to architects everywhere, and business boomed.”

To this point, the firm had simply been run out of an extra room in the Acoustics Laboratory. As Beranek described the arrangement:

The firm, Bolt & Beranek, had the blessings of MIT’s new President, James Killian. He offered to help us get started and rented us two rooms in the MIT Acoustics Laboratory for our use, but warned us that we would have to seek space outside of MIT if our needs expanded.

By the end of the business’ first twelve months, the room was already full of equipment the firm had bought to continue to service the growing number of contracts.

In late 1949, the young firm moved to another address in Cambridge. Gradually growing, in 1951 the firm would move again to take up two apartments and a basement in a mostly residential building on Elliot Street. By 1956, the acoustics-focused firm had built up to 50 full-time employees — many of them former Cambridge graduate students — along with an additional office in the architecture hotbed that was LA. This all happened without outside financing beyond the firm’s line of credit with the local bank.

In this period, the firm also added two more of the four founding part-time graduate students to its partnership. These new partners, Jordan Baruch and Samuel Labate, had worked with the firm for years and also completed graduate theses in areas relevant to the firm. Additionally, in 1953, BBN incorporated in order to shield itself from liabilities associated with the growing subsection of its business associated with controlling aircraft noise. The National Advisory Committee on Aeoronautics and companies that manufactured jet engines hired the firm to work on problems such as designing structures to minimize noise during engine testing. Projects like this had many practical uses, but also represented cutting-edge uses of the theory and experiment in acoustics that excited acoustics researchers. (Projects like this also benefited from the insights of psychologists. As it turned out, projects in noise reduction or concert hall design often turned out to be more about “perceived” decibels rather than actual decibel measurements.)

As all of this was happening, Beranek gradually reduced his teaching load at MIT — to 75% in 1951, 50% in 1953, etc. He resigned his tenured professorship in 1958. More and more work — from government contracts, private industry, and research grants — steadily rolled in. As the business grew and contracts rolled in, the MIT and Harvard research communities provided an ample supply of part-time minds and hands. Sometimes individuals were brought in because they were acutely needed. In other cases, the firm simply saw a talented individual and would find some excuse to bring them on.

As the business kept expanding, in size and scope, Beranek set his sights on building up more of a talent base in computing. During the war, Beranek remembered how much of the work of his Harvard Electro-Acoustic Laboratory and its partner lab, the Psycho-Acoustic Laboratory, related to problems dealing with information handling in warships. BBN already combined psychological insights in its acoustics work, working on problems like pilots’ ability to process noise in cockpits. Expanding into the arena of man-machine systems felt natural. To Beranek, this felt like a fascinating area of research with the potential to amplify human labor.

BBN enters the computing business

Beranek’s motto in making hires at the firm was, “Each new person hired should raise the average level of competence of the firm.” At its core this was a research firm. And in staffing the research firm, he felt it was vital that they only continue to hire people who they felt were as smart as them. With Beranek’s sights now on computing, he seemingly only had one man in mind to help lead the firm’s computing efforts: J.C.R. Licklider.

Licklider had been a young researcher in the Psycho-Acoustics Laboratory at Harvard during the war. Beranek was so impressed with him that, soon after being brought over to MIT, he pushed for Licklider to be hired there as well. Now, he was going to push Licklider to give up his tenured position at MIT to come to BBN. With the promise of generous stock options, the understanding that the firm did cutting-edge research work, and the title of “vice president in charge of man-machine and information systems,” Licklider came aboard in 1957.

One co-worker said of Licklider’s move that, “he was becoming entranced by computers at this point and felt he could pursue these interests best at BBN.” In the end, Licklider would go on to draw many of his former friends and colleagues from graduate school, the MIT faculty, and elsewhere in his academic travels to come to BBN. After moving to BBN, some simply continued with their pre-existing work under research contracts they’d won before joining BBN that paid for their services. Some were folded into BBN’s regular operations. Others found contracts to do odd things like write textbooks using BBN’s rare equipment.

But before this collecting of colleagues started, it was just Licklider joining the firm from his MIT position. No formal roadmap had been agreed upon with BBN management. Licklider quickly got things rolling, though. Beranek describes Licklider’s early tenure at the firm as follows:

Lick, as he insisted that we call him, was outgoing and always on the verge of a smile; he ended almost every second sentence with a slight chuckle, as though he had just made a humorous statement. He walked with a gentle step, often with a Coca-Cola in hand, and he always found the time to listen to new ideas. Relaxed and self-deprecating, Lick merged easily with the talent already at BBN. He and I worked together especially well: I cannot remember a time when we disagreed.

Licklider had been on staff only a few months when he told me, in the fall of 1957, that he wanted BBN to buy a digital computer for his group. When I pointed out that we already had a punched-card computer in the financial department and several analog computers in the experimental psychology group, he replied that they did not interest him. He wanted a then state-of-the-art digital machine produced by the Royal McBee Co., a subsidiary of Royal Typewriter.

“What will it cost?” I asked.

“Around $30,000.” (~$300,000 today) he replied, rather blandly, and noted that this price tag was a discount he had already negotiated.

I exclaimed, “BBN has never spent anything approaching that amount on a single research apparatus. What are you going to do with it?”

“I don’t know,” Lick responded, “but if BBN is going to be an important company in the future, it must be in computers.”

Although I hesitated at first — $30,000 for a computer with no apparent use seemed just too reckless — I had a great deal of faith in Lick’s convictions and finally agreed that BBN should risk the funds. I presented his request to Labate and Baruch, and with their approval, Lick brought BBN into the digital era. Lick sat at that computer many hours each day, literally hoarding the machine, learning how to do digital programming.

To be sure, this turn of events was probably not surprising to Beranek. Licklider had been deeply involved with computers at MIT through his affiliation with Lincoln Lab computing projects as well as interactions with Wes Clark and other early computing gurus. It was at MIT where Licklider, as a technically focused psychologist, was becoming an evangelist for work we would now categorize as human-computer interaction. But Licklider may have grown expensive more quickly than Beranek anticipated. As contracts and know-how were gradually building up in the computing area of BBN, the firm bought the first PDP-1 from the Digital Equipment Corporation two years after it purchased the first machine.

After describing the Royal McBee purchase in his chapter of “A Culture of Innovation” — an internal history of BBN by BBN alumni — Beranek described the obvious next step:

Then Lick and I took off for Washington D.C. to seek research contracts that would make use of this machine, which carried a price tag of $150,000 (~$1.5 million today). Our visits to the Department of Education, National Institutes of Health, National Science Foundation, NASA, and the Department of Defense proved Lick’s convictions correct, and we soon secured several important contracts.

When it came to talented researchers and academic engineers, BBN’s instincts as researchers often led the way. The organization was a firm and would not willingly light money on fire to pursue its interests. But if a talented group of researchers and engineers felt a problem area was promising, BBN might spend money to pursue it as long as management felt it could probably earn the money back.

In the case of purchasing the PDP-1, this was a more risky leap than the company usually took. But the risk was quickly mitigated, with BBN quickly selling government contracts in areas like man-machine cockpit system displays and computer-based learning. However, the primary grant driving Licklider’s work on computers at the firm was not a government grant. The grant that proved futuristic enough to support Licklider’s grand vision came from a grant-maker the reader would likely not expect.

Libraries of the Future

The Council on Library Resources was established by the Ford Foundation to study “libraries of the future.” The Ford Foundation was something like the Gates Foundation of its day and, thus, funded a broad swathe of areas. In particular, the libraries of the future contract arose from the growing “information problem” talked about in the research community in the 1960s. Namely, much more work was being published than any researcher could possibly make use of. Licklider and BBN were tapped to explore whether or not modern computing could do something about this problem.

(For those interested, the Council on Library Resources was advised by Warren Weaver, John Pierce, and Richard Bolt in the early stages of its existence. Weaver and Pierce have been featured in earlier pieces on this Substack.)

Contracts like this two-year contract were not likely where BBN made the lion’s share of its money. However, projects such as this — or projects such as those where NASA paid the company a fee to perform exploration and produce a textbook — allowed BBN to offset the costs of explorations its researchers already wanted to do.

Going above and beyond what the Council on Library Resources likely expected, Licklider’s team used this contract as an excuse to explore seemingly futurist technology. For example, Licklider and the project team dedicated much of the second half of the report to technical explorations under the assumption that future library-goers might interact with the library as a store of knowledge with a question-answering front end, rather than as a means to check out books.

The project seems to have gone a long way in offsetting the costs in a variety of explorations by BBN staff. Below, BBN-alum John Swets describes a question Fisher Black — of Black-Scholes Model fame — pursued:

Fischer Black, a mathematics graduate student at Harvard and part-time BBN employee, produced a series of question-answering systems that involved symbolic logic and computer programming, including one in which a non-human system first solved the “airport problem” posed by McCarthy. Based on some statements about a person’s whereabouts, transportation resources, and local geography, the system answers the question of how to drive to the airport (in part: walk from the desk to my garage, drive my car to the airport.)

(Eventual Turing Award winners John McCarthy and Marvin Minsky were also part-time members of the Libraries of the Future project team.)

Other ideas explored in the report include:

• The importance of improved random access memories for optimal information retrieval on large datasets — like one containing the entirety of scientific papers.

• Black’s question-answering system which successfully used first-order predicate calculus to represent information.1

• Several prototype indexing methods and iterative information retrieval methods for searching academic papers.

• And many qualitative aspects of PC user experience which Licklider is known for.

Many ideas in the report were not novel on their own. However, many of the systems outlined in the report were novel, practical advances that both fulfilled BBN’s contract obligations and the research ambitions of its employees. The report paints a remarkably prescient picture of how personal computers would eventually organize knowledge and interact with users. In reading the report, one can easily recognize many of the ideas for which Licklider would later become famous as the PC revolution got underway.

In this period, BBN was also working on more engineering-focused computing projects. The practical experience and staff the firm acquired in undertaking these projects gave the firm the much-needed experience and talent it would later need to complete the ARPAnet project. The most important of these projects may have been BBN’s work developing small machine time-sharing systems for hospitals.

BBN’s Hospital Time-Sharing Systems

In 1958, Ed Fredkin had only briefly been at Lincoln Lab before deciding to go into business for himself. To get started, he ordered himself a Royal McBee. He did not yet have the money to pay for the computer, so he went seeking out contracts. He consulted with his friends Frank Heart, then at Lincoln Lab, and Tom Marill, who had recently left Lincoln Lab for BBN, to see if they knew of any customers. Marill told Fredkin to see if BBN would be interested in doing some business with his company. So, he set a meeting with BBN’s new computing czar, J.C.R. Licklider. Fredkin’s meeting with the newly-joined Licklider ended with Licklider pitching Fredkin to, “come work at BBN.” Licklider convinced the partners to purchase Fredkin’s Royal McBee as part of his hiring arrangement. With that, Fredkin was added to Licklider’s growing team of computer engineers.

Licklider’s computing team buildup was still in its early stages, as Fredkin remembers it. If you said “computer,” people would say “analog or digital?” to show they at least knew something about computers. In Fredkin’s estimation, even Licklider was very much still learning the ropes. It was clear that Lick’s higher level ideas were fantastic. But as an engineer, Fredkin had this to say about him:

There was nothing I could do to get Lick to be a good programmer. He insisted on being a coder, and had wonderful high-level ideas; but what he always chose to code never made sense to me. I tried to straighten him out a number of times but couldn’t succeed. It’s just that at that time he didn’t have a knack for coding, either in the style of code or in the things he chose to code.

In short, in Fredkin’s estimation, Licklider seemed to be more bird than frog at that time.

But Fredkin may have had high standards. Licklider referred to Fredkin as a “young genius” and said the decision to bring him in to help grow the computing efforts was a no-brainer. Fredkin, who at the time was not the most fit for a corporate environment, found Licklider and BBN very willing to find a way to make the arrangement work. Licklider remembered that “he was having all kinds of psychological problems about getting work done.” So, BBN worked out an arrangement with Fredkin where, at the end of the month, they would figure out what his salary would be for the month. To Lick, this was just one of BBN’s many “fantastically interesting and flexible arrangements” that it worked out with its sharp employees.

Fredkin was instrumental in convincing BBN to buy the first PDP-1 computer and to bring on professors Marvin Minsky and John McCarthy as part-time employees. Soon after this arrangement began, Fredkin surprised McCarthy by saying that he believed some of McCarthy’s time-sharing concepts could actually be implemented on the rather small PDP-1. Fredkin’s combination of engineering design skills and professional connections from Lincoln Lab helped push this work forward. As McCarthy remembers:

Fredkin designed the architecture of an interrupt system and designed a control system for the drum to permit it to be used in a very efficient swapping mode. He convinced Ben Gurley, the chief engineer for D.E.C, to build this equipment.

Fredkin and his colleagues made multiple technical improvements to prove McCarthy’s ideas could be implemented on such a small machine — including inventing the concept of a swapping drum. Fredkin left the firm in 1961, with McCarthy taking over directing some of Fredkin’s time-sharing projects. But Fredkin’s efforts did a lot to jumpstart BBN’s engineering efforts in time-sharing.

This early time-sharing work eventually led to BBN being hand-picked for an extensive course of funded engineering research. The work came about when Jordan Baruch, one of BBN’s partners, was doing acoustics work for the Clinical Center at the NIH. His project involved instrumentation for the cardiac and neurological staff. During his weekly trips to the NIH, Baruch would go to the home of the Clinical Center’s Director, Jack Masur, for gin and jelly beans. On one of these evening visits the topic of computing in healthcare came up because Masur was asked to give a speech on the topic. Baruch seized the opportunity and shared his vision for the possibilities of computers with patient medical records. Masur used many of Baruch’s ideas in his speech. After the speech was a hit, Masur phoned Baruch and told him, “You should go do it.” Masur recommended Baruch apply for a grant from the NIH’s Division of General Medicine to build the system. BBN did. It won the $1 million (~$10 million today), three-year initial grant.

The contracting relationship would continue over the years, growing from a proof-of-concept prototype to an actual system deployed in a hospital. The specific goal of the research project was for BBN to develop a hospital computer system that would automate “the information gathering, processing, storage and retrieval that takes place in the modern hospital.” The work eventually grew into BBN installing terminals around Massachusetts General Hospital (MGH) which were connected to a time-shared computer at BBN. The initial applications were admissions, discharge, medication ordering, test ordering, and test reporting.

As the project wore on, the medical professors at MGH began using the system for certain types of interactive (albeit primitive) data analysis for their medical studies. BBN staff worked with them to develop these capabilities in the system. As former project manager Paul Castelman noted, “At the 1965 RAND/SDC conference on advanced data-management systems, BBN’s Hospital Research System was the only system reported that operated interactively rather than in batch mode.”

Four years after the start of the MGH project, work on the project began slowly winding down. As a technological proof-of-concept, the project was a success. But it seemed that the system’s deployment in a teaching hospital found difficult reception with some of its (apparently many) opinionated faculty. As project member Bill Mann remembered:

The feelings at MGH ranged from “very interesting” to “it may kill my patients, get it out of here,” with a strong bias towards the latter.

In the mid-1960s, the project’s originator, Jordan Baruch, was leaving BBN to spin off a (BBN-sanctioned) joint venture with GE based on the work called Medinet. The MGH system itself was being handed over to the hospital to operate. In this period, Frank Heart joined BBN as a Vice President in charge of managing technical engineering, with a particular interest in BBN’s life sciences efforts. Heart, in his first year and a half at the firm, helped spearhead BBN’s life sciences computation efforts in various areas of biomedical research.

However, not long after Heart’s joining word was swirling around that Larry Roberts had been brought to ARPA to orchestrate a contract that, as an engineering problem, was too fun for the team at BBN to pass up. Heart and several other Lincoln Lab alumni who had joined BBN in this same period were ideally suited to carry out Roberts’ project. They had a toolkit few in the computing research ecosystem had at the time: deep experience in designing and implementing real-time systems.

ARPAnet Context and Beginnings

The computing team that had been amassing at BBN contained several individuals from the small community specializing in real-time computing systems. In his oral history, Frank Heart explained why so few in the computing community had real-time systems experience, saying:

You know, in that period of time, the computer world was really somewhat divided into people who understood real-time systems and those who didn't. I mean, there were people who built computer systems using operating systems, and there were other people who built very finely tuned machine language programs for dealing with phone lines. And those two camps didn't interact a great deal. So the world was not full of people who knew how to make computers run in real-time, and connect to real-time systems. We were not the only group, but it was a somewhat small universe.

Making computers work via communication lines was no easy task. Larry Roberts, the IPTO PM who oversaw the ARPAnet project, had his own difficulties with the technology while at Lincoln Labs. In his last round of research at Lincoln Labs before joining ARPA, Roberts had been attempting to outline the open computer and communication issues that stood in the way of achieving practical inter-computer communications. As Roberts remembered, the results of this work he’d done with Tom Marill showed “that computers could work with each other, and we figured out how to do that — but we couldn’t get the communications to work on it at all.”

This problem, inter-computer networking, is the primary problem Roberts was brought to ARPA by IPTO Director Robert Taylor to work on. Roberts, who initially rejected Taylor’s offer since he so enjoyed his life as a researcher at Lincoln Lab, eventually had his arm twisted by Lincoln Lab staff. He had turned down Robert Taylor’s offer in early 1966. But, after failing to find another candidate as good as Roberts, Taylor went with something of a Plan B. Then-ARPA Director Charles Herzfeld, at Taylor’s bidding, called up Lincoln Lab’s Director to point out that ARPA was 51% of Lincoln’s funding. Herzfeld then re-emphasized just how much they would like to have Roberts’ services. Roberts accepted. Or, as Taylor summarized these events, “I blackmailed Larry Roberts into fame!”

For reasons beyond the diet blackmail, Roberts was coming around to what he viewed as a bit of a research management job. Roberts explained:

I was also coming to the point of view…that this research that we [the computing group at Lincoln] were doing was not getting to the rest of the world; that no matter what we did we could not get it known. It was part of that concept of building the network: how do we build a network to get it distributed so that people could use whatever was done? I was feeling that we were now probably twenty years ahead of what anybody was going to use and still there was no path for them to pick up…So I really was feeling a pull towards getting more into the real world, rather than remain in that sort of ivory tower.

This general sentiment, coupled with BBN’s openness to interesting research problems if the money was right, is partially why the firm was so successful in recruiting Lincoln Lab staff.

Roberts’ willingness to dedicate the next years of his career to the specific problem of orchestrating the building of the ARPAnet is not surprising. He had started heavily thinking about the problem four years prior. The event that pushed Roberts to begin dedicating his research agenda to the idea was a series of conversations he had at a 1962 computing conference with, among others, J.C.R. Licklider. Roberts said this of the ideas that came up talking with the group:

I came to the conclusion that the next thing, really, was making all of this incompatible work compatible with some sort of networking. In other words, we had all of these people doing different things everywhere, and they were all not sharing their research very well. So you could not use anything anybody else did. Everything I did was useless to the rest of the world, because it was on the TX-2 and it was a unique machine. So unless the software was transportable, the only thing it was useful for was written technical papers, which was a very slow process. So, what I concluded was that we had to do something about communications, and that really, the idea of the galactic network that Lick talked about, probably more than anybody, was something that we had to start seriously thinking about. So in a way networking grew out of Lick's talking about that, although Lick himself could not make anything happen because it was too early when he talked about it. But he did convince me it was important.

According to Robert Taylor, the recent development of time-sharing was the reason he felt his tenure as IPTO chief was a particularly good time to make a push for the ARPAnet project. Taylor said:

The thing that struck me about the time-sharing experience was that before there was a time-sharing system, let's say at MIT, then there were a lot of individual people who didn't know each other who were interested in computing in one way or another, and who were doing whatever they could, however they could. As soon as the time-sharing system became usable, these people began to know one another, share a lot of information, and ask of one another, "How do I use this? Where do I find that?" It was really phenomenal to see this computer become a medium that stimulated the formation of a human community.

The ARPAnet, in Taylor’s mind, was meant to make this “interactive notion” work one level higher — across universities rather than just within them.

Prior to Roberts’ arrival, Taylor had already gotten the necessary funds for the early stages of the project approved and set aside by Director Charles Herzfeld. The next step in the project is common to many major ARPA projects like this. Larry Roberts went around talking to anybody in the ARPA computing community who could conceivably be useful. He wanted to have an understanding of all the opinions from all the top people before he put a concrete proposal out into the world.

Many researchers and engineers contributed ideas that made it into the final proposal, such ideas to improve the network’s host-to-host protocols. But one researcher who was consulted in the planning stages of the RFP can be considered of particular importance. That researcher was Wes Clark of Lincoln Labs — who mentored several BBN employees while they were there. Clark, who his contemporaries considered a bit of an inventive genius, is a bit of an unsung hero of this generation of computing for his work in building early small computers — such as the TX-0, TX-2, and LINC. Clark’s thoughts helped convince Roberts that instead of using a large machine in the center of the country to help “run” the ARPAnet, Roberts should have a small computer at each site handle that. Clark’s advice was singularly important in determining the final structure the ARPAnet took — which was remarkably decentralized.

These small computers Clark proposed came to be known as “IMPs.” In the ARPAnet, the IMPs were meant to connect to both the large host computers at the ARPAnet site as well as the corresponding IMPs at other ARPAnet sites. If they could be successfully made, the use of IMPs would substantially ease the burden involved in connecting host sites to communication lines. Instead of each site having to find a way to connect directly to the telephone lines, they’d simply connect to an IMP. It would be the main contractor’s job to make an IMP reliable and easy enough to use to make the ARPAnet work.

Issued in 1968, the RFP was seeking a prime contractor to design, build, and install these IMPs. In essence, whoever won the contract would be principally in charge of the ultimate success or failure of the project. The ARPAnet nodes — largely research sites with a lot of IPTO funding already — would receive their own ARPA contracts to perform some amount of set-up, debugging, and research on their own end. The IMP contractor would be in charge of building IMPs that would be able to interface with most of the large computers at the time — which were not built to be interoperable. This IMP contractor also had to make sure that IMPs could communicate across noisy telephone lines, across thousands of miles, in real-time. This was the problem that had stumped Larry Roberts on a small scale while he was still at Lincoln Lab. This was a difficult research project as well as an intricate kind of engineering implementation problem.

The team at BBN thought this sounded like great fun. Each member of the proposal team — Frank Heart and Severo Ornstein on hardware, Dave Walden and William Crowther on software, and Robert Kahn on theory — make a point of repeatedly emphasizing that the project just seemed to be great “fun” and a fascinating engineering puzzle to solve. One or two felt it was a problem of massive importance. More often, they did not quite grasp how significant the project would become. Regardless, it was clear that this was a contract BBN wanted in on. So, Frank Heart and Robert Kahn — who had been consulted during the RFP process by Larry Roberts — took the lead in BBN’s proposal efforts. Kahn spearheaded the conceptual aspects of the proposal. Heart, who would eventually be put in charge of the project, spearheaded the more practical engineering aspects.

Kahn, who would later become IPTO chief, had also found his way to BBN around the same time Heart had. In 1966, Kahn took leave from MIT and joined BBN on the advice of a fellow engineering professor. Kahn’s research focus on mathematical aspects of communications was very theoretical. Despite Kahn having worked stints at Bell Labs during his Princeton Ph.D., he was still driven mostly by theoretical problems. So, Kahn’s senior colleague felt he would benefit from additional practical experience. Kahn referred to this piece of advice as “the best advice I ever got.” So, he headed to BBN. He initially intended to be there for only a year or two — he ended up being there for around six years. At BBN, he chose to work on networking.

He described his intentions as to “get my hands a little more dirty with some of the practical problems of everyday engineering.” He seemingly felt BBN was the best place to do this. He described BBN as follows:

It was a marvelous little firm at the time, a fraction of the size it is today. It was largely a group of very professional people doing an extension of what Harvard and MIT were doing, except on a full-time basis, without the responsibility of dealing with students and teaching.

With the RFP out, BBN put far more person time into the proposal than was normal for the firm. Five or six full-time people worked “very, very long hours” on the project for approximately three to six months. As Severo Ornstein remembered, “I spent a lot of time, I remember nights till 3 or 4 in the morning, working with Bob Kahn in the back room of my house in Newton on the proposal — designing the system and figuring out how it was all going to work.” Ornstein also noted that, “more dollars were spent preparing that proposal, more man hours charged to it, than I think had ever been done for any [BBN] project.”

As a reference, the firm strove for 70% of an individual’s time to be billable to some contract in some way. This inordinate use of time on a single proposal was a special case. To be sure, the special case was probably partially driven by the understanding that the contract could lead to longer-term contract extensions. But it seems that the primary reason the team spent as much time as they did was because both BBN employees and management believed this to be an exciting technical problem area — one which the firm felt they knew a lot about.

Many researchers on the team had known Larry Roberts at Lincoln Lab, knew him to be exceptionally smart, and figured he would understand how important their real-time computing experience was. Heart’s Lincoln Lab’s work with the Whirlwind computer required the processing of radar data in real time. As he put it, this meant “the computer had to accept the data at phone line rates and deal with each radar scan before the next one came along.” Heart continues, describing what he believed made his BBN engineering team so attractive in his chapter of “A Culture of Innovation,” saying:

This kind of computer use was unusual at the time, but at Lincoln Laboratory the group of people working with me became unusually expert at the real-time use of computers. At Lincoln, in a long series of projects, computers were connected to various radar antenna systems, radio antenna systems, sensors at underground seismic arrays, and sensors at underwater acoustic arrays. Each such project required a detailed understanding of the computer timing relative to the sequence of data arriving from the various sensor systems. This experience at Lincoln — tying computers to phone lines, and constructing hardware and computer programs that involved the timing constraints of such data handling — was a crucial attribute of my group at BBN that years later bid on and won the ARPAnet contract.

BBN’s main competition for the contract was large contractors like Raytheon and Western Union. Companies like this were significantly less focused on cutting-edge research than BBN but had deep experience managing large systems integration and implementation projects. Roberts stated in an oral history that, on paper, he thought that Raytheon’s bid looked quite reasonable. It wasn’t in a completely inferior league to BBN’s. But Roberts, himself having come up at Lincoln Labs with many BBN staff, did not ignore the advantages that the firm’s structure offered. He recalls:

Raytheon had a good proposal that competed equally with BBN, and the only distinguishing thing in the long run for my final decision was that BBN had a tighter team organized in a way that I thought would be more effective than a very steep commercial structure with lots of managers and vertically – so I didn’t believe they’d keep the schedules as well, but they had a good proposal.

To be clear, ARPA often trusted Raytheon with a variety of implementation contracts. So, it is not likely that Roberts meant that Raytheon would not reliably stick to an implementation schedule. Rather, he likely meant that BBN was far more likely to solve its way through all of the novel engineering challenges that stood between these proposals and an actual system being implemented. This was an extremely novel system. That was likely why BBN won the day.

Of course, all engineering projects are at least, in part, research projects; they all have many “unknown unknowns,” even once they are solved on paper or have been implemented before. Those standard challenges obviously existed in this project. However, the ARPAnet system had so many major “known unknowns” that this was as much a research project as an implementation project. The whole system was an experiment. In fact, the research-focused Robert Kahn stayed on with BBN for three more years after BBN won the contract because of this. There were practical, ongoing research problems — such as those in the area of systems design — that were going to require continuous re-working as the project progressed. As Kahn explained:

My notion originally was "Help them get the award; let them go build it and I will go back to doing what I was doing." But I pretty quickly came to realize that that just wasn't a practical notion. There were too many things that had to be thought about.

According to Roberts, companies like IBM even ‘no bid’ the contract because they didn’t think the request, as it was written, was possible. The price for the computing tasks required, they thought, was completely infeasible with the technology that existed. Roberts remembered:

IBM and CDC 'no bid' it because, they told me: "If we used Model 50s at these sites, you're going to go broke, and we think you're crazy." And what else is there, of course, besides Model 50s?

Many corporate researchers in the communications industry felt that the packet switching idea, which the network’s IMP-IMP communications relied upon, was completely infeasible. To players like AT&T, this was an academic idea that head-in-the-clouds researchers came up with; this was all fine and good on paper, but would just not hold up to the practical engineering reality. BBN obviously disagreed. But, as Frank Heart put it, “There was no proof. You couldn’t go to some book and find the answer; it was an experiment.”

The challenge, which some thought impossible at ARPA’s price point and given the noisiness of telephone lines, was set. BBN won the IMP contract. It was now their job to make it all work.

ARPAnet Operations

The initial one-year contract Roberts awarded BBN was to develop the IMP and deliver a four-node network connecting UCLA, the Stanford Research Institute, UC Santa Barbara, and the University of Utah. The contract was for about $1 million in 1969 (~$8 million today). The initial team that worked on the contract that year was made up of the initial five members who drew up the proposal along with about three more full-timers, primarily to work on hardware, and four or so additional part-timers. Several of the hires were Severo Ornstein’s best students from his course at Harvard.

The proposal team had, at least conceptually, worked its way through many problems in the extensive time it spent working on the proposal. So, the team was able to hit the ground running and make a push to have the four-node network operational by the end of the year. In all likelihood, Roberts likely expected the team might not make the deadline. In many cases, ARPA contracts would set extremely ambitious benchmarks that PMs, office directors, and contractors would later re-negotiate. But this did not turn out to be one of those cases.

Other Contractors on the Project

A number of subcontractors were also involved in the ARPAnet project. Each of the node engineering departments was a subcontractor. To a certain extent, these departments were responsible for getting their host computer set up with the IMP. BBN would do what it could to make this as easy as possible for them. But given the state of the technology and the talent present at the nodes, this was deemed to be the optimal approach. Additionally, ARPA used a contractor called DECCO to take charge of the logistically annoying process of purchasing time from AT&T on their telephone communication lines.2 Of course, DECCO purchased what BBN told them to purchase. Only BBN knew what it needed to successfully hook the IMPs up to the phone lines. But one would imagine BBN was still thankful to have DECCO to keep them somewhat insulated from this process.3

Two other contractors of note were the Network Analysis Corporation and the Network Measurement Center. ARPA wanted the Network Analysis Corporation to run simulations to help design optimal network topologies. In theory, this was reasonable. But, in Frank Heart’s estimation, the company did not prove as useful as ARPA expected in determining which sites to add next. In his eyes, practical matters such as whether or not departments were properly staffed or were ready to work on their IMP were mostly responsible for determining the network’s shape as it grew. The Network Measurement Center was run by Leonard Kleinrock at UCLA — the first node of the ARPAnet — and was responsible for taking a variety of measurements to study the network. The Network Measurement Center was, to Heart, somewhat academic in nature and not pivotal in the initial engineering or design of the ARPAnet. But it was useful in taking measurements pertaining to the health and efficiency of the network in its early stages.

Lastly, Honeywell was the subcontractor most pivotal to the contract’s success. BBN had no manufacturing facilities. Thus, they needed to contract with a computer manufacturer to have an IMP of their design actually built. At the time, few computer manufacturers did this. A standard company’s manufacturing process was not built to do wholly custom jobs. The processes were suited to be fiddled with, but not much more than that. But Honeywell had recently acquired a company called the Computer Control Corporation (CCC) that made them an exception. The team from CCC, as Heart put it, “understood a bit about the special systems business.” This was a unique enough skill that at least two contract bidders had used Honeywell as a subcontractor — BBN and Raytheon. Larry Roberts listed this as a reason that Raytheon’s bid was being considered alongside BBN’s.

Some of the Major Technical Problems

A variety of technical problems awaited the BBN team.

One of the major early problems the BBN team spent its time working through was the error control and correction problem. To work through this, the team was able to design special checksumming hardware that would be installed at both the transmitting and receiving end of the circuit. Every packet of information sent would have a 24-bit cyclic checksum appended to it which would be recomputed by the hardware at the receiving end to determine if there were any errors. If there was, the packet would be retransmitted.

How the nodes got there was the most theoretically-involved piece of this engineering experiment. The packet switching technologies we know, love, and hyper-optimize today were much more un-tested back then. This was one of two areas in which Robert Kahn’s presence proved extremely useful to the engineering-focused team. Kahn was able to quickly explain to them various aspects of packet switching algorithms as problems came up. (The other major theoretical area Kahn contributed to was in some mathematical aspects of phone line error problems.) The technical details of the routing algorithms the team initially deployed are described in the project’s completion report to ARPA as follows:

The approach taken was to use a distributed adaptive traffic routing algorithm which estimates on the basis of information from adjacent nodes the globally proper instantaneous path for each message in the face of varying input traffic loads and local line or node failures. Each IMP keeps tables containing an estimate of the output circuit corresponding to the minimum delay path to each other IMP, and a corresponding estimate of the delay. Periodically, each IMP sends its current routing estimates to its neighbors; whenever an IMP receives such a message it updates its internal estimates. Thus, information about changing conditions is regularly propagated throughout the network, and whenever a packet of traffic must be placed on the queue for an output circuit, the IMP uses its latest estimate of the best path.

Another major area of problems came from the technical aspects of the IMP-Host interfaces. The engineers spent substantial time hitting their heads against the wall to find ways to successfully connect IMPs to all different kinds of computers while, also, minimizing the amount of engineering work required by the node contractors to connect their machine to the IMP.

Solutions to all of these problems had to work in close to real-time. Individuals from remote machines were going to use the network to do much more than just send batch messages like emails. They were going to use their computers to remote into another computer as if the whole network was a single, time-shared machine. So, to make this all work at time latencies that made sense, the BBN team had to do things like “write very carefully tailored programs in machine language to optimize the capacity and the low delay of the data path in the switching node.” The team emphasized in its final report that, “Great attention was paid to minimizing the operating time of the inner loops of these programs.” Many areas of minutiae were all-important.

Many problems — from IMP design, to Host-IMP connection software and hardware, to error-correcting software — required this level of care. All of the high-level concepts — computers sending a message across a telephone line, linking two computers together via a communication line in a lab setting, basic message handling across noisy communications lines, etc. — had been demonstrated as feasible before. But BBN had to invent and engineer all that was required to make it go from technically possible to actually usable.

Viewed naively, the implemented packet switching technology was the team’s major “new” contribution. But, as most engineers will tell you, a project like this can be one big pile of unknown unknowns. That, in part, is probably why industry did not consider touching it in their R&D efforts. But, given the BBN staff’s continuous experience in recent years working on real-world time-sharing systems and real-time computer communications, they were probably the team that had the best sense of what the unknowns of the project might look like and how to deal with them as they came up.

The Honeywell Partnership

To get the IMP designed and built, the BBN team heavily leaned on Severo Ornstein. Ornstein had previously worked with Frank Heart and others on various computer projects at Lincoln Lab, worked with Wes Clark on smaller computers at WUSTL, and had since returned to Cambridge to work at BBN and do some lecturing at Harvard. Prior to the networking project, at BBN Ornstein had been working on projects related to the use of computers in education. Ornstein’s interest in the ARPAnet RFP caused him to dedicate more of his time to BBN.

While designing the IMP during the proposal process, Ornstein estimated that he and the team “did 90% of the design then.” But even when BBN had won the proposal, Ornstein still had to put substantial effort into working with Honeywell to get the computers made. Ornstein remembered:

The man from Honeywell who was assigned to build those interfaces from my drawings didn’t understand the drawings well and was not really careful. We ended up having to redo much of his work.

Much of BBN’s interactions with Honeywell may cast some doubt as to whether an industry actor could have carried out this contract nearly as effectively as BBN had. One account by Ornstein regarding the BBN team working their way through a confusing problem will help elucidate why exactly this is. The story both shines some light on what exactly the BBNers were spending their research time on prior to the implementation stages and how Honeywell dealt with problems. Ornstein explains:

They were not used to doing some of the kinds of things we were doing…For example, we discovered some design flaws in the machine. One of the reasons we chose the Honeywell 516 was that we thought it was a mature machine…that was not going to gives us grief. Well, we were wrong. We pushed it very hard…and we uncovered a bug that they could hardly believe, a synchronizer problem…Synchronizer problems are very, very subtle. We had to dig and dig and dig at them, and finally from their back room they produced a really smart enough guy — they do have a few — but, it was very hard to get this guy. We finally got him…when he came and we sat down and I finally had someone I could talk to who would understand what I was talking about and believe me. It was a subtle problem. The program would run fine in the machine for days on end, and at the end of three or four days suddenly the machine would just die, inexplicably. It was a very, very low frequency failure, so infrequent that you could never look at it on a scope; you could only see the effects afterwards. In fact, we had to build special hardware that beat on the trouble spot many, many times faster than normal usage would. Then finally, with all the lights out in the room, we could faintly see on the tracing scope an occasional failure. That was when the Honeywell people finally became convinced that there was a real problem…We showed them the trivial fix they could make to the machine…So, their people weren't absolutely top-notch people; they were okay. They were industrial-strength people, not research-strength people.

Of course, there were industrial operations with plenty of “research-strength” individuals back then — assuming Ornstein meant individuals with the brains and toolkits to consistently find answers to vexing problems like the one mentioned. Bell Labs, Kahn’s old employer, was just one example.

Not assigning this systems engineering-style contract to an academic department makes sense. Most academic departments are just not staffed and structured in a way to make projects like this feasible. But it’s unclear how an outfit like Raytheon would have done working through problems like the one described above. Would they have been able to solve them? If so, how quickly? We see that the Honeywell team was great at reproducing duplicate machines once they had already made one. But there were certain computer engineering principles and skills utilized in the project that some of the builders — from places like Lincoln Lab and Wes Clark’s WUSTL team — had much more experience with than most other groups. That was to BBN’s great advantage.

All of that being said, the formerly-university-employed engineering team still conceptualized this as more of a novel engineering problem than a research problem. Dave Walden, one of the project’s two founding software engineers who later became General Manager of BBN, described how the team viewed the problem as follows:

My view is that mainly what we were doing was we were very pragmatically doing engineering. We had to send these bits down the wire: how do you put a header on the front; how do you put a trailer on the back. There was theory of how you put error-correcting codes on it. Bob Kahn knew that theory and told us what it was. There were some constraints: this is the way that the 303 (or the 301 or whatever the Bell modem is) has to be interfaced to, but after that it was all pretty pragmatic. Not lots of theory of coming from someplace else.

The Team’s Management

Before diving into how the one-year contract’s implementation went, we should first get a better understanding of how the team was managed at BBN. Firstly, it was much smaller than one would expect.

As the first year of work progressed, the team remained small — about eight people. This is how Frank Heart preferred it. As he saw it, that small team of elite talent in the relevant sub-areas was all BBN needed. He described the project’s work style further, saying:

I think mostly I tend to believe important things get done by small groups of people who all know all about the whole project. That is, in those days all the software people knew something about hardware, and all the hardware people programmed. It wasn't a group of unconnected people. It was a set of people who all knew a lot about the whole project. I consider that pretty important in anything very big. So I suppose if you call it a management style, that would be something I'd state. I think also that they were a very, very unusually talented group. I think things tend to get done best by small groups of very, very good people — if you can possibly manage that. You can't always manage it. So if you again want to call it a management style, it is to get the very, very best people and in small numbers, so they can all know what they're all doing.

Heart was extremely averse to the team growing so large that communication had to be done on paper. He preferred their style of “very, very frequent interaction on problems.” As Walden remembered, “I don’t remember such a thing as we had a weekly progress meeting. We probably were more in tune with the progress than that; we probably did it hourly.” Heart and the entire, small team did their best to stay technically involved with each other’s work. Heart, as a rule, insisted on technically understanding every bit of the project. The individual team members attempted to do the same, which they felt was quite easy to do. The team all had offices and work stations immediately in the vicinity of each other. As things came up, they’d simply go talk with the relevant individual or call an impromptu meeting.

The Early Node Installations

With some level of difficulty, BBN worked their way through problem after problem like those in the technical problems and Honeywell sections. With a lot of persistence, the team got the IMPs specified in the proposal built and de-bugged. By September of 1969, the hardware and software for the first IMP was ready for installation.

Unlike many ARPA computing contractors, the UCLA department had a core of individuals who were particularly excited about the ARPAnet project. While many of the professors who owned computers saw the ARPAnet as an opportunity for ARPA to force them to further split their computer time, Leonard Kleinrock and others at UCLA had research interests relating to the network itself. Kleinrock, most notably, had written about the theoretical aspects of packet switching in his prior work. Because of this interest, he formed the Network Measurement Center to monitor certain aspects of the network as a subcontractor for the ARPAnet contract.

In early September 1969, BBN sent its first IMP off to UCLA. When shipping the finished IMPs to the host sites, BBN even went as far as to have staff ride alongside the machines on the plane, just in case. For these early sites, Ornstein and some of the software researchers like Dave Walden traveled to the sites to hook the IMP up to the phone lines and work through any setup issues. In the lead-up to BBN sending the machines, the firm shared a set of specs with the host sites so the sites could do as much work as possible beforehand. On the node end this work was usually led by some kind of hardware lead and software lead.

BBN had apparently done quite an effective job of communicating its technology to UCLA. Also, UCLA had apparently done a good job of internalizing the information and building accordingly. According to Leonard Kleinrock, within a day of initially connecting the IMP and UCLA’s SDS Sigma 7, messages were moving back and forth.

This was a massive success. Still, many questions could not be answered until the second installation was done. Only at that point could BBN see what happened when two nodes tried to communicate with each other. The next month, the BBN team installed its second node at the Stanford Research Institute — which had also been subcontracted by ARPA to do some research on the network itself. Once BBN and SRI had the IMP hooked up, the two node research teams attempted to send the first message over the ARPAnet. As Leonard Kleinrock remembers:

On October 29th, 1969, one of my programmers, Charley Kline, and I were in this room and we decided to log on to this [SRI’s] machine. Now, to log on, one has to type in “LOG” and the remote machine will type in the “IN” So our job simply was to type in “LOG.” Now, up at the other end, there was another programmer waiting to watch all of this. And we had a telephone connection between these two…so they could talk to each other. What happened is Charley typed the “L” and he asked, “You get the ‘L?’” And the answer was, “Got the ‘L.’” He typed the ‘O.’ “You get the ‘O?’” “Got the ‘O.’” He typed the ‘G.’ “You get the ‘G?’” Wacko! The system crashed. This machine [SRI’s] went down.

So the very first message on the internet ever was “LO,” as in, “Lo and behold!” You couldn’t ask for a better, more effective message.

Surely the BBN team considered it enough of a success that any letters were sent at all. The team proceeded to install a new IMP per month for the rest of the year, making four in total. Problems continuously came up through both BBN’s testing and the node researchers’ testing. BBN fixed the problems as they came up. It does not seem like any of these problems proved particularly devastating.

Kleinrock excitedly described the arrangement as follows:

So we had a four-node network running, and we began to test it and find out what some of the problems were. And we could break this network any time we wanted, we found faults, we had BBN test it, etc.

As problems were worked out, there were regular hardware modifications and software releases that had to be rolled out. The software releases were done, initially, by sending Dave Walden around to each of the sites with his paper tapes to get the software working on the machines. In this way, the first batch of nodes became operational and steadily improved. BBN did this on budget and within the one-year timeline of the contract. Larry Roberts and ARPA were, surely, extremely pleased with the progress. By the end of the year, ARPA contracted BBN to expand the network to 19 nodes.

Before discussing how the project scaled, let’s briefly explore Larry Roberts’ working relationship with the BBN team. Roberts was a very active manager in the ARPAnet project, not just its funder.

Working with Larry Roberts

In putting together the RFP for the ARPAnet, Roberts had looked deeply into many technical points and consulted the best members of the computing community. In doing this, he made several key decisions before the contract was ever in BBN’s hands. As Heart put it, “They [DARPA] picked the baud rate of 50 kilobits, they picked the sites, they picked the issues about the checksums. A lot of the work had been done by DARPA in advance.”

As Dave Walden framed things, BBN could surely be considered the ARPAnet’s operator and the primary driver of the engineering work, but ARPA was the high level manager. This entire operation, the ARPAnet itself, was meant to facilitate new capabilities for ARPAnet contractors. So, Roberts providing a constant guiding hand made sense. Also, the ARPA contracting community contained some of the best minds who could make technical suggestions to the BBN team. So, Roberts and ARPA continually facilitated meetings between BBN, node contractors, and others as they saw fit. The now-famous Network Working Group which debated many ARPAnet-relevant ideas, like what later came to be the TCP/IP protocols, was one example of these efforts.

Once the project was underway, Roberts largely left BBN in charge of making day-to-day decisions. They did not need to seek his approval on many things. But he did keep close tabs with the team and would raise objections from time to time. Roberts could technically “pull rank” on BBN as the project’s funder. But the arrangement did not really work like that. If Roberts didn’t agree with the team’s approach to a particular problem, they would discuss it over the phone or in person until they could come to some kind of agreement. In a contentious case, that might take some number of days. But, in most cases, a clear agreement was reached.

The BBN team held extreme respect for Roberts as an engineer. In turn, Roberts seemed to respect the BBN team greatly. Roberts had a direct line of communication not just with Heart, but with individuals in charge of component technologies on the team such as Severo Ornstein. According to Heart, normally a week wouldn’t go by where Roberts didn’t talk with somebody on the team. Ornstein, who was particularly casual, said he simply conversed with Roberts as he would any other colleague.

Beginning to Scale the Network

The initial contract to build the four-node network was a success. So, ARPA opted to give BBN a contract to spearhead the effort to 19 nodes.

In early 1970, BBN itself became the ARPAnet’s fifth node as the network continued to expand at roughly one node per month.

As the expansion continued, new engineering problems continually arose. While Frank Heart was known to be an extremely defensive engineer, things were still bound to work imperfectly. This whole project was an experiment and was a first-of-its-kind network, so one could only be a certain level of defensive without halting the project’s progress entirely.

In many areas, the BBN team shipped technology that it initially felt was merely good enough — with the understanding that they could continually improve component technologies on an ongoing basis. One case of this was the network’s protocols. The initial NCP protocols were deemed by the Network Working Group to be quite sub-optimal.4 So, over the years the NCP Protocols protocols were improved until they were entirely replaced by Kahn and Cerf’s now-famous TCP/IP protocols.

As the ARPAnet completion report notes, on occasion, some of the problems present in the operational network were more than small. In 1970, a major problem was demonstrated with IMP flow control and storage allocation. Node hosts, if they didn’t heed the error, could accidentally halt the network’s entire operation. Yet, while BBN and others took the many months required to fix the problem, the ARPAnet continued to provide adequate service to its nodes.

As Dave Walden recounted in his oral history:

Walden: …it almost immediately became obvious to anybody who hadn't believed it before that what we had originally implemented was insufficient. And what we had specified to the host and what they had implemented had a lot of ramifications. They were now done and now it didn't work all that well. So for the first year or two years, quite a long time, there were rules, informal rules in the network that you didn't pump stuff into it from the hosts so fast it swamped the net. Because we all knew the algorithms didn't work, I mean they broke, patently they broke when you pumped stuff at them too hard. And so this interim solution worked in that network — this is the kind of thing that somebody who is working on proving programs correct and stuff can't imagine. It actually was quite useful in those early days of experimentation because everyone agreed not to break it. You know how to break it; let's not break it. Meanwhile, let's go figure out how to fix it…

Interviewer: So it was on the order of a gentlemen's agreement. Just not to go break it.

Walden: Engineers' agreement.

Given that all the users were well-trained and capable engineers and researchers themselves, BBN could also get away with only gradually scaling up its service capabilities. Dave Walden running around with his tapes to implement software updates was one example of this. As the network expanded and the network began to operate more reliably, BBN was able to dedicate more of its team’s efforts to helping the network operate a little more automatically.

In these middle years of the project, BBN also began to expand its efforts to finding more efficient solutions to manage network issues, answer questions from nodes, etc. In the beginning, BBN’s answer to this problem was to throw Dave Walden at it. As Walden fondly remembers:

Up until September of '70, I had the program listing beside my telephone at home and would get telephone calls at home whenever something stopped. My telephone number was quite literally on the front of the original packet switches and they called my house over here in Allston and I would talk to whoever was there, using my listings. So that's what happened.

With ARPA sponsorship, BBN began running the Network Control Center (NCC) in service of this goal. In building up the NCC, BBN designed and implemented tools and software to regularly ping the network and keep tabs on network problems, monitor the status of hosts, etc. Alexander McKenzie became the “ARPAnet generalist” BBNer put in charge of the customer service aspects of the project as it grew. McKenzie proudly recounted stories of calling telephone companies and telling them a line was about to go down…and then it would. The companies were quite impressed because this was not an ability that they had yet.

Starting in 1972, BBN hired staff to offer de-bugging and other customer services to network nodes full-time — three people worked the day shift, two the late evening shift, and one the midnight shift. Through BBN’s and the NCC’s efforts, nodes were soon able to (in most cases) automatically report their status. Maintenance and debugging were then typically carried out remotely from the monitoring center. As the NCC’s capabilities grew, BBN was able to successfully place IMPs at locations where the staff had less and less computer training.

Additionally, as the network expanded BBN conducted research in an attempt to expand the types of IMP technology as well as the types of communication lines IMPs could be hooked up to. In 1971, BBN finished designing its first terminal interface message processor (TIP). These terminals were able to connect to the ARPAnet and do work on ARPAnet host machines without having to be run on a host computer of their own. On their own, TIPs were more expensive than IMPs. But the idea was extremely exciting to ARPA because the TIPs becoming operational meant one could hook up to the ARPAnet without having a large computer on hand. The first two TIPs, shown below, were installed at MITRE and NASA’s AMES research facility. The early 1970s saw the first TIPs installed, IMPs connecting to the network via satellite technology, the design of standard IMPs ten times more powerful than the initial ones, and the deployment of Ornstein’s Pluribus multi-processor IMP.

As the ARPAnet hit its 19th node, ARPA had already opted to give BBN a contract to expand the network to over 50 nodes. The first two years of the project had all occurred largely on schedule and on budget. With this next extension, the network even began to include nodes from remote geographic areas like Hawaii and Norway. This required the use of some of the project’s new technological capabilities as well as the project’s first interactions with foreign telephone operators. While these remote nodes did tend to be more buggy, they were largely functional and proved some of the new capabilities to be useful.

Throughout this period, Larry Roberts’ continual arm-twisting of node sites to do what they needed to do to make the node installations a success proved extremely useful in keeping things on track. As many of the sites’ primary computing funder, he could exercise this power quite easily. By 1975, the ARPAnet project was clearly on stable footing. So, responsibility for the project was transferred to the Defense Communications Agency (DCA). According to DARPA Historian Richard Van Atta, at the time the project was transferred to DCA the total budget outlays throughout the project had been $25 million in total (~$150 million today). That number covers the entire project, not just BBN’s portion. By anybody’s metrics, this should be considered a remarkably cheap success.

The First International Conference on Computer Communication

How exactly the outside world — meaning those in the technology community not affiliated with an ARPAnet host site — became aware of the ARPAnet’s success deserves a brief explanation. Even in 1972, many in the communications community doubted this cross-country packet switching network idea would be technically possible anytime soon.

Robert Kahn’s final project as an official member of BBN helped rectify this. In October 1972, he put on a massive demonstration of the ARPAnet at the first-ever International Conference on Computer Communication (ICCC). The conference itself was an extremely useful forcing mechanism for BBN and each of the node contractors. This provided tangible pressure, with a concrete deadline, to thoroughly debug many pieces of technology, sure-up certain protocols, etc. The academics responded much better to the deadlines or threats of their funding being diminished than they did to the mere presence of bugs in the system. Additionally, the existence of the conference provided a push to ensure that as many host sites as possible were functional across as many use cases as possible. To this point, many nodes had only been outfitted with the hardware and software to use some of the ARPAnet’s possible applications.

Dozens of nodes contributed computer hosts to the demonstration. The conference was a sort of DIY showcase where individuals could hook up to the computer hosts via TIPs and work with them the way node users did every day. This could be for things as simple as e-mail, a rapidly growing use case of the users, as well as much more complicated programs. As Roberts recalled:

I think this was the watershed event that made people suddenly realize that packet switching was a real technology. We had thousands of people who went through that — I don't know what the exact number is — at least a thousand who went through that particular exhibit.

Or, as the ARPAnet completion report put it:

The demonstration itself was a spectacular success; with everything working amazingly well, many visitors remarked that the ARPANET technology "really is real."

ARPAnet Results

On a technical level, the ARPAnet experiment was a success in many ways. The project proved that a large network could be built in such a way that node failures were localized and did not crash the rest of the network. It convincingly demonstrated adaptive routing algorithms and packet switching theories. To a large extent, the ARPAnet proved that a decentralized network like this could run itself without a central command center. Finally, from a user perspective, the IMPs and TIPs proved reliable and easy enough to use to facilitate the time-sharing of diverse computers, thousands of miles from one another, for a variety of tasks. The working ARPAnet also succeeded in its role as a test bed for the entire field of computer communications technology research.

But, of course, the success of the ARPAnet project goes beyond the technical successes of the project. As BBN’s ARPAnet completion report stated:

The ARPANET project also proved the feasibility of achieving closely knit communities of technical interest over a widespread geographic area; it is possible that this social feasibility demonstration is as important as the many technical feasibility demonstrations.

Possibly the clearest indicator of this growth is how rapidly internode traffic grew over the network through the mid-1970s.

As the usage of the network grew, BBN feared that even the new algorithms being used to handle routing, flow control, and congestion control would not be able to keep up. But, as the team noted, “Luckily, the improvements in the algorithms managed to stay slightly ahead of the growth in network size and traffic.” In BBN’s estimation, the diminishment in growth rate was because at a certain point, the existing host machines being accessed over the network were simply reaching capacity.

Throughout this growth period, acute problems present in the network lessened, BBN sharpened its processes, demand increased, and more research was being done that relied on the network in some way. Several of the BBNers, in their oral history interviews, talked excitedly about how the network gradually became a “utility.” When prompted by the interviewer to say more, Heart explained:

A utility is something people depend upon. Like the electricity, or the phones, or the lights, or the railroads, or the airplanes. Yes, it was a utility. That's the thing that was the amazing surprise. It was started as an experiment to connect four sites, and it became a utility much, much faster than anybody would have guessed. People began to depend upon it. And that was a problem, because that meant when you changed it, or it had problems, they all got mad. So that was a two-edged sword. But it was also very exciting.

Alexander McKenzie noted that, while the ARPAnet was never as reliable as a power company, the team did eventually succeed in keeping the IMPs up 98% or 99% of the time. As this transition to “utility” took place, NCC-style activities took more of an emphasis. Research and operational activities that optimized the reliability of the network slowly became prioritized over network research activities that created knowledge but interfered with service. This was likely disappointing to some BBNers. But it can also be seen as merely a symptom of the project’s success.

The network was becoming so convenient that a substantial percentage of network traffic was intra-node traffic. Computer hosts at one site were communicating with others in the same location. In other words, communication over the ARPAnet had become, for many tasks, preferable to even walking next door to see a colleague.

In 1972, BBN even began attempting to form a sanctioned spin-off to commercialize the technology in the wider market. ARPA was spiritually supportive of this effort even if they did not fund it. In fact, helping get this firm, Telenet, up and running was Larry Roberts’ first job upon leaving ARPA. He had gone to ARPA to make inter-computer networking a reality. Upon leaving, he was heading into a role where he could attempt to get the technology to as many users as possible. Telenet would go on to have a level of success as a growing network provider — but would not be considered a smashing success. The fact that the ARPA-funded technology showed commercial potential as little six years after the project’s initial funding should also be considered a major success.

If one wanted to nitpick, one could point out that ARPA’s major intended application of the network was as a way to reduce computing costs. This never did become the primary use of the network. As computing costs gradually decreased, this particular use of the ARPAnet became relatively unimportant. The network found better uses.

ARPAnet Lessons Learned (and Caveats)

Many small-scale, PM-relevant lessons are present in this report. As one example, the BBN team maintaining extremely close connections with the academic community while implementing sooner rather than later proved to be a key decision. The academically-driven Network Working Group, with its debates on network protocols in the group’s early years, was at a stalemate regarding what protocols were best. The BBN team could have waited far too long to get the perfect set of protocol recommendations from the group. Instead, BBN implemented the NCP protocols and gradually, with much input from the group, upgraded to the much-improved TCP/IP protocols. Another interesting lesson was Roberts’ use of a demonstration as a forcing function to get all the nodes outfitted with the full suite of ARPAnet capabilities.

Minor learnings aside, it is important that the reader fully appreciate just how important having a contractor like BBN was to the success of the project. Not only was the research and prototyping stage of this difficult project done well and on schedule, but BBN seamlessly transitioned it into a utility. Appreciating why they were able to do this is extremely instructive.

The BBN team was staffed with top minds who cared deeply about the research enterprise but operated within the structure of a firm. Dave Walden, explaining why exactly BBN had earned its nickname as the “third university of Cambridge,” emphasized the following:

BBN provided a support structure for self-motivated researchers who were able and willing to find sponsors for the research they wanted to do. The work was not directed in a top-down fashion; within broad limits, senior researchers were free to pursue their own interests, if they could find the necessary financial backing.

To complement this, BBN also had the necessary staff, experience, and structure to manage successful projects as they grew into larger implementation projects. The firm’s hospital time-sharing work is just one example of BBN managing an implementation project with cutting-edge technology. A project at BBN could start in the deeply exploratory research stage and naturally scale into a utility.

The ARPAnet project showcased exactly what BBN was optimized to do. When given the right kind of problem and the right situation, BBN’s model could help facilitate marvelous feats. The ARPAnet itself, just like Edison’s light bulb 90 years prior, proved to be both a field-changing test bed for scientific experiments as well as a world-changing commercial technology. BBN was ideally suited to make a breakthrough of this nature possible. While the mission creep that comes with a profit motive can often steer a commercial firm off of the course of truly ambitious research, BBN was less susceptible to this in these years. If its research somehow became too derivative, the firm would not have been able to recruit staff anywhere near the competence level it sought.

If every scientific/commercial area had a small research firm like BBN, the scope of what would become possible with an ARPA-like PM’s funding would expand greatly. In one’s role as a PM, if the opportunity to help incentivize or facilitate an organization to operate a little more like BBN, there seem to be few downsides — and massive upside — to do so. The more novel a systems contract becomes, the more suitable it is for a firm like BBN.

BBN created such a compelling, applied environment in which to do research that they convinced many professors to give up tenured positions at MIT to come work at the firm. The firm knew exactly how to make use of these bright, applied researchers. Dave Walden put extreme emphasis on how much the “native wit” of the initial technical team he joined — Frank Heart, Severo Ornstein, and Will Crowther — was required to make the project a success, saying:

They are smart guys. There wasn't much theory in how you build a packet switching network. There was a communications theory, but that was all pretty abstract. One just got out there and did it. All the stuff that is now taught in courses in communications about networks and protocols and all of that, I would say we were mainly (as part of the entire community of the host people) inventing it. The academic analysis tended to come later in a lot of cases.

I’ll conclude with Frank Heart’s fond words describing the organization in “A Culture of Innovation:”

First and foremost, BBN was a great place for a technical person to work, and most people really liked working there. It was a middle ground between academia and the commercial world, with the meritocracy and individual freedom of the academic world, along with the potential reward structure and potential impact on the world of commercial ventures. In the case of the ARPAnet and the subsequent explosion of network activity, it was an extremely unusual opportunity for a technical person to “ride a rocket” of change in the world.”

Thanks for reading:)

Share

Subscribe now

If there is demand, I can write a separate piece on how BBN’s culture and management emphasis changed from those described in the piece as the decades wore on. But I considered that exploration out of scope for this particular piece.

Pattern Language Tags:

• Utilizing a contractor made up of individuals with research-style goals and training working within a ‘firm’ structure.

• Building out an experimental test bed for a research field.

• Promoting a coordination/service mechanism to reduce material costs and increase research feedback cycles.

This piece is a part of a FreakTakes series. The goal is to put together a series of administrative histories on specific DARPA projects just as I have done for many industrial R&D labs and other research orgs on FreakTakes. The goal — once I have covered ~20-30 projects — is to put together a larger ‘ARPA Playbook’ which helps individuals such as PMs in ARPA-like orgs navigate the growing catalog of pieces in a way that helps them find what they need to make the best decisions possible. In service of that, I am including in each post a bulleted list of ‘pattern language tags’ that encompass some categories of DARPA project strategies that describe the approaches contained in the piece — which will later be used to organize the ARPA Playbook document. These tags and the piece itself should all be considered in draft form until around the Spring of 2024. In the meantime, please feel free to reach out to me on Twitter or email (egillia3 | at | alumni | dot | stanford | dot | edu) to recommend additions/changes to the tags or the pieces. Also, if you have any ideas for projects from ARPA history — good, bad, or complicated — that would be interesting for me to dive into, please feel free to share them!

Specific Links:

The following oral histories and related materials were a joy to read through and primarily what the detailed ARPAnet portions of the piece were based on. The ARPAnet Completion Report was simply used to fill in the gaps. These give a far more detailed account of how things actually worked on the project than the completion report ever could.

• Frank Heart’s UMN Oral History

• Larry Roberts’ UMN Oral History

• Larry Roberts’ Computer History Museum Oral History

• Robert Kahn’s 1989 UMN Oral History

• Robert Kahn’s 1990 UMN Oral History

• Dave Walden’s UMN Oral History

• Severo Ornstein’s UMN Oral History

• Alexander Mckenzie’s UMN Oral History

  • Contains many insights on the customer/user service mechanisms BBN utilized as the network grew in the early-to-mid 1970s.

• Robert Taylor’s UMN Oral History

  • Taylor dives into why exactly he felt his tenure as IPTO chief was a good time to attack the network problem and how he went about the early stages of the work.

• Leonard Kleinrock explaining early ARPAnet node installation and testing

• The History of Telenet and the Commercialization of Packet Switching in the U.S.

  • For more on the early history of commercial packet switching, check out this IEEE paper in which Larry Roberts is a co-author

• Licklider et al.’s Libraries of the Future Report

• J.C.R. Licklider Oral History

General Links:

• ARPAnet Completion Report

  • This report was a treasure trove of information related to contracting details, specifics on technical accomplishments, and various graphics summarizing ARPAnet’s growth.

  • The report also highlights how the actual work deviated from the plan, what was added/removed, etc. as the project went on.

  • Lastly, the report lists many of the technical problems the BBN team had to work through at various points.

• A Culture of Innovation: Insider Accounts of Computing and Life at BBN

  • Besides being the source of many quotes and pieces of information in the piece, this is the most thorough account of how BBN actually operated on a day to day basis. Each chapter is written by a different, long-time (and often high-ranking) BBN employee.

  • As one example, the book contains a long section from Frank Heart on how overhead rates impacted project selection and how BBN’s billing system worked. Explanations like this for all sorts of aspects of the operation are included in the book. 

Pattern Language Tags:

• Building out an experimental test bed for a research field

• Pushing forward the technological base via speculative machine building contracts

• Putting out contractors for specific “integrators” in addition to prime contractors and basic researchers

• Using customer-relevant applications as a demand pull mechanism for a research field

• Setting up your testing environment in the same machines and locations you will deploy in

Introduction

DARPA’s customer relationships and the individual agency of its PMs are possibly the two biggest differentiators of the DARPA model from bigger budget science funders like the NSF and NIH. Even in DARPA’s most basic research funding decisions, the question of “how can this be used by the military services?” finds its way into the discussion. In some generations of the organization, being “useful to the military services” is interpreted quite broadly, with justifications such as, “what’s good for the nation will be good for DARPA.” This approach was often taken by DARPA’s Information Processing Technology Office (IPTO) in the 1960s. Arthur Norberg and Judy O’Neill — who wrote an in-depth history of IPTO from its earliest days, including interviews with prominent individuals such as Robert Taylor — summarized the priorities of early IPTO directors as follows: “A contribution to the country which included a contribution to the military is what IPTO directors sought.”  With work such as the development of VLSI computer chips, that ethos has often proved to be an extremely correct heuristic.

However, in different generations of the organization — whether driven by wartime, fiscal austerity in DC, or the preferences of individual directors — being “useful to the military services” has been interpreted more narrowly. The first prominent example of this is DARPA under Director George Heilmeier. Another example of an era of DARPA where this goal was interpreted more narrowly was during Tony Tether’s years at DARPA as Director — starting in 2001. In this era, Tether went as far as to make many funding decisions contingent on a specific armed service putting in a budget wedge to continue developing the project past a certain stage. Neither approach is right or wrong. ARPA-like orgs often need to be able to adapt to changing situations like these.

Regardless, whether your customer is the US military, medical patients, or a certain subset of academics, all ARPA-like orgs serve some customer. This piece will dive into how DARPA’s Strategic Computing Initiative (SC) served its military customers in the 1980s — an era in Washington DC that demanded DARPA justify most of its spending as having clear, near-term value to the armed services. 

Context

The three projects covered in this piece are all examples of DARPA’s attempts to find early military use cases of the exciting advances that had been coming out of their computing portfolio since the 1960s. Some pieces of the IPTO portfolio clearly followed Robert Kahn’s — who headed IPTO at DARPA — vision of “pushing” technology out into the world to find applications. Meanwhile, other areas of IPTO’s portfolio relied more strongly on using projects to stimulate demand “pull” for technological capabilities. 

Robert Cooper — the DARPA Director at the time — was the counterbalance to Kahn and his beliefs. The program that Kahn envisioned was, put plainly, not considered sellable in DC. To sell the program, Cooper helped center the pitch around three new military systems that would result from the program if its individual research programs panned out in a moderately successful way. The three early descriptions of the applications became the centerpiece of the report that pitched SC to Capitol Hill. These applications were meant not just to provide the non-technical politicians a clear sense of what they were paying for, but, also, Cooper wanted to use the applications to help steer IPTO’s research funding in what the SC management hoped would be a coherent and politically explainable direction. 

Beginnings of All Three Programs

From an intellectual standpoint, Khan was considered the figurehead of DARPA’s 1980s Strategic Computing Initiative. The initiative was meant to expand on the prior decades of computing research breakthroughs as well as find useful military applications for the growing body of work. However, it would be DARPA Director Robert Cooper  — who had a Ph.D. background but favored industry-focused, goal-oriented engineering research more than Kahn — who would succeed in selling the program to Washington. A large part of the reason he proved to be a better fit for this sales task than Kahn was because he, more than Kahn, was willing and able to outline specific application technologies that SC’s technology portfolio could contribute to once all the work had been done. Khan felt that technological development was uncertain and that doing this was something of a fool’s errand. Cooper also seemed to understand that what they did at DARPA was an uncertain enterprise, but, also, that if DARPA wanted an expanded computing budget, then this was the kind of pitching that DARPA would need to do.

Roland and Shiman — who wrote a history of SC and interviewed many of its key players in the process — described Cooper’s thought process:

“Technology base” sounds fine but it does not mean much to those outside the computer community. Just what will that technology base allow you to do? What is its connection to the real world? How will it serve the defense mission? What new weapons and capabilities will it put in the hands of the commander in the field? These are the real metrics that allow you to measure the worth of such an investment. And any such goals have to be time specific. It is not enough to say that a robotic pilot’s assistant will be possible at some time in the future; there has to be a target date. Between now and then there must be benchmarks and milestones, metrics of progress demonstrating that the program is on course and on time.

Clinton Kelly, who originally joined DARPA as a PM in its Cybernetics Office before moving to the Defense Sciences Office when Cybernetics shut down, was asked by Robert Cooper to help define the applications efforts. Kelly, an engineering Ph.D. who had also run a company related to automating intelligence decision-making, had been funding a variety of somewhat odd mechanical machines while at DARPA — such as a robot that could do backflips. Kelly was known to be a great technology integrator and organizer, so he was very well-suited to the task Cooper had assigned him. The three applications that Kelly helped outline incorporated what Kelly and others saw to be the likely capabilities that would come from Kahn’s technology base and would intrigue the military services. The three major SC applications that Kelly helped jumpstart through his work on Cooper’s report were Pilot’s associate, Battle Management, and Autonomous Land Vehicle.

Progress towards these three applications would often be used to help steer and benchmark the program in the coming years. Many of the benchmarks included in the original SC report — such as real-time understanding of speech with vocabularies of 10,000 words understandable in an office environment or 500 words in a noisy environment — were set with the needs of these applications specifically in mind. Stated goals like helping a fleet commander plan for multiple targets or providing the military the ability to send an unmanned reconnaissance vehicle across a minefield helped ground the vision of the program to its funders and customers who were less likely to see the importance of the stale computer benchmarks — such as those used in the computer architecture section of the SC write-up.

While Khan eventually relented, he still saw these applications as Cooper imposing a NASA-style of operations on DARPA. NASA focused on working towards challenging point solutions rather than broad, flexible technological development — as IPTO historically had. As the years went on and work progressed, work on these projects proved that Kahn and Cooper both had good points. Regardless, the applications proved successful in helping DARPA win over an expanded computing budget for SC. In 1983, work on the program commenced.

Pilot’s Associate Operations

The Pilot’s Associate (PA) application was meant to assist aircraft commanders in all cockpit decision-making and planning that could reasonably be augmented by sensors and then-modern computing. The PA planned to incorporate the new hardware, AI/software, and natural language understanding capabilities expected to emerge from DARPA’s funding of the computer technology base. DARPA hoped that the PA would eventually help the aircraft commander perform tasks like preparing and revising mission plans. They hoped for the system to be small enough to fit into a cockpit and outfitted with a natural language understanding system so a pilot could deliver some verbal commands while they used their hands to operate the aircraft. The PA would be a “virtual crew member.” There were even comparisons, at the time, to the system functioning as a pilot’s own R2-D2 unit — although others in the military pushed back on this specific analogy.

Once SC was announced, DARPA’s first step in working on the PA application project was contracting Perceptronics — a research company that DARPA was currently using as a contractor for its SIMNET computerized combat simulation training project — and other contractors to carry out exploratory studies in the area. These studies were meant to more concretely outline how the system would work, as the description of PA in the original SC plan was quite vague. These studies established that, most likely, five separate “expert systems” could work in concert to carry out the functions of the final PA application. 

In the era of strategic computing, “expert system” was a common term to reference software programs — often built by combining the efforts of programmers and specific domain experts — which used codified sets of rules to help carry out tasks in a faster and more accurate fashion. In their simplest forms, expert systems might just be made up of meticulously constructed “if, then” rules. Expert systems were, at the time, considered a branch of AI research — albeit a less exciting one than more generalizable AI systems to AI research purists. However, in an era when generalizable AI was often not living up to any level of commercial promise, expert systems programs, which nowadays would simply be considered “software programs,” were proving interesting to the military and large companies alike. These programs showed real promise in bringing the fruits of the AI research community to a wide variety of sectors and customers. Unlike more pure AI systems at the time, expert systems work had actually even proven profitable in the private market.

So, it is not surprising that PA’s project planning was going to rely on the use of the more application-ready expert systems paradigm. In the case of PA, the preliminary studies determined that the five specific separate expert systems that would make up the final software of the machine would encompass the following areas: mission planning, systems status, situation assessment, tactics, and front-end communication with the pilot. As the PA project went on, most of the direct planning for the PA system was done by scientists and engineers at the Wright Aeronautical Laboratory at the Wright Air Force base in Dayton, Ohio.  The Air Force’s excitement to largely take the reins of the project from its early days was likely due to the fact, in the years leading up to this point, the Air Force had begun developing its own labs to explore the application of AI to Air Force matters. Major Carl Lizza, who had a masters degree in computer science, was the program manager on Wright’s end for much of the project. On DARPA’s end, John Retelle was the program’s PM until 1987, when another DARPA PM took over. With high level buy-in and excitement from the Air Force, minimal direction was needed from DARPA on the direct development and implementation of the PA system. The customer buy-in from the Air Force was a quite positive sign for DARPA. 

Following the study stage of the project, the implementation of the PA program began in 1986. McDonnell Aircraft Company was the prime contractor for the air-to-ground system. Three total companies made up this portion of the contract. Lockheed was the prime for the air-to-air system. Eight companies and university research teams were on that contract. McDonnell was awarded $8.8 million (~$23 million today) from DARPA and Lockheed $13.2 million (~$38 million today). Each company was also willing to cover 50 percent of the costs of the development work because they felt working on the contract would give them an advantage in bidding on the big-budget contract for the Advanced Tactical Fighter which would be awarded in the coming years. These performers would look to take advantage of the equipment and methods that had been coming out of the DARPA portfolio as well as industry in recent years — employing LISP, Symbolics, and Sun machines as well as many of the new expert systems development methods which DARPA often funded.

The project would also make heavy use of expert boards in planning its work schedule as well as work with the customers in developing successive versions of prototypes. As is common in military R&D applications projects, the team leveraged expert advisory boards from academia and industry on an ongoing basis. Help from these sorts of boards seems less common when making small decisions in projects further away from implementation — such as those made in the Defense Sciences Office in comparison to the systems technology offices. Also, as this was absolutely a product for a paying customer, the performer teams heavily involved the users of the eventual product throughout the design process. In fact, much of the development work by the industry contractors looked quite similar to modern software development to modern readers. The project relied heavily on the rapid prototyping approach to managing its workflow and benchmarking its progress. Several demos were set to happen in the first three years of the project that would prove the system’s function and value. After several successful demos and the proof of concept were completed, the work could then progress to doing the development work to ensure that the system worked as needed in live combat situations. From a high level, that was how the project was meant to progress.

Non-PA, but PA-relevant research and engineering work that was in earlier technology readiness stages was overseen by other PMs in the SC portfolio. This bit of bookkeeping is of practical importance to how the projects were managed. The PA-relevant work on the broader technology base — that was still related to the application and might, at some point, make it into the PA system — generally fell to PMs in IPTO. These PMs working on other pieces of the broader portfolio did not “answer to” the PMs of the PA, Battle Management, or ALV applications, in spite of the importance of those three projects to SC. The work in these other PMs’ portfolios, of course, was meant to prove useful to projects like the PA. However, the PA team did not have any direct power to influence other PMs’ priorities to be more in line with PA’s direct needs. The PMs in the broader SC portfolio were meant to help develop better component technology to help projects like PA in the future, but they were generally able to decide on the best possible deployment of their own portfolios. 

As the PA project was in its early planning stages, it seemed clear that a sufficient level of expert systems knowledge and experience was generally available to make an application like PA feasible at the time. But at least two notable technical risks existed that the PA team was relying on developments from the broader SC portfolio to overcome. The first risk was the need for a computer to be developed that was powerful enough — and small enough — to make the system work in real time. It was not going to be clear exactly how much more powerful a computer would need to be to make the PA work until more expert systems development was done by contractors on the project. But it seemed clear that a computer about an order of magnitude more powerful might be necessary before the system could be considered deployable in a cockpit. The second area of notable technical risk was the need to develop a natural language understanding system that could process a basic vocabulary of at least 50 or so words in noisy, cockpit-like conditions. This area of technical risk was likely less important than the issue of computing power, but it was still quite important if the system was going to work the way those in charge of the program wanted it to. 

In spite of there being notable areas of component technology that required improvement, Director Cooper felt it was important that the SC applications programs begin roughly at the same time as its work developing the broader technology base. Politically, this increased SC spending package was meant to be about applications. Waiting years to start working on the applications projects was not really an option. So, work on the three applications projects began, more or less, right after SC was announced. The SC team, however, did go out of its way to ensure that the more basic research projects got off and running sooner rather than later. Meanwhile, the applications projects often had study stages and preliminary planning periods that may have taken a year or two. I do not get the sense that DARPA made similar attempts to speed this process along the way they did to ensure that computer architecture researchers, for example, got the machines they needed for their research sooner rather than later. 

In March 1988, the first major operational milestone for the PA project was hit when the Lockheed Aeronautical Systems Company successfully demonstrated the system it had developed in a non-real-time cockpit simulation. In this demo, the system successfully performed flight tasks like identifying and dealing with a stuck fuel valve, creating a new mission plan as the situation changed, advising the pilot on evasive action, and deploying flares. The machine was able to complete the simulation’s 30-minutes worth of real-time computations in 180-minutes — meaning, in this early stage, the system was about one-sixth as fast as it needed to be. The slow performance was not considered behind schedule by any means. Firstly, much of the software was not even run on the most powerful machines possible at this stage. Additionally, in the early internal writings on the project, it was believed that by around 1990, some parallel computing technology from the DARPA architectures portfolio would be advanced enough to transition the expert systems to run on those machines. 

This bit of planning is quite interesting. Those planning SC’s applications very clearly understood the pace at which computing hardware and architecture was advancing in these years and incorporated a level of anticipation of that progress into their planning. For context, in this period the BBN Butterfly computer — which was a coarse-grained parallel processor and not so powerful yet — was one of the few operational parallel architectures. Massively parallel architectures were still in their very early stages at the time SC was getting underway. By 1993, those at the helm of SC felt that small computers would be able to run the types of systems required to operate the PA in real-time and then some. 

It was the job of the other PMs in the SC portfolio to seed investments and help orchestrate progress via the performer community to make this projected computing progress a reality. I will briefly give the reader an idea of what some of this work looked like. In the period between the mid-1970s and mid-1980s, technological development was occurring elsewhere in the hardware section of the SC portfolio and in industry that would end up becoming far more famous than the PA project itself — but that would, of course, enable many projects like the PA to become a reality in the coming decades. Some of this progress was funded by industry on its own, some by government and DARPA dollars, and often both. While NASA and the Air Force were buying chips from the young Fairchild Semiconductors, DARPA money had been finding its way to Fairchild through projects like the ILLIAC IV (covered in the FreakTakes Thinking Machines piece). Texas Instruments was hard at work on many of its own R&D projects, but many were also pursued in concert with actors like DARPA. For example, in 1983 DARPA approved $6 million to TI to develop compact LISP machine hardware — implementing a LISP processor on a single chip — as long as TI would fund $6 million in simultaneous software research for the device. Projects like the PA were at the top of mind when funding this TI project. Additionally, DARPA had funded much of the early Warp computer research at CMU which the university, later in the 1980s, worked with Intel to help miniaturize in developing the iWarp — whose first prototype was operational around 1990. Since this area of the history of computing progress is extensive and, to some extent, understood by many reading this piece, I will say no more on the progress in computing power in this period. Suffice it to say that the actual progress of the field did turn out to be in line with the lofty expectations of those planning SC as they outlined projects like PA in the early 1980s.

As this computing work was progressing — which would help the PA project team overcome its first major technical risk — SC’s portfolio of work on the second major technical risk, speech recognition systems, was also moving along quite well. While many consider speech recognition systems a relatively modern phenomenon, research in these systems was progressing quite steadily for several different performers in the SC ecosystem in the 1980s. By 1987, CMU had developed a speech system that could recognize 1,000 words with an accuracy of over 90% at real-time speed. BBN had also developed a similar system, and IRUS, which possessed a vocabulary of approximately 4,500 words in 1986 and was generally considered ready for testing in applications like the Navy-relevant Battle Management (the substantially larger vocabulary is because Battle Management’s speech recognition system was meant to be used in less noisy environments). Unsurprisingly, much of the progress occurring in the area was aided by improvements happening in computer hardware — in this period largely due to successes in VLSI development (covered more in the FreakTakes MOSIS piece). 

The progress in component technology was happening on the side of the work being done by the teams working directly on the PA project. Extremely pleased with the 1988 demo which showcased the general feasibility and usefulness of the system in practical situations, and with much of the work on relevant component technology in the SC portfolio going according to plan,  the Air Force elected to have the Lockheed/Air Force development team begin adjusting their development efforts. They hoped to have a system that could be put into aircraft to help them fly missions within three years. After one more successful November 1989 demo which showcased some additional functionality — such as reasoning using more imperfect sensor data, re-implementing programs in faster programming languages, and updating the machines used to run certain expert systems to faster versions —  the Air Force shifted the project into its further development phases. The coming stages were meant to get the machine ready to perform in more battle-like timelines and uncertainty conditions.

Unfortunately, the project would not ever make it across the finish line. The project’s fate seems to have become clear around 1991. Much of the component technology was strong, and some of these component improvements and learnings would find their way into similar future systems. For example, the Lockheed team produced extremely useful new insights when it came to the systems engineering of multiple coordinated expert systems combined in a single application. When the expert systems were combined, there were issues with each expert system following its own objective function and sometimes giving the pilot conflicting advice. The Lockheed team came up with a concept called a plan and goal graph to help the system better infer the pilot’s true wishes and deliver advice that was more coherent and in line with the pilot’s wishes. However, certain vital pieces of functionality were just not coming together the way the team had hoped as they attempted to make the system work in real-world situations. Certain pieces of the project that appeared conceptually feasible in the early stages were proving to be hairier than had been anticipated. The technology base was simply not capable of doing certain key tasks like recognizing certain maneuvers and accurately predicting the geometry that would result. But, in many ways, the program had served its intended purpose in DARPA’s portfolio.

Wright’s chief engineer on the project, Captain Sheila Banks, and its program manager, Major Carl Lizza, seemed to clearly understand that while the PA might not make it into a standard Air Force cockpit, it should not be considered a failure from the point of view of Robert Cooper’s DARPA. The two concluded a 1991 paper they had written on the state of the PA program by writing the following:

In accordance with the program goal to provide a pull on the technology base, the program often needed nonexistent technology to implement the design. Uncertainty-reasoning approaches, pilot modeling techniques, and development tools are areas that lacked sufficient research to implement the Pilot’s Associate vision. Verification and validation of knowledge-based systems constitute another area that is lacking techniques and is quickly becoming a program issue. The actual implementation of a system of this complexity uncovers many gaps in technology still to be addressed by the research community.

The project helped facilitate a level of practical learning in the SC portfolio and provided a level of clarity to many basic researchers trying to understand what contexts, roughly, they were attempting to produce technology for. But the project did not result in any immediate revolution in how Air Force pilots operated in their cockpits.

Battle Management Operations

I will give a slightly more high level accounting of some of Battle Management’s operational details. In many ways, the operational structure of the project was similar to PA’s, but the project resulted in more of a clear-cut win in the short run. Where certain project details are either instructive or interesting, I will be more thorough. But I would rather leave more time for ALV as its messy history and complex legacy are extremely instructive. (For more operational details relating to specific hardware, expert systems, and contractors see this document from Battle Management PM John Flynn.)

The Battle Management application project was launched with the goal of deploying the power of expert systems to help Naval decision-makers make better inferences about enemy forces, keep better tabs on their own forces, generate strike options, come up with operations plans, as well as a variety of other tasks — which any modern reader surely knows computers could assist Naval leadership in carrying out. The vision of the system was quite similar to the PA, but in the context of the Navy rather than the Air Force. As was the case with the PA system, the Battle Management system’s wish list of capabilities would likely require improvements in overall computing power from the rest of the technology base. Initial estimates for the computing power required for the system were around 10 billion operations per second — approximately a one or two order of magnitude increase in peak performance in comparison to what was available at the start of the program. 

While Battle Management was initially conceptualized as one program in SC’s founding document, it quickly became three separate programs: Fleet Command Center Battle Management (FCCBM), Carrier Battle Group Battle Management (CBGBM), and AirLand Battle Management. In this section, I’ll focus on the two projects that focused on the Navy as a customer —  FCCBM and CBGBM — for brevity’s sake. Both systems, similar to PA, were meant to use a GUI, natural language system, and expert systems to help assist Naval commanders in coming up with and executing operational strategies on the ship level (CBGBM) as well as on the theater level (FCCBM). 

CBGBM — the system that would be installed aboard a ship — was considered a continuation of a program by Allen Newell and others at CMU that had been ongoing since the 1970s. With the announcement of SC, CMU had proposed continuing the work area under the well-funded SC umbrella. DARPA believed this was a good idea and helped coordinate Naval buy-in to the project. The Naval Ocean Systems Center in San Diego officially bought-in to the CBGBM project early on. Similarly, for FCCBM — the fleet level battle manager — DARPA was able to secure buy-in from the HQ of the Pacific Fleet at Pearl Harbor. As the Air Force did with PA, the Navy agreed to provide a level of financial support to these projects at this stage. With that support DARPA had early buy-in from its customer into the project. DARPA’s working relationship with the Navy on the project would prove to be an exceptionally functional one — enabling a far more effective prototyping approach than was seen in the PA project.

The FCCBM’s (the system meant to help make plans on the fleet level) rapid prototyping strategy, implemented under PM Al Brandenstein, was executed in coordination with the team at Pearl Harbor. Roland and Shiman describe the project’s prototyping strategy as follows:

A test bed was established at CINCPACFLT [Pearl Harbor] in a room near the operational headquarters. The same data that came into headquarters were fed into the test bed, which was carefully isolated so that it could not accidentally affect real-world activities. Experienced Navy officers and technicians tested it under operational conditions and provided feedback to the developers. The resulting upgrades would improve the knowledge base and interface of the system.

This extremely tight customer relationship in the development process ensured several things in the early stages of the project: faster feedback cycles, increased buy-in from the customer, and extreme clarity on the requirements of the system. 

As the implementation stages of the projects got underway, the FCCBM team installed its computer hardware at Pearl Harbor in 1985. By March 1986, the test bed environment had been completed. With that infrastructure and scaffolding in place, the expert systems contractors could ensure their systems were tested on data with the highest level of fidelity to real-world data possible — because it was real world data. In June 1986, the Force Requirements Expert System — the first of the five expert systems for the project — which had been in development since 1984, was installed and began testing in the test bed. By the end of August the system was successfully demoed to DARPA and Naval leadership. This demo, as well as much of the work on the project, was heavily based on the development of BBN and the Naval Ocean Systems Center’s IRUS natural language generator — a project from DARPA’s technology base. By this time in 1986, IRUS had already achieved a vocabulary of almost 5,000 words and — being an experimental system developed with the knowledge that the Navy would be its initial user — even had a vocabulary and language generation style that could comprehend and respond to queries in language familiar to Naval operations staff.

By 1987, the FCCBM had already proved its worth to Navy staff. The system was capable of performing tasks that used to take fifteen hours in as little as 90 minutes. Research teams frequently seem to generate results like this on paper that either don’t turn out to be true in practice, or, alternatively, are never installed due to fear from customers that the paper results won’t hold up in deployment. FCCBM was able to avoid both of these issues. Since the system was already installed at Pearl Harbor and working on its actual data streams, the results were far from mere paper results. Additionally, the pain and annoyance of initial installation were much less of a factor in the decision than they might usually be because the system was already installed at Pearl Harbor. The first expert system quickly went into operation helping monitor ships in the Pacific while some of the other expert systems — which were further behind in readiness level — were tested until they were deployment ready.

Following a similar approach, CBGBM (the Battle Management system designed to be put on individual ships) was installed for testing on a ship in early 1986. Within a few months, the use of the machine in fleet exercises convinced the Navy that a modified version of the system would be as useful as they’d hoped. Improvements were made to the system before going out on a six-month tour later that year. The technology, at that point, was very much still in the prototyping stage because it could not operate in real time. But as a development project, CBGBM was proving its utility to the Navy. 

While these two projects generally progressed quite smoothly, there was one notable friction on the operations side of the project: classified information. Some individuals who worked with the contractors did not have security clearances because they couldn’t get them. Others just didn’t want them. The university research contractors, in particular, often didn’t want clearances. CMU, as an example, was one of the systems integrators for the CBGBM project and had a very good working relationship with the Navy, but largely did not have security clearances. So, this meant that one of the main systems integrators could not — for the most part — actually look at any of the golden, real-world data in the database at Pearl Harbor. Instead, their workflow had to look something like the following: discuss with the Navy what sort of information would be reasonable to use, do their R&D using fabricated data that they hoped would look like the real thing, and twice a year hand over their updated system to the Navy team and wait to see how it worked. While that was a hassle, it seemed to be a small price to pay for having the best minds on the project — the CMU team's work from campus seemed to port over to the real systems on the ships well enough.  So, the parties made it work. The Navy would even send staffers to CMU for long stays to ensure that as much knowledge exchange as possible was done. 

The work on the Battle Management applications, similar to PA, was enabled by the development work going on in the greater SC portfolio. This is particularly true of the following areas of the portfolio: computer hardware, natural language understanding, and speech recognition. Beyond the improvements in computing hardware going on in the broader SC portfolio — which was explored in the PA section and had similar implications to the Battle Management projects — much of the speech recognition and natural language understanding (NLU) research in the SC portfolio was done explicitly with Battle Management in mind. 

I will provide some clarity as to what some of this work looked like — which I will also cover more in a coming piece on DARPA workhorse computing contractor BBN — since several researchers and ex-DARPA PMs have personally expressed surprise that DARPA’s speech recognition and NLU portfolio has operational successes going back four decades. To many, this area of engineering feels quite young since many of the best methods in use today have been enabled by quite recent technology. However, as far back as the late 1970s, contractors in the speech recognition and NLU areas of the DARPA portfolio were building quite impressive applications with what — by today’s standards — would be considered somewhat quaint computing power and natural language methods.

While DARPA had largely reduced the contracts it let in speech recognition in the mid-1970s under Director Heilmeier, some speech recognition work continued in this era that would prove key to programs like BBN’s IRUS system that was deployed in the Battle Management system in the late 1980s. Several key pieces of work going on in the interim were: Baum et al.’s continued work developing hidden Markov models (HMM) at the Institute for Defense Analysis, Jim Baker’s HMM-enabled Dragon speech recognition system built at CMU, and BBN’s applied work on various speech recognition and compression projects for paying customers. On the natural language understanding front, BBNers and researchers related to BBN — such as the Harvard and MIT grad students of professors who worked part-time at BBN — made new discoveries and first-of-their-kind implementations in augmented transition networks and structured inherited networks in this period. Both of these technologies — which seem to have been replaced by more advanced methods by the late 1990s — proved extremely useful in BBN’s IRUS system which proved the feasibility of certain natural language tasks in the context of applications like Battle Management.

Even while DARPA was not giving contracts for speech recognition, believers in the technology at frequent DARPA contractors like MIT, CMU, and BBN that believed in the technology continued to push the field forward in the meantime. By 1986, the IRUS system already had a speech recognition vocabulary of almost 5,000 words and an accuracy rate of over 90% which was enabled by some of BBN’s work on alternative models. The relatively massive size of IRUS’s vocabulary — in comparison to the vocabulary of under 100 words in the PA’s speech recognition system — was largely due to the noisy conditions of Air Force cockpits. This constraint limited the scope of the PA’s vocabulary to simple vocabs and something like fifty commands along the lines of “Fire!.” In contrast, DARPA knew from the beginning that Battle Management would be a more conducive environment for speech recognition systems. So, from the beginning, SC management had ambitions of building a system capable of utilizing a vocabulary in the 1,000 to 10,000-word range in the next five years or so. 

As the battle management application projects were ongoing, there were eight different contractors across nine projects in the broader DARPA ecosystem pursuing various research projects in the area of speech recognition. To help wrangle all of these findings into technology that could be folded into future generations of battle management systems, CMU was given a contract as an “integrator.”

In the SC portfolio, integrators were sometimes given separate contracts and tasked with the primary responsibility of finding ways to integrate the cutting-edge findings from the performers’ work in the broader technology base — who were supposed to be keeping applications in mind anyway — into the final applications. An integrator contract was given in the PA project as well. In that project, five groups of performers who were working on eight different DARPA programs fed relevant results to TI, who was tasked with integrating the new knowledge into subsequent generations of the PA. In theory, with prime contractors like Lockheed and university researchers doing their jobs, this role should not be necessary. But in practice, the integrators could often make a big difference in the level of ambition of the integration work.  

In the end, the speech recognition work went so well that SC management never needed to change the initial, lofty goal of achieving a system with a vocabulary of roughly 10,000 words. And that's saying something; oftentimes, lofty goals are set at the beginning of projects like this with the understanding that, after a period, the PM, relevant performers, and DARPA management will touch base and agree on more reasonable goals. However, revising the goals was not necessary in this case.

The other area of battle management’s language work — NLU — also had integrators that it leaned on. The broader NLU applications work was made up of two integrators and five other teams of performers. The integrators were both teams. The first team, in charge of integrating text understanding, was made up of NYU researchers and the Systems Development Corporation — a frequent DARPA contractor since its systems engineering work on the SAGE air defense system. The second team, in charge of integrating broader work in natural language query processing, was made up of BBN and the Information Sciences Institute (who is the star of the FreakTakes piece on MOSIS). This natural language understanding work also met the initial, non-quantitative goals that SC’s planners set out: the final users were able to provide the system new information and get understandable feedback in return explaining the system's “thought process.”

The Naval Battle Management projects clearly accomplished their short term goals. The primary goal of the program, on paper, had been to help the Navy with supporting and planning in areas where time constraints are extremely critical. And the systems did just that. For some important tasks done at locations like Pearl Harbor, they reduced the time it took to do certain planning tasks from 15 hours to 90 minutes — along with the increase in computational reliability that comes with using a computer. Additionally, these Battle Management systems were in place and working within five years. This not only meant that the projects were a success as quickly as possible, but also that the projects helped DARPA Directors in doing things like justifying budgets and convincing increasingly austere Washington decision-makers in the late 1980s that their work was saving money for the taxpayers even if it often looked somewhat speculative. In many ways, Battle Management was the PR success that the ALV was hoped to be.

Autonomous Land Vehicle Operations

The third application placed at the heart of SC’s founding document was the Autonomous Land Vehicle (ALV). As the SC program wore on, ALV would also come to be one of SC’s biggest disappointments — at least in the short term. The ALV was envisioned as an autonomous recon vehicle that could navigate battlefields — obstacles and all — to conduct reconnaissance missions and report its findings back to soldiers who did not have to leave safer positions to obtain the information. 

From a mechanical perspective, DARPA hoped for a machine that could travel over thirty miles in a single trip with top speeds of almost forty miles per hour. Of course, machinery existed that could mechanically hit these benchmarks. However, DARPA wanted a machine that could do all of this while being autonomously operated. To do this, many computational abilities would need to be developed for the system. The computational tasks required for an ALV to successfully work in battle included: route planning, navigating via visual analysis of physical landmarks, rerouting and avoiding surprising obstacles that were not in the route plan, ingesting useful information on the target, and transmitting that information to a military team. At the time, it was believed that performing these tasks would require an expert system that could run as many as three times more rules per second than current expert systems. Additionally, DARPA believed that the computer required to run the machine would not just need to be small — something like 4 ft x 4ft x 4ft — but would also need to be powerful enough to perform ten to one hundred billion von Neumann instructions per second. This was a tall task, as contemporary von Neumann machines were generally limited to around thirty or forty million instructions per second. This need for a three-order-of-magnitude improvement in computing meant that the ALV project achieving its goals relied even more strongly on improvements in computing power than the PA and Battle Management. 

The project sought to tackle problems in computer vision that modern autonomous vehicle research groups are still working through today. However, Clinton Kelly was confident that enough leg work had been done on the research front for DARPA to begin creating a program as a technology pull to bring DARPA’s vision work to practicality. This confidence derived in no small part from DARPA’s Image Understanding program which had kicked off in the mid-1970s. The Image Understanding (IU) program — sponsored by IPTO and then-Director J.C.R. Licklider — was formally spun up as a cohesive program on the heels of progress in DARPA-funded projects like DENDRAL in the 1960s.

DENDRAL was not computer vision per se — in that it could not actually ingest a photo — but the system could utilize a text representation of some molecular structures and used expert systems to perform analyses on them. Projects like this demonstrated that, once information taken in visually can be coded symbolically, systems were in progress that could make intelligent use of the information. Somewhat naturally, “signal-to-symbol mapping” became a key point of emphasis in the IU program — which would allow visual information from cameras or sensors to be encoded in such a way that systems could act intelligently on the information. Somewhat surprisingly, there were even some practical successes from the early-IU years. One example was DARPA and the Defense Mapping Agency’s (DMA) joint funding of a system — housed at the Stanford Research Institute — to automate certain cartographic feature analysis and assist the DMA in certain mapping and charting functions. However, most of the work done by the performers in IU in this period was quite far from applications largely because the field was young and there were not many clear-cut applications for the work just yet. However, it should be noted that as IU was getting underway in 1975, Licklider and others hoped for the work to progress in such a way that applications were within reach by 1980. 

Even with the ALV project kicking off in 1985, ten years after the start of IU, many veteran vision researchers felt this push for an ALV-type application might be a bit premature. Kelly and Ronald Ohlander — the DARPA PM who oversaw vision-related projects in this era — polled some veteran IU researchers to ask what they felt would be reasonable benchmarks and timelines for the project. Kelly told Roland and Shiman in an interview that he felt the answers were far too conservative and that he “took those numbers…and sort of added twenty percent to them.” When the researchers pushed back, they settled on a  compromise. Ohlander noted to Roland and Shiman that he thought the program to be “a real stretch” — far more than the other applications.

This general sentiment could also be seen with ALV’s lack of early buy-in from its intended customer: the Army. The Army was the intended customer for the ALV, but Army leadership saw no acute need for a robotic vehicle at the time and, thus, did not commit to helping fund its early development. The project, however, was seen by Kelly — and likely Cooper — as a fantastic pull mechanism for DARPA’s image understanding work. DARPA — as it did with other projects in this period such as the ILLIAC IV and Connection Machine — was hoping that providing funding and a contract for a machine with clear goals that required improvements to its component technology could help bring a young area of research to a new level. As Richard Van Atta wrote in one of his technical histories of DARPA:

Before the ALV project, the vision program had centered on defining the terms and building a lexicon to discuss the mechanics of vision and image understanding. The ALV was one of the first real tests of IU techniques in practice.

Even if the technology proved to be out of reach of his generation of researchers, Ohlander acknowledged that the researchers’ work deploying their ideas on a real-world test bed would expose the shortcomings of their approaches and help move the field along faster than it would otherwise.

With the kick-off of SC, funding for all vision-related work in the SC portfolio expanded rapidly. IU’s budget abruptly increased from its usual $2-$3 million a year to $8-$9 million a year. The ALV application program itself was, to some extent, co-managed by Clinton Kelly as well as its official PM William Isler.  They planned for one prime contractor to be in charge of engineering the vehicle and gradually incorporating the new component technology from the technology base into the machine on an ongoing basis. The program managers felt that one of the major aerospace companies from the regular defense contracting pool would be a good fit for the project. While these groups did not have experience in computer vision — almost nobody did — they did have experience overseeing projects on this scale of complexity and had all proven quite adept at large-scale systems integration. Some of these companies, at least, had experience with machines like remotely operated vehicles and basic forms of AI systems. As they were submitting proposals to DARPA’s January 1984 qualified sources sought, these aerospace companies understood that the project would be carried forward with whichever one of them won in partnership with at least one university department. The partnership was intended to help promote the integration of the expanding capabilities of the DARPA technology base into prototypes of the ALV as quickly as possible. 

The ultimate winner was Martin Marietta — which would later become the “Martin” in Lockheed Martin.1  The contract was to last 42 months and was for $10.6 million — with an optional extension of $6 million for an additional 24 months. DARPA chose the University of Maryland (UMD) as the university research partner to assist the company in folding technology improvements from the technology base into the machine in the early stages of the project. UMD was a well-known entity to the AI portion of DARPA and its PMs as it was one of the primary university research centers contributing to IPTO’s Image Understanding work. So, the university’s primary task was to get the practical vision system that they had been working on folded into the machine so the Martin team could begin working towards the goals of its demos as early as possible. 

As implementation work got underway, the ALV quickly adopted an accelerated schedule. Towards the end of 1984, DARPA moved up the date of the machine’s first demonstration to May of 1985 — six months before the previous initial demonstration. The vehicle would have to identify and follow a basic road in a demo with no obstacles of note. As Martin Marietta rushed to quickly assemble its first vehicle, it largely relied on off-the-shelf components that were tried and true. The mechanical parts such as the chassis and engine were standard, commercially available parts. The software and hardware for the machine’s computing were not exactly cutting edge. One of the two types of computers chosen — a VICOM image processor — may have been chosen because one of the University of Maryland Researchers had extensive experience working with the machine and stood the best chance of quickly getting the vision program up and running with familiar equipment. The second type of computer — Intel’s single-board computers, which were more or less personal computers — was used to control the vehicle itself. The camera that fed into the vision system was a color video camera mounted at the top of the vehicle and paired with several charge-coupled devices to transmit the visual information to the computer in a usable format.

The first demo was quite basic. The goal was for the machine to follow the center line of a road at Martin Marietta’s testing facility for about 1,000 meters. To give the reader an idea of how the vision system encoded real-world image data in a usable way, please see the following figure provided in a 1985 Martin Marietta report explaining what a digital scene model would look like for a given road image.

With basic image processing, some engineering cleverness, and standard geometry for calculating things like distance traveled and curve angles for steering, the machine seemed like it might be able to navigate a basic course quite well. The machine was powered by two major categories of visual decision-making software. The first category was the software dedicated to understanding exactly what the ALV’s camera was seeing. This software processed and enhanced the images so it could then use some of UMD’s edge-tracking algorithms to find the various boundaries in the image. Using these boundaries, the software calculated where the center line seemed to be. Another special algorithm would attempt to plot a 3D grid.

This work was passed to the second software system which was meant to perform some basic reasoning tasks using the current 3D grid model as well as the 3D grids from the timestamps immediately preceding the current one. At this stage of the process, the reasoning system largely attempted to assess the reliability of the current image, given what it had seen in the previous images. If the image seemed unreliable, the system attempted to make up a more believable path that fit with the images seen previously. If this switch happened, in that moment the reasoning system was seen as driving the vehicle rather than the vision system — since the vehicle was operating off of a pre-stored model rather than the current, low-confidence model built by the vision system. This backup plan was necessary for achieving the goals set by the demo because, given the slow processing speeds, the vehicle’s speed would often outrun its visual processing. So, this was necessary to keep the vehicle moving without stopping too frequently or going too slowly. 

The first demo — a standard road-following task — went off rather successfully. The vehicle traveled the approximately 1,000 meter course in about 1,000 seconds and did so traveling about 85% of the way under vision control rather than reasoning control. The machine did this while generating over 200 scene models and trajectories to help inform the vehicle along the way. On the heels of this success, the Martin and UMD team were right back to work because the next demonstration was in six months. And this demonstration was a test that was considered less of a layup than the first one had been. The course for the demonstration — which the ALV had to navigate at 10 km/h — required that the ALV navigate a sharp curve at least 3 km/h, progress along several straight stretches of road, and, lastly, stop at a T intersection to turn around and repeat the course. It had to do all of this without the reasoning system overriding the vision system.

To do this, the team would have to add more sensor information than just the single video camera to take in all the required information. Additionally, they had to re-work the vision software to somehow make use of all of this new information. In attempting to navigate this much harder problem, Martin relied on a DARPA-funded laser range finder that had been developed by the Environmental Research Institute of Michigan. The device could be attached to the front of the vehicle and was used to scan — up and down as well as side to side — the terrain in front of the vehicle to identify obstacles and other important features. Those working on the software attempted to find a way to combine this knowledge into a single 3D model of the scene. This knowledge was combined with the reasoning system which now contained a digital map of the track itself which the machine could use as a reference — the Engineering Topographical Laboratory had developed the map. The vehicle, impressively, passed this second test as well in November 1985. At this point, the ALV application seemed to be making extremely satisfactory progress. The technology the ALV used in these early demos — including the laser range finders — was not exactly cutting-edge, but the machine was hitting its practical benchmarks and hopefully could begin subbing in more cutting-edge component technology as the demonstration benchmarks continually raised the bar for the ALV. 

While the work on these early demos was happening at Martin Marietta, Martin and other ALV-related component contractors were also holding meetings at regular intervals — often quarterly or as frequently as monthly — discussing project needs, developments in the technology base, and how to delegate tasks. These conversations with research teams working on broader vision work were particularly important for work on the demos scheduled to happen in later-1986 and beyond — when it seemed like Martin would begin having acute needs to integrate more work from the technology base into the ALV to meet the goals of the demos. Much of what went on at these meetings was not so different from what happens at many DARPA PI meetings — with the standard presentations of progress and teams getting on the same page.

One logistical point of particular note was that, as some of the higher level planning meetings played out, the project structure was almost very heavily shaped based on the arbitrary preferences of the component contractors.  The contractors who found themselves working on the reasoning system vs. the vision system had very different interests and incentives. Most of the reasoning contractors were commercial firms and had the incentive to fight for a broader scope of work and more tasks getting assigned to the reasoning system because that meant better financial returns to the company. Most of the vision contractors, such as the University of Maryland, were academics and largely content to let that happen — or, rather, not as committed to fighting for a broader scope of work as the companies — if it meant they got to focus primarily on specific vision sub-tasks that were more in line with the size and scope of projects academics often undertook. For example, the private companies felt that building up much of the systems mapping capabilities, knowledge base, and systems-level work should fall to them.

CMU and one of its project leaders — Chuck Thorpe — seemed to be the strong voice of dissent on many of these matters from the academic community. Ever since CMU had become a major contractor in DARPA’s Image Understanding work, it had taken a rather systems-level approach to attacking vision problems — working with civil and mechanical engineering professors like Red Whittaker to implement their vision work in robotic vehicles — rather than primarily focusing on the component level. The CMU team far preferred this approach to things like paper studies — in spite of the increased difficulty. So, as the project progressed and these conversations were ongoing, both sides had at least one party fighting to work on component work as well as some amount of integration work prior to Martin’s work integrating components in the last mile — often right before a demo.

While that future planning was taking place in the background, it was becoming clear that Martin Marietta’s pace of actual technical progress was not as positive as the demos had been. Their work implementing basic vision algorithms proved finicky as small changes like the lighting, road conditions, and shadows caused the vehicle to behave very differently from one day to the next. Each new software program or change seemed to cause a new problem which took time to debug. It was messy work. Additionally, to the dismay of Martin with its strict demo timelines, it began to seem like a large number of specific vision algorithms might be required to handle different, niche situations such as even conditions, uneven conditions, etc. This would take significantly more work than if one unified vision approach could cover most situations — as the team had initially hoped. 

Meanwhile, demo timelines began to get tighter and tighter. More demo-like activities kept being added as more PR was brought to the program by Clinton Kelly — news releases were issued, the press was invited to watch some of the vehicle's runs, etc. In the midst of all this demo pressure, the ALV project began to embody more of a negative “demo or die” principle than embodying a pull mechanism to effectively move the technology base forward. As Martin’s actual progress slowed and the demo schedule tightened, the company began to focus only on achieving the benchmarks of the next demonstration at the expense of developing truly useful, new technology. For example, for one of the demonstrations, the obstacle avoidance algorithm — put together by one of the component technology contractor companies — worked in such a narrow set of circumstances that it could not even be used successfully in a parking lot, but it could work on the roads the demonstration took place on. 

As Martin was rushed in these middle years of the program, the ALV was clearly not acting as the test bed for the SCVision projects — the name encompassing much of the vision work in PM Ronald Ohlander’s AI portfolio — that it was meant to be. Much of Martin’s staff time and equipment was dedicated to hitting the next demo and not finding adequate time to serve this more future-looking function. It does seem that the team was truly busy and not just blowing the researchers off. By April of 1986, the 1.5-year-old ALV had logged 100,000 miles and needed to have its engine replaced since it had worn out. In one two-month period of testing prior to a demo, 400 separate tests were conducted using the machine. Preparations for a subsequent demo might begin as much as six months before — often not long after the last demonstration had just finished. In essence, things were becoming very crunched at Martin.

In 1985, in the period between the first and second ALV demonstrations, Ronald Ohlander — a former student of Raj Reddy at CMU — decided it might be necessary to rely on a contractor separate from Martin Marietta to integrate the component technology improvements from the technology base more ambitiously. Ohlander wanted to issue a contract for a scaled-down version of the ALV operating from within the SCVision program. The integration contractor in charge of the machine would be in charge of testing the computer vision algorithms of other research teams that fell under the SCVision umbrella.

Ohlander, in doing this, adeptly observed that the prime contractor on the ALV project might be under pressure to meet its contract milestones and not be sufficiently motivated to fold the most cutting-edge component technology coming out of his SCVision program into the ALV. In addition, he seems to have felt a level of unease relying on a machine, contractor, and program management from a different DARPA office — Isler and Kelly largely operated out of the Engineering Applications Office —  to demonstrate the usefulness of the investments coming out of his own office — IPTO. Ohlander explained the decision to Roland and Shiman in an interview, stating:

It wasn’t to outshine them or anything like that…but we were…used to doing things in universities with very little money and getting sometimes some startling results. So I…had a hankering to just see what a university could do with a lot less money. 

In the end, the contract would go to Carnegie Mellon. CMU had already, unlike most of the component contractors in the SCVision portfolio, been testing its component technology on its own autonomous vehicle: the Terregator. CMU’s used the Terregator — for often slow and halting runs — to test the algorithms developed for its road-following work. The Terregator proved extremely useful to this research, but its use had also helped the CMU team learn enough to outgrow the machine.

In CMU’s ongoing efforts to implement sensor information from a video camera, a laser scanner, and sonar into one workable vehicle, the CMU researchers had come to realize that they needed funding for a new vehicle. CMU needed a vehicle that could not just accommodate the growing amount of sensor equipment, but also one that could carry out its tests with computers and grad students either on board or able to travel alongside the vehicle. This would allow bugs to be fixed and iterations between ideas to happen much faster. The vehicle that the CMU computer vision and robotics community was casting about trying to get funding for would come to be known as the NavLab. Fortunately, what they were seeking money for seemed to be exactly what Ohlander felt the SCVision portfolio needed. The funding for the NavLab began around early 1986. The funding set aside for the first two vehicles was $1.2 million, and it was estimated that any additional NavLab vehicles would cost around $265,000.

As Martin was somewhat frantically taken up with its demo schedule, CMU, with its longer time windows, was  free to focus on untested technology as well as technology that ran quite slowly but seemed promising in terms of building more accurate models of its environment. As the CMU people saw it, if anything proved promising they could upgrade the computing machine used to operate that piece of the system later on. The NavLab team was working closely with CMU’s architecture team that was developing Warp systolic array machines from within Squires’ computer architectures portfolio. The CMU architecture team was developing its equipment with vision applications in mind as an early use case. As a result, they were happy to change the order of functionality they implemented in order to account for the acute needs of the SCVision team. Additionally, if a piece of vision technology proved promising to the NavLab that could not be sufficiently improved with the immediate help of a Warp or some other new-age computing machine, the vision researchers likely figured that a piece of hardware powerful enough would likely exist soon to help push the idea along — even if not in the next year. Lacking the hard metrics of yearly demos with speed requirements loosely matching how fast the vehicle would need to go in the field, the CMU team could afford to think more along the lines of  “how do we build a machine that is as accurate as possible, even if it moves really slowly today?”

As CMU took up the mantle of building SCVision’s true test bed vehicles, it also took over the role of SCVision portfolio’s true “integrator” for Martin’s ALV. Carnegie submitted a proposal to carry out this integration work along with building the NavLab. From a high level — in spite of neither the ALV nor SCVision projects having set aside money for an integration contractor — this made sense. Martin was meant to integrate work from the technology base on its own, but around mid-1985, it was becoming clear that they were not doing an effective job on their own. Academic departments were often close to the evolving frontier of their own field, but often did not have the project management skills and scaffolding to do great integration work. CMU was somewhat exceptional in that it did this sort of thing quite frequently. Clinton Kelly, while apparently not thrilled about the idea since he was a massive supporter of the ALV project, agreed to the arrangement. 

As 1986 went on, Martin Marietta was able to overcome issues with its vision system well enough to continue passing its demos which required the machine to go on larger tracks at faster speeds — such as 20 km/h on a 10km track with intersections and some obstacles. As 1987 approached, plans were being made to upgrade the computing equipment in the machine — some of the new computers were developed in Squires’ architectures portfolio. Additionally, as the demos in the coming years were slated to take a marked step up in terms of the complexity of their missions and terrains, the Martin team’s relations with CMU as the new integration contractor — a bit cold at first — warmed considerably.

Personnel from both orgs kept in regular touch and visited each other’s sites. Martin even went as far as to enroll a member of its staff as a grad student under Chuck Thorpe to serve as a liaison between the two. In mid-1986, CMU began helping Martin install software for map planning, obstacle detection, sensor calibration, and new image recognition functions that it had developed and integrated on its own machines on campus.  This period also saw specialized tools developed so CMU could more easily run its code without alterations on Martin’s machines — which ideally would have been done earlier in the project, but better late than never. The machine was steadily improving. In the summer of 1987, when one of the private sector contractors — Hughes — was at Martin’s Denver testing facility to test its planning and obstacle avoidance software, the ALV made its first extended drive in country conditions on a route it had selected for itself using only map data and sensor inputs. As Roland and Shiman wrote of the rather triumphant run:

The vehicle successfully drove around gullies, bushes, rock outcrops, and steep slopes.

However, that mid-1987 period would prove to be the high water mark of the ALV application program. After the ALV’s November 1987 demo, a panel of DARPA officials and technology base researchers met to discuss phase two of the program. Roland and Shiman described the titanic shift in how the ALV project and the SCVision portfolio operated in the wake of that meeting as follows:

Takeo Kanade of CMU, while lauding Martin’s efforts, criticized the program as “too much demo-driven.” The demonstration requirements were independent of the actual state-of-the-art in the technology base, he argued. “Instead of integrating the technologies developed in the SC tech base, a large portion of Martin Marietta’s effort is spent ‘shopping’ for existing techniques which can be put together just for the sake of a demonstration.” Based on the recommendations of the panel, DARPA quietly abandoned the milestones and ended the ALV’s development program. For phase II, Martin was to maintain the vehicle as a “national test bed” for the vision community, a very expensive hand servant for the researchers.

For all practical purposes, therefore, the ambitious ALV program ended in the fall of 1987, only three years after it had begun. When phase II began the following March, Martin dutifully encouraged the researchers to make use of the test bed but attracted few takers. The researchers, many of whom were not particularly concerned about the “real-world” application of their technology, showed more interest in gathering images from ALV’s sensors that they could take back to their labs.

The cost of the test bed became very difficult to justify. Phase I of the program had cost over $13 million, not a large sum by defense procurement standards, perhaps, but large by the standards of computer research. Even with the reduced level of effort during phase II, the test bed alone would cost $3–4 million per year, while associated planning, vision, and sensor support projects added another $2 million. Perhaps most disappointingly for Kelly, the army showed little interest in the program, not yet having any requirement for robotic combat vehicles. One officer, who completely misunderstood the concept of the ALV program, complained that the vehicle was militarily useless: huge, slow, and painted white, it would be too easy a target on the battlefield. In April 1988, only days after Kelly left the agency, DARPA canceled the program. Martin stopped work that winter.

Autonomous Land Vehicle Results

The Autonomous Land Vehicle hit its agreed upon, quantitative benchmarks in its early demonstrations. It also incorporated many technologies from the SC technology base into successive iterations of the ALV such as BBN’s Butterfly processor, CMU’s Warp, and other vision technologies that CMU and others had helped develop. If the Army had been excited by the far-off sounding application and the ALV’s progress in the early demos, there would likely have been a much higher chance that the program continued on schedule. However, since the Army was not yet interested in autonomous recon vehicles, the ALV’s biggest service to DARPA and its portfolio was as a technology pull mechanism to help move its technology base forward in a coordinated fashion. And since Martin Marietta had proven rather lackluster in the integration department, DARPA’s decision in late 1987 to wind down the ALV project and, instead, put more emphasis on the NavLab makes sense.

Work in the spirit of the ALV largely lived on in CMU’s NavLab efforts. CMU’s somewhat unique setup — running an engineering research program equipped with de-facto program managers from within an academic department — proved extremely effective in carrying out this project. With the organization dedicated to attacking an extremely concrete goal like autonomous driving, the CMU team was able to keep a clarity of vision throughout the project — while maintaining an organized structure that was less strict than a company but more organized than a traditional academic department. CMU would go on to make very noteworthy contributions in the years since the ALV was wound down. In an IEEE oral history of CMU NavLab program manager Chuck Thorpe, Thorpe describes the steady practical progress of the NavLab team, comparing it to his newborn son, saying:

My son was born in ’86, and we had contests to see when NavLab 1 was moving at a crawling speed, my son was moving at a crawling speed. When NavLab 1 picked up speed, my son was walking and learning to run. NavLab 1 got going a little faster, my son got a tricycle. I thought it was going to be a 16-year contest to see who was going to drive the Pennsylvania Turnpike first, because they were each progressing at about the same speed.

Even without the consistent proof of progress demos such as those that drove the ALV project, steady progress was being made by the interdisciplinary, practically-minded, academic CMU team. But the team’s progress very much received one of those unexpected jolts of progress that groups of researchers diving down riskier rabbit holes with some regularity hope to experience. One of CMU’s grad students working on the project, Dean Pomerleau, published a 1988 paper on a system called ALVINN that would be a major step in showing us the future of the field. Thorpe finished the paragraph he started comparing the NavLab and its steady progress to his son, recounting:

...I thought it was going to be a 16-year contest to see who was going to drive the Pennsylvania Turnpike first, because they were each progressing at about the same speed. Unfortunately, this really smart graduate student named Dean Pomerleau came along and built a neural network technique, which managed to be the most efficient use of the computing researchers that we had, and the most innovative use of neural nets at the time, so in 1990, he was ready to go out and drive the Pennsylvania Turnpike.

Thorpe then recounted how, by 1995, a later version of the NavLab running on a newer vision system, RALPH, took things even further. RALPH’s approach — which Pomerleau also came up with —  combined pieces of ALVINN’s neural net approach with a new model-based approach which was similar in spirit to simpler approaches used in the ALV. Using RALPH’s vision system, the NavLab was taken by Pomerleau and another student from D.C. to San Diego, driving 98% of the way autonomously. Things like “new asphalt in Kansas, at night, with no stripes painted on it,” flummoxed the machine. But it was plain for all to see: the technology was taking massive steps.

Lessons Learned (and Caveats) From All Three Programs

The legacy of the ALV project is more complicated than that of PA and Battle Management, but it may have done more good in shaping the technology base than the other two projects combined. All three projects had their bright spots as well as their weaknesses.

In terms of customer relationships, PA and Battle Management had significantly better relationships with their armed services customers than the ALV had with the Army. Battle Management’s practical success in terms of positive impact on its customer, the Navy, seems to have outdone the practical success of the PA. But it is hard to make that out to be solely the fault of the PA or its management team. After all, in many situations in ARPA-like orgs, you simply get the customer you get.

That being said, Battle Management certainly provides a great exemplar of how to — with the right customer — integrate a test bed as close to the production process as possible to minimize the number of annoyances that can arise when translating technology built in lab environments to real-world environments. This strategy from the Battle Management team also helped mitigate the organizational/logistical hold-ups that often result when looking to implement technology from a remote test bed.

The PA, in spite of its lack of success in pushing a truly useful product out of the program, proved to be a somewhat successful pull mechanism for the speech recognition and natural language understanding technology base. Let’s remember, to many, it was not crystal clear that speech recognition and NLU research was ready for translation into products. The PA application and associated funding on related problems in the technology base helped harness the efforts of DARPA’s performers to work on common problems. This, in the end, helped push certain areas of research into practice. In other cases, the PA highlighted areas that the field needed to work on.

The ALV, in contrast to these two projects, never had the customer relationship of PA and Battle Management. Not even close, really. The Army did not want the ALV; the Air Force wanted a working PA and the Navy was extremely enthusiastic about Battle Management. However, those at DARPA understood that the Army was not excited about getting involved early on with this project. So to judge the program as a near-term failure as an application is somewhat fair, but also largely misses the point. The ALV was something different than the other two applications: an extreme pull on the DARPA technology base. And, in that, the program did a much better job. A large sum of money and energy may have been spent non-optimally pursuing the Martin Marietta work when it was the prime contractor. But DARPA was also relatively quick to redirect the portfolio after several years of watching its broad array of contractors work and assessing their approaches and their results. In the end, CMU proved to be better suited to help push the vision area of the portfolio forward than Martin and the ALV. Fortunately, DARPA was light on its feet in making that change in strategy happen.

It is somewhat hard to tell what the legacy of the DARPA’s vision work during SC would have been had it not been for the exceptional setup, management, and overall culture at CMU with its DARPA contracts. The CMU computer science and engineering teams involved with DARPA embodied some of the most positive aspects of academia and some of the most positive aspects of industry. I will explore the setup of the CMU department and how they fulfilled their DARPA contracts further in a coming piece. What matters, for the scope of this piece, is that the CMU department was exceptionally equipped and internally motivated to go after a problem like practical vision applications, and a large amount of DARPA money found them and enabled them to go on to do exceptional work.

Each of these three projects provides much to learn from for a PM orchestrating and balancing a portfolio of investments that is looking to show some level of practical success while also hoping to maintain a high level of ambition. Hopefully this piece proves of some use to them.

Thanks for reading. Subscribe and stay tuned for coming pieces.

Subscribe now

Share

PA Links:

• Pilot's Associate: a cooperative, knowledge-based system application

  • A report from the Air Force program managers for the PA detailing the performance of the PA’s subsystems in its demos. 

• DARPA’s John Retelle meeting, presentation, and Q&A on the PA to the Transportation Research Board

  • Contains a mention of being able to transition the program to parallel computing machines around 1990. 

• Rizza, the Wright PM on the PA project, speaking at the 1991 Space Operations, Applications, and Research meeting

  • In the document, Rizza explains how the systems and principles that helped make the PA something of a success could be applied to NASA’s work with managing the movement and maintenance of deployed satellites — which was very man-power intensive at the time

• Bachman, Gunning, and Burke’s Integrated Artificial Intelligence Systems

  • Interesting throughout, but its primary relevance in this section of the write-up is the authors’ clear description of why exactly the machine was not pursued further from a component technology capability standpoint.

Battle Management Links:

• 1986 document from John Flynn outlining some aspects of the early demos

  • DARPA Naval Battle Management PM and then long-time BBN employee goes into some depth on the specific hardware, expert systems, user experience, and contractors on the projects.

• Computer Effort Falling Behind article from Michael Schrage written for the Washington Post in 1984

  • Certain pieces of the article did not age particularly well when it comes to a program like Battle Management — a Pentagon insider even referred to these programs as “duds.” Within two years of the article being written, battle management was becoming a clear success. 

  • But other pieces of the article — like the tension between Kahn and Conway — were very fair and got at something that was a real complication for the program for years.

  • The article provides a clear view of how programs like these might be covered in the press if things do not hit the ground running. 

•  Chapters 14 and 15 of A Culture of Innovation — a book on BBN’s 

  • From a high level, the book dives into BBN’s culture, structure, and details on many of its most famous projects

  • Chapters 14 and 15 provide great detail on the specifics of the researchers, contracts, and work details surrounding BBN’s speech work in the early-1970s all the way through the early-1990s.

Autonomous Land Vehicle Links

• Chapter 14 of Van Atta’s DARPA Technical Accomplishments. Volume 2. An Historical Review of Selected DARPA Projects 

  • This chapter outlines much of the lead-up to this generation of DARPA’s vision work.  Particularly useful is its focus on the Image Understanding program which kicked off in the mid-1970s but had its roots in work from the prior decade. 

  • The work dives into the details of specific projects as well as what specific performers worked on what research areas in different eras. 

• CMU Strategic Computing Vision Project Report: 1984 to 1985  

  • This report outlines the first year of CMU’s work on the application and DARPA-relevant vision work as a whole.

  • Particularly goes into more depth on the technical details of various CMU vision research approaches in the early years of SCVision in using technologies like sonar, laser vision, stereo cameras, and video cameras.

• Martin Marietta road-following write-up: The Autonomous Land Vehicle (ALV) Preliminary Road-Following Demonstration 

  • Martin Marietta describes the finer technical details of the ALV during its first demo.

  • The write-up also dives into how Martin Marietta hoped the system would progress from demonstration to demonstration as well as their planning in this nascent stage.

• Dean Pomerleau’s 1988 paper, ALVINN: An Autonomous Land Vehicle in a Neural Network

  • The paper dives into Pomerleau’s initial implementation and results using a 3-layer backpropagation neural network trained on approximately 1200 simulated images of roads to train the model used to steer the car. At the point of the paper being written, the performance of this system seemed roughly equivalent to CMU’s more manual methods — often utilizing expert systems-style reasoning given the sensor data and clever edge-finding heuristics — of training models to steer the machine to that point.

• Chuck Thorpe IEEE oral history

  • Explores many of the finer details of how the projects came together and progressed at CMU. It is great fun to read for anybody interested. 

• Misc.

  • Developing Technologies for Army Autonomous Land Vehicles — 1985 document written by army engineers on the program area

  • Implementation and performance of a complex vision system on a systolic array machine

  • Norberg’s oral history of Alan Newell

General Links:

• Strategic Computing: How DARPA Built the Computer Age

• Strategic Computing’s founding document

Pattern Language Tags:

• Encouraging modularity and open interfaces

• Democratizing an area of research via cost reduction

• Promoting a coordination/service mechanism to reduce material costs and increase research feedback cycles

• Facilitating tool/hardware improvements in a key technology area far from its suspected theoretical frontier

Introduction

In the early 1980s, a large amount of money and energy from DARPA and the DoD was shifting towards computing infrastructure and applications. While many involved with those organizations already had one (hopeful) eye on long-term applications such as AI, it was clear that almost all aspects of DARPA’s AI plans, in some way, leaned on ease of access to computer chips for the research community.

So, as a suite of DARPA and DARPA-adjacent projects sprung up — all in the hopes of systematically building up the greater universe of chip capacity and computing applications — DARPA’s MOSIS project was established to ensure that researchers working on computing problems could have faster and cheaper access to computer chips. In this early era of computing research — when the technical capabilities of chips were orders of magnitude more limited than they are today — it was the norm for many researchers and engineers to design bespoke chips for the particular task at hand. MOSIS would provide the middleman service to all researchers who needed their designs fabricated into chips more cheaply, quickly, and discretely than would otherwise be available.

Context

In the late-1970s, printing chips was extremely expensive. For independent researchers paying a separate facility to print their design, the price of a single wafer could have a price tag in the tens of thousands. Since hardware was often specially developed for individual applications or experiments, single research projects often required printing individually designed chips. The cost was a big problem and the process itself was a major annoyance to researchers. Manufacturers who printed chips often had specialized syntax in which they encoded the digital designs into their machines. Going back and forth with a manufacturer was a major tax on researchers’ time and timelines. Also, researchers handing all of their designs to manufacturers was a point of concern. If their project was a success and they attempted to commercialize it, one of the researcher’s now-competitors would have their digital designs in hand. The system of fabricating chips, at the time, was just not set up for convenient access by the community of individual researchers. That is why MOSIS was created.

MOSIS could make things cheaper and easier for these researchers. While individual silicon wafers and runs with manufacturers were expensive, the average chip that needed to be fabricated for a standard research project was quite small — about 1/20th the size of a wafer of silicon. Since the field was new and there was not exactly a mass market of hobbyists who would pay for many chips to be run in parallel, a market/middleman service had not been made for this particular service. Without a solution, new research might be far more expensive, time-consuming, and sparse than it otherwise could be.

In the backdrop of all of this was the belief – encompassed by a 1976 RAND report by Carver Mead, Ivan Sutherland, and Thomas Everhart – that the integrated circuit revolution had only half run its course. Another four orders of magnitude improvement in chip manufacture seemed theoretically possible. Despite this potential, the authors believed that many industry players largely seemed to persist in incremental development. They believed DARPA should take steps to get involved and help speed up ambitious development and research work in the field.

Beginnings

The demand and usefulness of the eventual MOSIS service were first proven through Carver Mead’s chip design course at Caltech — leveraging a more computational, less manual pipeline for designing chips he had developed — and subsequently through courses modeled after Mead’s course at MIT and elsewhere.

Mead wanted to teach his students how to design VLSI (Very Large-Scale Integration) chips. Since there was no single industry standard to teach to because each manufacturer had its own protocols, Mead developed a general set of protocols out of the disparate standards that would allow students to learn the principles in a specific way — but in such a way that the students would still be useful to any arbitrary company they may go work for after graduation. Above all, Mead wanted students to learn to design chips by actually designing their own chips.

To make this goal feasible, printing the chips would have to be cheaper. So Mead figured out a way to put many chip designs onto the same wafer to reduce printing costs. Mead turned to Lynn Conway — Head of LSI System Design at Xerox PARC — and Xerox PARC fulfilled the role of silicon broker in this first iteration of the course. And it was a success. Mead and Conway succeeded in the course's goal of trying to ‘totally isolate the designer from all the trivia that fabrication requires’ to enable students to learn chip design faster.

These learnings and design standards were eventually codified into what became the bible of VLSI design, a book co-authored by Mead and Conway titled Introduction to VLSI Systems. This book and the course's approach were how the course’s teachings began to reach more schools and researchers. The approach demystified chip design and enabled researchers to learn chip design in a course setting — rather than having to consult for a chip manufacturer to learn all the little tips and tricks of the trade as was common at the time. You didn’t need to know everything about chip manufacturing to design workable chips anymore. In 1978, Conway went to MIT to teach MIT’s iteration of the course for the first time. By 1980, things were progressing. In the spring alone, “250 designers from 15 universities and R&D organizations submitted 171 projects for implementation.” These specific chips were fabricated by HP.

With the potential of the service proven at several schools and across multiple manufacturers, Mead, Conway, and others began pushing more strongly for a DARPA project to take over and scale this work. At the time, most DARPA PMs made efforts to be in close contact with as many potential PIs as possible in relevant research areas. In this particular case, Conway — while at Xerox PARC — began pushing Kahn — then head of IPTO — to push this work forward. Although, it is very likely that if Conway had not made this push, Mead could also have reached out to one of the IPTO PMs who frequently worked with CalTech.

The Information Sciences Institute (ISI) — which had an affiliation with USC but was independent of the university in many ways — was granted the role of DARPA “silicon broker.” In the 1980s, they became one of IPTO’s go-to performers to carry out a variety of services like this which required research familiarity as well as the organizational skills of a company — such as orchestrating the Machine Acquisition Program in the adjoining piece. ISI was started by a group of researchers who felt their former employer, RAND, was too geared towards paper studies and did little in the way of directing research and work on technology itself. ISI was an unfunded research arm of USC — paying its own way through (mostly DARPA) grants and contracts. Their regular work included performing services for DARPA, conducting applied research, creating technology transfer programs, and more. ISI had often conducted work somewhere in between applied research and DARPA program administration before, and, thus, was very well-suited to this new role.

Operations

Once the program was established, the user experience was quite straightforward. Those who had access to MOSIS — largely NSF researchers, DARPA grantees, and other individuals whose funds were largely from the government — communicated with the MOSIS central office via the ARPANET. MOSIS put together a user manual explaining how electronic mail via the ARPANET worked — those were the early days of the internet — and how to email designs and queries to the mail system in a structured way so a program could sort them.

Researchers sent their designs — in CIF files — to MOSIS. MOSIS, initially using a system designed by Ron Ayres of ISI, took it from there. The system:

• Checked the CIF file to ensure it conformed to the basic design standards outlined by MOSIS in the user manual.

• Packed sets of projects onto a smaller set of dies.

• Translated each die into MEBES format.

• Made bonding diagrams for manufacturers.

• Most importantly, produced tapes that the foundries used to make masks.

Danny Cohen and George Lewecki of ISI, in a 1981 talk on the early MOSIS system, described how the process continued from there, saying, “The next step in the process is mask fabrication. Mask houses expect two types of things from us: tapes with MEBES files and job decks. MEBES files contained the information that the mask houses used to make bitmaps (which are made into masks). A job deck…contains the specifications for each MEBES file — parity, record size, etc.”

The talk continued, “Fabrication itself is very simple because somebody else does it. Once the masks are made, all we have to do is drive three, four, maybe ten miles in Silicon Valley with the masks to a wafer fabricator. (It is wise to drive slowly to make sure the masks don’t break.) After that, if we’re lucky — and typically we are — we end up with a couple of wafers.” MOSIS completed its job by dicing, bonding, packaging, and shipping chips.

The MOSIS staff also took measures to ensure that they were as ‘lucky’ as possible. And when they weren’t lucky, they made arrangements that ensured as few defective chips as possible were handed to users. They accomplished this in several ways. MOSIS carried out basic probing tests on transistors, inverters, etc. ISI kept track of defect rates for any given type of job from all of their fabricators to 1) ensure that they continued to work with the best fabricators possible and 2) that they ordered enough duplicates of any given part for a researcher to ensure a 90% probability that a given print would yield a working chip for the researchers — helping keep research projects on schedule.

ISI also carried out different forms of applied research to ensure progress in a variety of operationally important areas. Some of these areas included exploring the feasibility of expanding the service into new chip types, working to incorporate new coding language into ISI’s workflow, using new varieties of machines in their work; etc. In general, it was ISI’s job to do any research work required to make the service work as efficiently as possible and take advantage of the rapid technological improvements happening in the field of computing. While the ISI team did not set out to do fundamental research in its own right, the ISI team’s skills as researchers may still have been important to the project’s success. In the hunt to make chips smaller, new technical problems constantly arose in fabrication processes. Courses of research were often required to work through these problems.

One ISI research project involved finding a way to translate the different design languages used by researchers into those used by manufacturers. Another case — discussed in the Cohen and Lewecki talk at CalTech — outlined how the ISI team worked around a difficulty in testing new softwares to see which software was best:

The problem arises in comparing two masks, one produced by the old [software] and the other produced by the new system. Are the patterns really the same? A microscope is supposed to help, but it can’t do a good job. We tried many ways and finally worked out a very strange technique. Suppose you want to compare mask A and mask B. What we did was to overprint A and B bar [the reverse of B], and A bar and B. In this way we discovered all the changes. We did all the printing on one plate so we wouldn’t have to use a special microscope.

One final — often forgotten — aspect of MOSIS’ success was that it provided a layer of security and confidentiality to researchers and manufacturers. Researchers came to trust ISI agents and their system for communicating design instructions to manufacturers. As Roland and Shiman — who wrote an in-depth history on DARPA’s 1980s computing efforts — noted, “Only ISI had the electronic version of their designs; versions on masks or even on the chip itself were prohibitively difficult to decode.” Manufacturers also maintained open lines of communication with ISI as they became convinced that ISI respected their proprietary information. Many recognize that ISI’s service in this project was ill-suited to a traditional university department. However, as this remark from Roland and Shiman also indicates, this particular piece of ISI’s service may have been hard to replicate for certain companies in DARPA’s performer pool as well. 

Results

The MOSIS program is widely considered a success.

The number of wafers MOSIS printed rose steadily throughout the 1980s as demand increased and the field developed. The number of wafers printed grew from 258 in 1981 to 1,790 by 1985. The cost of the average chip for a researcher often came in around 5%-10% of what it would have cost if a researcher had to pay for the entire wafer. In 1988, George Lewecki noted that a single 2µ chip in a MOSIS run could cost as little as $258 (~$650 today). For the most part, DoD and DARPA contractors got the service for free, NSF grantees paid for the service out of their research grants (and eventually received the service free), and any commercial actors had to pay full price.

Throughout the 1980s, the average turnaround for chip times from MOSIS gradually decreased from around 10 weeks to about 8 weeks. The ability to print smaller and smaller features for a given cost and time frame continuously improved by large margins as the field evolved. MOSIS’ somewhat stagnant turnaround time was a conscious decision because allowing more projects to fill the queue helped ISI take further steps to reduce costs.

MOSIS was considered a quite efficient operation. A 12-month period from 1987-1988 saw $9 million flow to the MOSIS program. George Lewecki noted that 74.3% of MOSIS expenses went to fabrication costs, 13.1% to salaries, and 11.4% to ISI overhead.

Lessons Learned (and Caveats)

Some in industry were annoyed that MOSIS might support future competitors of theirs. Private researchers could get free access to resources that TI, Intel, and IBM had to pay for. These researchers were not just looking to expand the knowledge frontier, but spin out companies from their university research if things worked out. Conversely, some within the government were annoyed that the MOSIS service — which was primarily targeted at university researchers — could be used by commercial actors at cost. Essentially, they felt the years of infrastructure and know-how MOSIS built up were not being priced in, and the service was essentially providing an industry subsidy.

To this writer, those criticisms are not to be taken too seriously. But a third area of criticism is worth examining. Some didn’t like that MOSIS seemed to be a never-ending program. MOSIS was providing a service that required some research and also helped the research community, but once it settled into its patterns it was not considered high-risk — even if it was seen to be high-payoff. DARPA tended to focus on higher-risk R&D projects. However, MOSIS also did not seem like a sure-fire commercial bet to spin off into its own firm. While the service was successful, it was not considered a good bet for venture capital by many familiar with the program. It was more of a service and contemporary VCs preferred products. Additionally, 98% of the service's customers — in this young and research-focused area — were from DARPA, NSF, or DoD funding sources. Competing firms had little need for MOSIS' main service of subdividing wafers. 

If left on its own in the private sector, one could not be sure MOSIS would survive. For example, it could end up in some sort of ruinous cost-cutting competition with a competing manufacturer and end up out of business. But, still, the service was too good to terminate because it offered a cost-effective way to support a vital area of research and teaching. So, instead, MOSIS was moved under the umbrella of more traditional defense funding to continue operation indefinitely. Funding for the program by the DoD and NSF ended in 1998. However, the program does still exist today under the ISI umbrella at USC. Its continued existence — in an era of very different technological conditions than those that created the original need for the program — may be considered, by some, a data point in favor of “sunset clauses” for certain projects. But I will not address that point further as it is somewhat out of scope of this series.

All in all, the program is considered a success. The MOSIS program succeeded in lowering the barriers to doing ambitious research in a broad research area whose progress was all-important to DARPA and the DoD’s computing goals; microelectronics research could be done by individual researchers for a fraction of the cost it would have previously cost. Additionally, as Kuan and West write, MOSIS “modularized [decomposing a problem into separate modules] the semiconductor ecosystem” by enabling semiconductor design to be done productively on its own. With MOSIS, DARPA was able to help alter the structure of a field in a way that industry incumbents may not have done on their own — but was likely better for the progress of the field.

Previously, design was generally done by those with the skills, experience, and machines to manufacture semiconductors — large industry players. The MOSIS service was able to help unbundle these two separate tasks and pave the way for the semiconductor industry to subdivide into design-only  “fabless” firms and production-only foundries. With so much of this fabless design research being done by the academic community — rather than existing private sector incumbents — the field itself was able to be built out in the open, with researchers making rapid progress by leveraging one another’s discoveries.

Some might believe that rising tides lift all boats and, thus, any infrastructure program in an area that turns out to be explosively productive — like mid-1980s computing hardware — will look good in retrospect. However, one should keep in mind that not all of DARPA’s Strategic Computing infrastructure projects with similar goals from this same era are considered a success in a similar vein. For more on that point, see the accompanying piece on DARPA’s Strategic Computing Machine Acquisition Program. Some feel there could have been a world where the MOSIS investment’s heavy reliance on Mead-Conway standards artificially incentivized researchers to develop λ proportions at the expense of others. The accompanying piece describes a program that made a somewhat analogous error. However, I believe that MOSIS’ placement as more of a broker than a capital purchaser provided it a greater level of flexibility to work around this category of issue that ailed the Machine Acquisition Program.

Check out the adjoining piece here

• Strategic Computing's Machine Acquisition Program

Thanks for reading! Subscribe to continue following along with the series!

Subscribe now

Share

Specific Links:

• MOSIS User Manual .txt file

  • Particularly useful if the reader wants to understand how MOSIS instructed self-onboarding onto a complicated technical service.

• Danny Cohen and George Lewecki giving a talk at Caltech on the project in 1981

  • Particularly useful if the reader wants to understand: 

    • How MOSIS ran into problems and solved them using research or other qc measures

    • How the MOSIS service operated in more detail

• Chapter 4 of Strategic Computing: How DARPA Built the Computer Age

  • Particularly useful if the reader wants to understand how MOSIS progressed from just an idea, through its early years, and into a mature program. 

  • Also covers the politics involved in maintaining the program. 

  • Additionally, any details about specific chip design measurements and program decisions are covered at moderate length in this chapter.

• Kuan and West’s Interfaces, modularity and ecosystem emergence: How DARPA modularized the semiconductor ecosystem

  • Particularly useful if the reader would like to understand (in more detail) how MOSIS was able to unbundle design from manufacturing capacity and the effects of this on the commercial and research actors in the space.

• Misc.

  • 1983 ISI report on the present and future of MOSIS

  • MOSIS: A Gateway to Silicon

  • MOSIS 1983 Annual Technical Report

  • Ronald Ayres narrative history of MOSIS software

General Links:

• Strategic Computing: How DARPA Built the Computer Age 

• Introduction to VLSI Systems

• Keith Uncapher’s Oral History

  • Dives into the founding of ISI as well as the significance of the MOSIS system.

• Robert Kahn’s 1989 Oral History

  • Briefly dives into the overarching need for the program and industry’s trepidation to provide fab capacity to the program at the start.

Pattern Language Tags:

• Promoting a coordination/service mechanism to reduce material costs and increase research feedback cycles

Introduction

In the early 1980s, as DARPA’s Strategic Computing Initiative was getting underway, lack of cheap computing power was one of the primary concerns of DARPA performers. The work of MOSIS in reducing the cost of VLSI chips for chip research was one major “infrastructure” investment by DARPA that helped overcome a cost issue in an area of strategic importance for the research community. Another group of performers plagued by a similar problem was DARPA’s corp of software/AI researchers and their need to acquire high cost machines to continue with their current research agendas. PM Ronald Ohlander — who managed many AI projects for IPTO as the Strategic Computing program began — undertook the SC Machine Acquisition Program to help DARPA researchers overcome this cost problem. The program — carried out in a similar period to MOSIS and riding a similar technological wave — would have much more mixed results than MOSIS.

Beginnings

With the launch of SC in 1983, Ronald Ohlander launched the Machine Acquisition Program under the SC umbrella. The program was meant to help boost the productivity of SC’s AI research community. The Machine Acquisition Program would use SC resources to buy those in the AI and software research community LISP machines — the high level programming language popular for AI research at the time — to meet their needs. The program sought to do so with significant cost savings by negotiating bulk discounts with manufacturers. One major drawback was that these specialized LISP machines could only be networked to other LISP machines at the time and could not be time shared, but the AI research community was confident that these machines would be a massive factor in enabling cutting-edge AI research in the coming years.

So, Ohlander went forward with the program.  ISI — the org that had won the MOSIS contract — also won the right to run this much lower touch contract. Approximately six months after ISI first proposed to serve as the machine acquisition center for SC, it had won a sole-source justification to service the early part of the contract and made its first purchase — in March 1984. With ISI’s first purchase of 30 Symbolics 3600 LISP computers — which cost a total of $2.8 million total at the time (~$8 million today) — the program was off and running.

Operations

ISI charged no overhead rate to DARPA for the contract and only asked to be compensated for the cost of the machines purchased and the time and resources ISI invested into the contract. The primary service provided by ISI — besides buying machines for researchers on SC’s behalf — was working with the staff at computer manufacturers to negotiate bulk discounts for the researchers. Given how limited the market for LISP machines was — which is primarily what the program would specialize in buying — the demand for LISP machines by DARPA researchers represented a substantial portion of the market. So, ISI’s job was primarily to find a way to negotiate deals so DARPA researchers’ collective demand could translate to lower prices. 

Beyond this function, ISI carried out a handful of services to enhance the user experience of the researchers and limit the headaches that are sure to arise when acquiring new machines in a young manufacturing area. ISI would operate the machines at its office for about thirty to sixty days to ensure there were no problems. And, when there were problems, they would coordinate with the manufacturer to either repair the machine or have a replacement sent. 

Ohlander and DARPA required that Common LISP be offered on every machine purchased by the program and pushed the research community and industry to settle on the specification of a common LISP language. Ohlander had taken the temperature of the AI research community and they felt that LISP was clearly the best language for their work now and would continue to be moving forward. So, Ohlander and DARPA did their part to ensure that these researchers had more LISP machines sooner rather than later, that they could be acquired as cheaply as possible, and that the community settled on common LISP standards so they could seamlessly build on each other’s work and communicate findings. 

Results

By the end of 1985, when the program was in full swing, the Machine Acquisition Program had spent over $8 million on LISP machines for the SC research community. These machines would have retailed for around $50,000 to $100,000 but were acquired for about a 35%-45% discount by the ISI team. Additionally, ISI often secured massive discounts for additional services such as follow-up maintenance from manufacturers. On one occasion, ISI even secured a maintenance fee of $76,176 from Symbolics for services with a book value of $495,000. DARPA received all these cost savings for only the service cost paid to ISI — which was equivalent to about 2% of the cost of what they spent on these massively discounted machines. 

However, the program quickly encountered a change in the technical landscape that caused some headaches. Ohlander had set up the program to give the researchers as much choice as possible. DARPA was not going to tell the researchers which computers were best suited to pursue their research goals. So, ISI planned to buy whichever machines DARPA’s performers asked them to buy. The program was set up with the four main 1983 manufacturers of LISP machines in mind — Symbolics, TI, LISP Machines Inc, and Xerox. The LISP machines from these vendors were supposed to represent the near future of the field — so the researchers had said. But as early as 1986, the research community had almost entirely shifted its preferences, now preferring the program to acquire them the new general-purpose workstations such as Sun’s Sparc, which retailed for three or four times less than the LISP machines. These machines, made possible by substantial commercial advances in computer architecture and manufacture, were powerful enough to run LISP and more versatile than the more expensive, specialized LISP machines.

The program’s activities contracted substantially during the budget crisis in 1987 and 1988. When the program revived, preferences had once again begun to shift. Performers were beginning to become interested in the new lines of parallel processing machines — which SC had spent significant resources helping develop. 

Lessons Learned (and Caveats)

Between 1984 and 1992 the Machine Acquisition Program spent over $15 million on equipment for its research contractors. From a cost-savings perspective, the program was absolutely a success. And from a short-term point of view, all parties had their needs satisfied. Researchers got the machines they wanted when they wanted them. ISI even did a lot of installation and debugging for them. Manufacturers in the relatively small market for LISP machines received a much appreciated demand boost. And DARPA was able to speed up the velocity in which research was done in its AI community, receive its products and services at a substantial discount, and get the research community to agree on common LISP standards to ensure its money was spent on researchers who could iterate on each others’ work as quickly as possible.

But does all of that overshadow the fact that the majority of the money the program spent was on machines that were essentially obsolete within a couple years of their initial purchase? To use an objective term like “mistake” with the benefit of hindsight is probably unfair. The research frontier is constantly shifting and the role of people like DARPA PMs is to shape that frontier, adapt rapidly when things change, and also make peace with the fact that money will sometimes be spent on dead ends. Whether or not the decision to fund the program was considered a mistake internally would depend on whether Ohlander and DARPA management had adequately considered this possible outcome and accepted the risks, or if they were somewhat blindsided by the outcome.

Looking at the landscape as they would have seen it, in 1983, is a useful exercise for PMs. In 1983, those who worked on SC programs generally understood that most of the AI/software applications coming out of the work SC was funding would build on top of the computer hardware and architecture developments DARPA was funding in the early 1980s. While projecting technological developments can be messy, it seemed quite clear that orders of magnitude improvements in computing were coming and would impact the researchers who made use of computing equipment in five years or less. In a very active way, SC was explicitly aiming to help achieve the orders of magnitude improvements in computation that they knew to be possible as rapidly as possible. In fact, as I explore in my piece on SC’s three main applications (coming out this Tuesday), some of the programs were counting on one or two orders of magnitude improvements coming by the end of the decade. The rapidly evolving hardware landscape was an area that many SC PMs kept even more abreast of than those in the AI research community. Yet, in spite of the rapid changes occurring in the field, the Machine Acquisition Program invested a yearly amount similar to what SC was spending on MOSIS to buy machines that were only useful to the AI research community as long as the current state-of-the-art only improved along a very foreseeable path — foreseeable to the AI researchers in particular. If Ohlander and DARPA knew this might happen, but felt $15 million was a small cost to pay for getting the AI researchers moving on their projects as fast as possible, it would be hard to call the work this program did a mistake. But if DARPA simply placed too much faith in the foresight of the AI research community when it came to near-term hardware developments, then this is surely a lesson to learn from. 

Taking a step back and comparing the similar MOSIS program to Machine Acquisition, it does seem that the MOSIS budget had less downside risk in spite of also being a DARPA program dedicated to investing in subsidized computing hardware for researchers. The MOSIS budget was spent on infrastructure that was less likely to be rendered obsolete by technological advances elsewhere in the SC portfolio or industry. In addition, MOSIS also put itself in the (less risky) position of broker, rather than the (more risky) position of large equipment purchaser. The Machine Acquisition Program was on the hook for a lot of obsolescent, big-ticket hardware that cost $50,000 to $100,000 per unit. MOSIS, on the other hand, was simply paying others for runs on their big-ticket hardware. Given this difference in capital ownership, the MOSIS program was better structured to adapt to unforeseen developments such as those that befell the Machine Acquisition Program. The ISI team, surely, would have been forced to scramble in response to new research developments showing that VLSI standards or λ proportions were no longer useful and the future looked different than they expected. But it does not seem that they would have been in as much trouble as the Machine Acquisition Program if they guessed wrong. 

MOSIS had its risks, but it did not share with the Machine Acquisition Program the additional risk of government-purchased big-ticket research equipment sitting in the corner of a lab and serving minimal purpose to the research movement. For MOSIS, if certain specs proved obsolete, certain manufacturers might be similarly hit hard, but MOSIS could simply switch runs to those who were better suited. Of course, MOSIS could only make this switch once it re-wrote some of its software and re-designed some of its services. And while that is inconvenient, this sort of reworking, such as re-writing software or adjusting the Mead-Conway standards,  was already built into the organization’s processes. Minor changes of this sort were routine. It is likely that MOSIS could, within reason, likely do whatever internal research work was required to adapt their service to the new world and continue on — shaken but in decent standing. 

In summary, the Machine Acquisition Program, in comparison to MOSIS, both incurred more of a financial burden in terms of expensive machine ownership while also being more liable to having those machines rendered obsolete by other investments in SC’s portfolio. The program could still be considered a worthwhile investment if program managers believed one of the following:

• It was important to get the AI research community working on DARPA applications projects sooner rather than later. This was likely at least partially a factor since DARPA Director Robert Cooper is said to have insisted that SC begin work on applications programs concurrently with the programs developing the technology base.

• There was a chance that very useful near-term applications and learnings would arise from the AI researcher community’s current lines of work with LISP machines — even if they did go obsolete quite quickly.

• The PMs considered the money spent and the known risks incurred a small price to pay to seed early work in an interesting research area.

Assessing the project’s execution, it is unclear if the researchers — who were allowed to purchase whatever equipment they wanted — were effectively incentivized and informed enough on the state of hardware research to properly make purchases with DARPA’s money. From one POV, the researchers are the experts and should have final say on what equipment is best to carry out their own research. But, on the other hand, all of the financial downside — and incentives for making prudent decisions like waiting three or so years before spinning up a program like this in the AI community — lied with DARPA. Some could accuse those in charge of the program of not properly weighing SC’s own internal knowledge of the developing technological frontier in hardware research when purchasing millions in soon-to-be-obsolete machines. In retrospect, Ohlander and DARPA could have weighed their own knowledge of the evolving microelectronics landscape more heavily than they did when making decisions on this program. 

But it is probably not worth dwelling on what could have been. PM work is often about making “big if true” bets and being willing to incur the losses as long as the wins are big wins. But it is still helpful to know about and learn from DARPA bets that did not pan out. And Machine Acquisition is, to most, a bet that did not pan out. To be sure, the Machine Acquisition Program should probably be remembered for the money it saved as well as the money it had a part in spending fruitlessly.

The Machine Acquisition Program had similar goals and (somewhat) similar strategies to wildly successful DARPA projects like MOSIS (with its goal of reducing chip costs with shared fabrication runs), the ARPAnet (with its early goal of using the network to share research machines and reduce costs), and DARPA’s computer time sharing program (which enabled more researchers to use a single machine). In the end, the Machine Acquisition Program’s outcomes were simply a victim of the risks that Ohlander and SC leadership (hopefully knowingly) took when they acquired infrastructure for researchers in a field whose equipment was rapidly evolving. 

While the rising tide of computer chip development did raise all boats in the computer research field, the tide did not have the same positive effect on all existing projects within the SC portfolio. The Machine Acquisition Program provides a useful case study for those making investments that build on top of a rapidly evolving field of technology.

Thanks for reading! Subscribe if you’d like to continue following along with the series.

Subscribe now

Share

Specific Links:

• Chapter 4 of Strategic Computing: How DARPA Built the Computer Age

  • Particularly useful if the reader wants to understand how Ohlander attempted to subtly nudge researchers away from inordinately selecting Symbolics machines. Symbolics, in the beginning, was the only vendor that had a track record but also was the only vendor that did not offer ISI a steep discount.

  • Also contains further details about the program’s early history and various operational details.

• ISI’s Annual Reports

  • Section 16 of the 1985 Annual Report outlines the early goals and work of the program. Additionally, this section briefly outlines some areas of military application ISI believed the research area could eventually contribute to.

  • Summer 1983 - Summer 1984 Annual Technical Report contains brief descriptions of ISI’s first year of acquiring machines and operating the program.

General Links:

• Strategic Computing: How DARPA Built the Computer Ag

Pattern Language Tags:

• Pushing forward the technological base via speculative machine building contracts

• Gray coding

• Incentivizing a group of related goals by organizing them all into a single contract

• Selecting multiple industry contractors for a trial period before choosing the final contractor

• Funding a conservative project to use as a measuring stick for more ambitious projects in the area

Introduction

DARPA’s multi-decade investment in developing early versions of parallel processing computers provides a lot to learn from for ARPA-style PMs. Two of DARPA’s most prominent, long-standing investments in this area took two completely different approaches to push the technology base forward in a coordinated fashion. Both are viewed as successes to some and abject failures to others. In this piece, I will dive into the operational structures that informed DARPA’s investments both in the ILLIAC IV parallel processing computer in the late 1960s — the project was based out of the University of Illinois under a researcher named Daniel Slotnick — and in the young Thinking Machines Company in the 1980s and their Connection Machine computer — which brought them fame and infamy in the computing world in the years in which DARPA was most heavily involved with the company. 

To start, I’ll dive into DARPA’s investment in the ILIAC IV. The cost overruns and time delays for which this project became internally famous very heavily informed the approach that Stephen Squires took to managing DARPA’s investments in Thinking Machines and his computer architectures portfolio as a whole in the 1980s.

ILLIAC IV’s Context

When DARPA Director Charles Herzfeld took over in the mid-1960s, computing research was not nearly as pressing of a defense research issue as others like ballistic missile defense or nuclear test detection. The previous director, Jack Ruina, often only met with the then-IPTO Director J.C.R. Licklider once or twice a month. Now, that is in large part because he both trusted Licklider and attempted to have a hands-off management style. However, it also speaks to the lack of priority that the Information Processing Techniques Office (IPTO) had within the DARPA portfolio at the time. PMs in hotter areas of the portfolio were said to have quickly met with Ruina as many as five times a day to discuss small issues. 

As the reader surely knows, great work was done by Licklider and his portfolio of performers in the early 1960s with this approach. As Herzfeld took over, the progression of the field was apparent to him and he thought it had the potential to help improve programs all across DARPA and the DoD. He recounted his approach to managing the IPTO as he took over, saying:

I thought that my job vis-a-vis the computing program, when I was Deputy Director and Director, was first of all to make sure to get the best office directors we could get, and second, to help them do what they thought they needed to do, and third, to look for applications across the projects for computing, when you apply the new computing ideas, technology and its capabilities as widely as possible.

Ivan Sutherland — Licklider’s young, hand-picked replacement who had gotten his Ph.D. at MIT and had been at CMU for several years since graduating — became IPTO’s Director under Herzfeld. Sutherland, while young, had been making a name for himself in his field. His Sketchpad program, written for his MIT thesis, would later help win him a Turing Award.

Herzfeld also lived up to his word of helping good ideas from this office find funding. One example of this was when, in 1965, Robert Taylor brought forward the network proposal to Herzfeld that would become the ARPANET. Herzfeld, thinking the idea had promise, quickly determined that ARPA should fund the idea and took some money that had been set aside from a lower-priority program — almost $500,000 (~$5 million today) — to get started immediately. They would get funds from Congress to continue the program in the following fiscal year. This kind of administrative move, while not rare in managing the DARPA budget at the time, was more likely to be done for higher-priority projects.

As IPTO’s technology base was becoming more mature, its budget began to increase and PMs were looking to fund projects that were further along in the development process — IPTO’s early-1960s budget had been almost entirely allocated to early-stage research.1 One development area of interest was the idea of building a more efficient computer leveraging hardware that was more parallel in nature. If implemented successfully, it was thought that the machine could help the DoD solve many pressing problems that required more computing power — such as calculating nuclear weapons effects or real-time processing of radar data.

ILLIAC IV’s Beginnings

In 1965, Sutherland reached out to Daniel Slotnick a month after Slotnick had arrived at the University of Illinois to set up a visit at Slotnick’s new office in Urbana-Champaign. Slotnick had become acquainted with Sutherland while he was working on his SOLOMON parallel computing machine at Westinghouse; this area of work at Westinghouse had many DoD ties. In their meeting, Sutherland would ask Slotnick if he was interested in developing a large parallel computer at Illinois. 

At least, that is how the events unfolded according to Daniel Slotnick’s IEEE memoir. Sutherland, in an oral history conducted by William Aspray, seems to remember things differently. According to his memory, Slotnick came into his office while he was still at Westinghouse and funding for his project was drying up, and an interaction resembling the following ensued:

He came into the office and said," Basically, I want to do these things and I cannot do it at Westinghouse, because they will not let me do it. Where should I go?" I introduced him to some of the university folk. So that one was fairly a long time in the doing.

This, also, is a quite believable ordering of events given my understanding of how things worked at IPTO in the 1960s. Whether Slotnick’s account or Sutherland’s is more accurate does not matter much. Regardless, in 1965 Slotnick and Sutherland had a conversation about building a parallel computing machine — which they had clearly discussed before — and Sutherland let Slotnick know that this might be a good time to submit a proposal now that he was at the University of Illinois. When the proposal was submitted, other IPTO staff as well as Herzfeld supported funding the proposal.2 In the next few paragraphs describing the details of how the finer points of the contract materialized, I’ll rely on Slotnick’s more clear account of the events. Certain small details might differ between the two accounts, but none that should matter much for the takeaways of this piece.

The project would be quite similar to Slotnick’s work at Westinghouse, and it was an area of obvious interest to IPTO’s mission. After Slotnick thought it over, he agreed. Sutherland initially proposed starting the project with a small study phase. But this tentative approach was not agreeable to Slotnick. Arriving at Illinois, the disappointment of the SOLOMON project was fresh on Slotnick’s mind. Slotnick’s intensive work on the SOLOMON project jointly funded by the DoD and Westinghouse for two years, Slotnick’s Westinghouse team steadily grew to around 100 people. In spite of the team’s promising early work on the technology, the project’s steady progress came to an abrupt halt when the project’s main sponsor in the DoD suffered a tragic drowning accident. The man’s replacement did not back the project in the same way. Quickly, the project went from pushing for the funds to build a full-scale version of the machine to having no funding at all. Not long after, Slotnick found himself at Illinois.

With this disappointment fresh on his mind, Slotnick said he was “anxious to do something else.” He only wished to pursue the project with DARPA sponsorship if Sutherland meant business.  Slotnick described his reaction to Sutherland’s proposal of a modest seed funding stage as insistent:

I absolutely refused. I wanted a $2 million payment at the outset and a contract for a total of $10 million. I did this to make sure that the ARPA commitment was real and had passed the highest levels of review.” Sutherland must have understood. Slotnick continued,  “Ivan agreed. I wrote the proposal and a few weeks later we had our contract.”

The proposed machine was heavily inspired by Slotnik’s time spent developing the SOLOMON prototype machines. The newly proposed machine — the ILLIAC IV, named after three earlier computers built at Illinois — sought to achieve over one billion operations per second. Beyond the project’s aim to produce a machine with a substantial improvement in computing power, its management sought to roll out improvements to much of the component technology in the machine. When all was said and done, Slotnick and the DARPA team involved believed the machine would have several useful areas for military application. DARPA and the armed services research organizations planned to deploy the resultant machine to help advance research problems in climate dynamics, anti-submarine warfare, ballistic missile defense, and the army’s multiple target problem. Additionally, a nuclear research office was interested in the capability of the computer to help solve the climate modification problem. Lastly, IPTO itself was particularly interested in the computer’s applications to researchers in cryptanalysis, simulation of fluid flow, seismic array processing, economics, and other problems ideally structured to utilize the computer’s parallel processing capabilities.

Sutherland — as the IPTO Director — was very explicitly attempting to use the ILLIAC IV project to push the boundaries of several component fields of computing simultaneously. As later-DARPA Director Stephen Lukasik described the ILLIAC IV team’s full frontal assault approach, IPTO and DARPA were “pushing machine architecture, software for parallel machines…medium scale integrated circuits — and then the application of that to a variety of important defense problems.”

ILLIAC IV’s Operations

In addition to this project helping push forward the technology base in the area of parallel computing, Sutherland hoped that it could help spark some efforts from industry in building new computer components relevant to this general area of technology. So, while the overarching contract for the project was issued to the University of Illinois team under Slotnick, the work of an industry subcontractor to build the machines the Illinois team designed would also be vital to the project’s success. In addition, contracts for the research and engineering development of various components of the machine would be given out to other parties from the IPTO performer pool. However, the work of these additional parties would generally be less important to the project’s success than the work of the Illinois team and the computer manufacturer chosen to produce the final machine.

The Illinois team’s initial design specified a machine with four modules of 64 processors under the control of a single construction unit which would allow them to simultaneously work on the same problem. As Slotnick began to assemble a team at Illinois, the team also began to figure out which computer manufacturer to partner with. Slotnick was not upset by the project’s partnership structure. From the beginning, he felt this was a natural approach to doing a project like the ILLIAC IV. Slotnick had come from industry and understood the comparative advantages of industry and academia when it came to designing and building new computers at that time. As Slotnick put it, “The days when a university could design and fabricate a big machine by itself were over, and we decided that while we would do the architectural design and most of the software and applications work, we would rely on industry for detail design and fabrication.”

Since selecting the right contractor to build the machine was pivotal to the success of the contract, Slotnick and the DARPA team held a competition. Slotnick and the DARPA team outlined the major approaches and incorporated them into a bid set. In August 1966, after “many months of intensive contacts with industry,” RCA, the Burroughs Corporation, and Univac were all awarded 8-month contracts to continue working on the early stages of the contract and collaborating with the Illinois team. After this period, in 1967, the Burroughs Corporation was selected as the best fit to continue on as a partner in building the ILLIAC IV. Burroughs had partnered with Texas Instruments in its bid. The plan was for TI to develop the integrated circuits for the machine’s all-important process element (PE) array. With that trial period over and the DARPA and the Illinois teams having gotten the chance to work with all three companies for a time, the industry partner for the project was decided upon.

Either Larry Roberts or Robert Taylor — Taylor had become the new IPTO Director in the intervening period and Roberts was the PM who focused on the ILLIAC IV project — and the Illinois team awarded the related contracts to work on other component parts of the ILLIAC IV to the following non-U of I teams: 

• The design and implementation of the FORTRAN compiler for the ILIAC IV to the Applied Data Research Corporation

• Various hardware and software to help link the ILLIAC IV and a trillion bit memory with the ARPANET to the Computer Corporation of America

• And work on the development of the interactive front end processor to BBN.

As the project progressed, the design and construction phase was riddled with research problems and setbacks that forced design changes on the research team, delays, and cost overruns. Two main issues contributed to the early cost overruns and time delays in the first couple of years of the project. The first was TI’s inability to produce the 64-pin emitter-coupled logic (ECL) packages. This caused a delay of over a year and a $4 million cost overrun. Slotnick documented that TI was not exactly forthright in taking the blame, writing, “A great deal of inconclusive finger-pointing went on between TI and its suppliers.” Regardless, the problem — and loss of cash and time — remained. The second major problem was Burroughs’ inability to make workable magnetic thin-film PE memories for the evolving design of the machine. This problem caused an additional year of delay and another $2 million cost overrun. Additional problems existed with packaging, circuit design, and interconnections for the large machine. But these two problems, in the early years of the project, caused the most headaches.

Slotnick, in his writings, explains how these very early setbacks began to affect project planning. Certain goals became more modest and the team had to substitute in and out certain component technologies to fit in better with the evolving design and changing circumstances. Slotnick writes: 

We retreated from the 64-pin packages to standard 16-pin dual in-line packages. In doing so, however, everything got bigger and more expensive. A lot of logic design was salvageable as a consequence of making the 16-pin packages logically derivative of the abandoned 64-pin packages, but board layout, back-panel wiring, and all system level hardware, of course, had to be restarted from scratch. The memory situation was even messier. Burroughs had a large investment in its thin films and didn’t want to give up on them even after my own independent review had concluded they still represented an intolerable development risk. Semiconductor manufacturers were just beginning to gear up for memory chip manufacturing; the manufacturing means were clearly at hand, or at least so it seemed to me, but the chips were not. I made the painful decision to drop films and go with semiconductors.

In making this semiconductor pivot, Slotnick believed the young Fairchild Semiconductor was easily the most qualified supplier. While Burroughs railed against the selection of the young company, a panel of independent ARPA experts — assembled due to Burroughs’ ‘furor’ at Slotnick’s selection of Fairchild — confirmed the decision. Fairchild seemed to be one of the only — if not the only — suppliers delivering high-speed semiconductor memory at the time. 

As the project wore on, the projected cost of the project roughly tripled and Slotnick was now only promising to deliver a system a fourth the size of the one originally promised. In spite of these shifts, Larry Roberts — who would become IPTO Director himself and who was already known as the “father of computer networks” — remained behind the project. Not only did Roberts, as Slotnick jokingly put it, share in his “sick attachment to really big pieces of hardware,” he was very interested in the list of applications which the machine was set to work on — including numerical weather prediction, sonar, radar, seismic signal processing, and an assortment of other computations array computers tended to do well. But, as Slotnick saw it, to justify the escalating costs, making the machine available to as large a community of users as possible became even more essential. The larger the scale of users the machine achieved, the more justified the escalating costs would be. Slotnick believed that this was a major factor that drove making the ILLIAC IV easily available over the ARPANET to be a much stronger point of emphasis for the project as the years wore on.

In 1970, five years into the constantly delayed project, the Illinois team was finally beginning to hone in on a workable machine. But, even then, there was still an element of instability in the project’s operations. From a technical standpoint, things were beginning to get on more sure footing. The PE had been successfully redone with the new TI integrated circuits, PE memory chips were being delivered by Fairchild, and other pieces of equipment were getting produced on schedule. However, cultural conflicts were now coming to a head on the Illinois campus. Not only were stresses between Slotnick and those faculty not involved with the million-dollar-a-month project being run out of an academic department coming to a head, but Vietnam War-era turmoil from the students was making it more difficult to peacefully continue the defense-funded project on campus. In the midst of all of these non-scientific issues, the project was first moved from within the academic department to become its own free-standing center at the university. Then, in January 1971, ARPA decided to move the system to the NASA Ames Research Center when it became feasible. The move was carried out by Burroughs in 1972 and may have happened earlier in the technological development process than DARPA would have preferred.

Hardware problems plagued the machine in its early years at Ames. Early circuit batches failed at high rates and in ways that were often difficult to detect. Additionally, many of the back-panel connections and terminating transistors functioned poorly. While successful runs were made as early as 1973, the team at Ames did not have the machine working in a way that could be considered reliable until 1975. However, even after the machine was completed, one problem continued to cause severe headaches: producing software to run on the machine. This would be an area of lasting difficulty that plagued the machine for its entire lifetime — which was admittedly not so long. The difficulty in writing software for the machine would be remembered when the next generation of IPTO leaders sought to push the area of parallel computing to the next level.

The ILLIAC IV would only be used until about 1981 when NASA replaced the machine with a more reliable and easy-to-use successor

ILLIAC IV’s Results

By the time ILLIAC IV was finally constructed, the total cost was $31 million (over $200 million today) — substantially higher than the original, expected price tag of $8 million. Beyond the cost problem, the final machine only had one quadrant of 64 processors rather than the originally planned four. Several of the many experimental components of the machine could not be made to work and existing components had to be used in their place. The resulting machine only achieved one-twentieth of the computing power that the researchers originally sought to achieve — the initial goal was one billion operations per second. Also, even though the ILLIAC IV was considered finished in 1972, much of the 1972 to 1975 period was spent in an intensive effort to correct problems and improve the reliability of the machine as researchers affiliated with the armed services were attempting to use the machine for practical applications. 

Notwithstanding the setbacks, once the ILLIAC IV became operational in 1975, it was thought to be the fastest machine in the world for certain types of problems — calculations of fluid flow being one example. In the late 1970s, the computer was used in service of many of the armed services and DARPA applications on which it was intended to be used. This includes the army’s multiple target problem, research in cryptanalysis, seismic array processing, economics, and more. For the few years in which the machine was in operation, it may have been the best computer in the world for solving problems whose calculations could be performed in an “all at once” manner and were, thus, ideally suited to being solved by a parallel processing computer. 

From the standpoint of developing the technology base, the component technology of the computer helped improve many pieces of the technology base. Some of the prominent examples:

• The first large-scale computer to employ emitter-coupled logic in its processors — taking the place of transistor-transistor logic. This technology would go on to be used in many high-speed computers in subsequent years. 

• Its circuit boards were the first to be designed with the aid of a computer. Computer design automation is now widely used in industry.

• The design of its storage technology — which consisted of 64 disks that could be read from and written to concurrently — led to higher speed input/output.

• The machine’s laser memory system as an additional, tertiary memory had a capacity in the trillion-bit range and read-in and read-out rates in the million bits per second range.

• ILLIAC IV’s architecture employed a single instruction stream to control the multiple data streams involved in interprocessor communication.

• ILLIAC IV was the first large system to employ a semiconductor primary store.

On top of all of the new functionality, the program also succeeded in (eventually) producing a machine built by a joint industry-university collaboration. Slotnick and the Illinois team did not believe they’d have been better off working on the project with no help from industry. In fact, Slotnick later reflected that he believed running such an operation with him and the university as primary contractor could have been a mistake — or something to learn from at the very least. He referred to trying to administratively manage such a technically complex operation with that sort of headcount, scale, budget, and operational needs as having “all the sense of trying to build  a battleship in a bathtub.”

ILLIAC IV’s Lessons Learned (and Caveats)

Joining all of these non-obvious developments into a project meant to produce a single, useful machine caused substantial time delays and cost escalation. But the field of computing got more than nothing in return for the tradeoff. The ambitious design of the machine created a technology “pull” and stimulated the development of new component technologies that had the potential to find broad use. 

For example, when the project ran into issues incorporating the originally planned film memories into the design, Slotnick turned to the nascent Fairchild Semiconductors to design the project’s all-important memories to fit with its new design. The new design was seen to be quite risky since it required advances in semiconductor art, engineering design, and production. But Slotnick was convinced the Fairchild team could do the work. This confidence may have come from the fact that Slotnick’s old mentor, Rex Rice, headed memory system operations at Fairchild. It also may have been due to Fairchild’s rapidly growing reputation as the new provider of semiconductors to the NASA program and certain Air Force programs. Regardless, in the end, Slotnick was right to put his faith in the Fairchild team because they successfully produced the required memory chips and the ILLIAC IV was the first to use chips of this design. Some believe that this project helped speed up the pace with which semiconductor memories became commercially available — although given the field was growing so quickly, it is hard to tell if such a claim is true. 

Taking a step back, it is reasonable to question whether or not the ILLIAC IV’s approach is the ideal strategy to orchestrate new technological developments. As always, the suitability of the approach depends on the situation and what the PM thinks is appropriate given the context. Developing so many new pieces of component technology to use, for the first time, in one machine seems to leave one obviously more prone to cost overruns and time delays than most projects. The upside is that the approach provides new funding and a focused direction to help bring what could become very useful component technology into existence. In the best case, the approach can push a field forward much faster than if one had taken a more measured approach. But the approach does also run the substantial risk of generating demand for and funding pieces of component technology that might be useful to the current project, but will not be useful to the field as a whole in later years. For example, if the components required by the design of the machine are not reflective of the needs of the field as it progressed outside of that individual project.

In essence, the  approach of the ILLIAC IV team should be considered a high-risk, high-reward strategy. It probably won’t achieve outcomes as stable as those of the gray coding approach covered in the following Connection Machine sections, but there are still situations where the full frontal assault approach of the ILLIAC IV has a high enough expected value to justify the risks. With this approach, one may run the risk of cost overruns, delays in producing a working machine, and developing components that the broader field turns out to not need. But the approach also might help a technological area make discontinuous leaps in component technology and overall system performance that would not happen by changing one component at a time. Particularly in the early days of a technological field, changing multiple design components at once is often an attractive approach given that existing approaches might be extremely far from what researchers know to be theoretically possible.3 (See this earlier FreakTakes piece on the field of early molecular biology to read more about this strategy.)

Depending on the political and technical context of the project, the full frontal assault approach might be just what is required. But, in other situations, it might be considered far too risky. Stephen Squires, with his computer architectures program in the more mature DARPA computing portfolio of the 1980s, considered the approach far too risky.

Connection Machine’s Beginnings

While the final ILLIAC IV machine did work, Squires took the management of the program as a cautionary tale of what not to do when developing machines within IPTO. In his eyes, the cost overruns and delays in producing the machine were entirely foreseeable given the structure of the project. Squires believed the “full frontal assault” approach of the ILLIAC IV project — with novel approaches to the machine’s architecture, processors, memory, storage, and other components — pushed too many untried technologies at once. Problems in developing or implementing any one of the components could negatively impact the entire project. In the uncertain enterprise that is research, Squires believed this was an untenable risk profile. 

Squires wanted to encourage projects funded through his architectures program to limit the number of new technologies a project worked on at a time — usually to one. He called the approach “gray coding.” Roland and Shiman — who wrote the most extensive history on DARPA’s Strategic Computing Initiative of the 1980s and extensively interviewed Squires — expanded on the thinking behind the approach as follows:

That way, problems could be easily traced, isolated, and corrected, and the chances of catastrophic failure of the system would be minimized if not eliminated. Advance of the field would not occur in one fell swoop by ‘frontal assault’ of some ‘point solution,’ but through the selected development and installation of key technologies, all well-timed and well-coordinated. Advance would occur over a series of generations. Once a technology or key concept had proven reliable, a new component or design would be implemented and tested.

Squires portion of DARPA would still pursue several technological trajectories at once — which Squires called the ensemble model —  to allow the research ecosystem to prove out which one held the most promise. 

Additionally, learning from the ILLIAC IV example and his own experience working on systems software at the NSA, Squires insisted that software development be emphasized as much as hardware work for the architectures being developed. In the case of the ILLIAC IV, the development largely focused on getting the hardware working and later shifted efforts towards generating useful applications via software development. Many believe the ILLIAC IV team emphasizing the hardware first and software later was a large part of what took the machines so long to finally get working at the back half of the project. Squires was going to do things differently.

Squires planned for the architectures program to progress in three phases — each lasting roughly three years. The projects funded in the first phase would generally seek to refine promising architecture ideas and develop microelectronics components that would later be used in building parallel machines. Projects funded in the second phase would generally seek to build full prototypes of the most promising architectures. The projects funded in the third phase would integrate technology that now (hopefully) existed into composite systems that could support DARPA–relevant applications — such as military applications in computer vision, natural language understanding, simulations, and more. Squires hoped that the machines being developed would achieve 100 million floating point operations per second by 1987, a billion by 1989, and a trillion by 1992. In service of this goal, all of the early work on smaller-scale prototypes had to be easily scalable — for example, being designed in such a way that they could be easily expanded by simply increasing the number or power of processors in the prototype. As all of this work progressed, work on new programming languages, compilers, tools, and concepts that better served parallel programming would be also funded.

With the high level plan in place, Squires began by traveling around and further familiarizing himself with the pool of potential performers that might be funded and meeting with those in industry — such as computer manufacturers. In addition, this familiarizing process also entailed a February 1984 call for qualified sources asking for potential contractors to explain what they could do in the areas of signal, symbolic, or multifunction processing if they were funded by Squires’ architectures program.

Squires, at the time, primarily issued the formal solicitation for informational purposes — as a way to see what was out there, not necessarily planning to immediately award any contracts to the proposals submitted. But one proposal jumped off the page from the pool of submissions — two-thirds of which from industry and one-third from academia. That proposal was a big black notebook sent in by a newly formed company called Thinking Machines Corporation (TMC) outlining their technical and business plans for a computer called the Connection Machine.

Connection Machine’s Operations

The proposed machine was exactly what Squires was looking for. The machine the company proposed was an ambitious, truly parallel hardware architecture that seemed like it could scale up by simply adding more or better processors. In spite of the ambitious design, Squires still saw the approach as conservative — in a positive way. Not only had the Thinking Machines Corporation (TMC) already developed a workable VLSI design for the processor chips — which was one notable source of technological risk already mitigated — but the company also set out to construct the architecture using as many commonly used components as possible. In fact, in several areas they went as far as to use components that were far simpler than the ones commonly used in other machines at the time. One example is that, for its first machine, TMC planned to use the simplest 1-bit processors instead of the more powerful 4, 8, or 16-bit processors commonly in use at the time. In addition, the Connection Machine would not even be a stand-alone machine in the early iterations. The front end used to access the machine would be a more commonly used computer — a LISP machine in this case. In this early stage, SC’s main goal was to demonstrate the feasibility of basic parallel processing concepts while not adding needless complications to projects that did not serve the main goal. TMC — maybe not surprisingly since its founder came from a lab that was quite used to working with DARPA PMs — clearly got the memo and submitted a proposal that perfectly complied with the gray coding approach while maintaining an ambitious vision.

After some meetings between DARPA staff and TMC executives, it seemed clear to DARPA that TMC as a venture was well thought out from both the business and technical perspectives. In early 1984, $3 million (~$9 million today) was approved to fund TMC’s work on a small prototype of the machine containing 16k processors — with the option for an additional $1.65 million to scale up the prototype to 64k processors. Raising an additional $16 million in private investments around this time from private investors, TMC was well-capitalized and ready to begin work on its first machine.

Even having laid out a plan of action that complied with Squires’ gray coding framework, the work was daunting for the company. Danny Hillis, the CEO of TMC who came up with the idea for the Connection Machine during his Ph.D. under Marvin Minksy at MIT, said the early work building the simplified version of the Connection Machine underway in 1984-1985 was “overwhelming.” He continued, describing the early work of the company in a 1989 Physics Today article:

We had to design our own silicon integrated circuits, with processors and a router. We also had to invent packaging and cooling mechanisms, write compilers and assemblers, devise ways of testing processors simultaneously, and so on. Even simple problems like wiring the boards together took on a whole new meaning when working with tens of thousands of processors. In retrospect, if we had had any understanding of how complicated the project was going to be, we never would have started.

This reflection from Hillis probably lends some credibility to Squires’ gray coding approach having been a good fit for this particular project. The design and need to develop new software approaches for making use of the novel design would also bring with it more difficulties. There was no need to compound on these difficulties by ensuring the initial prototype had the most cutting-edge component tech possible. Novel engineering development projects like this were sure to run into plenty of issues that required novel solutions to overcome. As Hillis stated, they ran into so many of these in the early days that they might not have even undertaken the project had they understood how painful it would be. As the project grew more complicated, Hillis seemed thankful to have Manhattan Project alum Richard Feynman around as an intern for some summers — Feynman’s son Carl had worked with Hillis when Hillis was a grad student and Carl was an undergrad at MIT. Feynman seemed relatively influential in pushing the company to take a page out of the Manhattan Project’s management playbook and designate a group lead in each area of technology — such as software, packaging, electronics, etc. Feynman — who was initially brought on to help brainstorm applications for the computer — also pushed the company to hold regular seminars of invited speakers, often scientists, who the company believed might have interesting use cases for the machine.

While the project did have its difficulties in the early years, in spite of Hillis’s statement about the project being “overwhelming,” the project was considered the gem of the architectures portfolio in the early years of SC. Squires’ program had also invested in other prototype-stage projects for at least a two year trial stage — which all had some positive developments as well as some downsides. Each of the general-purpose, prototype-stage projects was meant to help in the architectures program’s larger goals of identifying promising approaches to parallel computing and providing experience working with the technology. Simultaneously, smaller projects called “accelerators” were funded with the goal of doing development work to produce components that could later improve the performance of the general-purpose machines. Besides the Connection Machine, some of the architectures program’s other general-purpose prototype projects included:

• BBN’s Butterfly computer series

  • The Butterfly computer had been in the DARPA funding pipeline since 1977, with funding going to one of DARPA’s workhouse computing research contractors, BBN. At the start of the architectures program, a model of the Butterfly with 10 processors already existed. BBN received funding to continue to scale the prototype to 128 processors. The butterfly computer was a coarse-grained shared-memory machine — coarse-grained machines have fewer but more powerful processors capable of running entire programs in a moderately parallel but not truly parallel fashion. This technology was both further along in development than smooth-grained parallel computers — like the Connection Machine — were. Additionally, machines like the Butterfly could easily make use of traditional programming methodologies rather than developing new ones. For those reasons, the Butterfly was meant to serve as the benchmark against which progress on other machines in the portfolio would be measured. 

• CMU’s Warp

  • The architectures program also contracted CMU to build Warp. Warp was a systolic array machine that sought to achieve more efficient performance using an approach in which processors are connected sequentially and data flows from one processor to the next with each performing a different operation — analogous to an assembly line. CMU had received this contract based on its prior demonstration of the systolic approach having built a programmable systolic chip.

• Columbia’s DADO

  • DADO was designed as a coarse-grained parallel machine. DADO was one of two tree-structured machines — in which a central processing unit was connected to two others, and each of those to two others, etc. — started at Columbia University in 1981. DARPA discontinued funding for this project after the early prototype stage.

• Columbia’s Non-Von

  • The second of Columbia’s two tree-structured machines, the Non-Von, was somewhere in between the DADO and the Connection Machine in terms of how fine-grained it was. A machine like this one would likely prove simpler to program than a Connection Machine, but would also not achieve quite the level of parallelization. The Non-Von sought to prove useful in managing and manipulating large databases. DARPA also discontinued funding for this project after the early prototype stage.

• TI’s compact LISP machine

  • TI sought DARPA funding to fund its development of a miniature version of a LISP machine. To do this, TI planned to implement LISP processors onto individual chips. TI hoped that the entire machine would fit on nine small printed circuit cards and, in the end, would be small enough to be embedded in computers of military systems such as within a pilot’s cockpit to help in expert decision making. DARPA agreed to the $6 million in hardware costs and TI, sharing the costs, funded the $6 million in software development.

But none of these projects seemed to carry as much promise as the Connection Machine and its model of performing identical operations simultaneously across its many, many processors. However, each of the other machines did often have at least one piece of component technology in development that DARPA was intrigued to pursue further. For example, the Non-Von project developed a way of using multiple disk heads in operating its storage system. TMC later incorporated this idea into one of its later prototypes — all SC-funded developments were eligible to be used by other SC contractors. This is just one case of discontinued portfolio projects still finding help push the technology base forward in Squires’ portfolio.

As the Connection Machine project got underway, Hillis had arranged for DARPA funding to install a LISP machine at the chip foundry TMC worked with in California. With that machine installed, TMC could test chips at the foundry rather than having them shipped to Massachusetts to test them there. By mid-May 1985, a little over a year after TMC had received its first DARPA grant, the 16k processor prototype had been finished — more than a month ahead of schedule. After DARPA checked out the machine in the TMC offices, DARPA immediately invoked the option for the 64k processor machine. This scaled-up prototype of the machine, too, would be finished ahead of schedule — by the end of 1985. Hillis’ design seemed to be scaling the way he and DARPA had hoped. And for all of the sub-problems that the initial design could not account for, the very talented (and growing) TMC research team — largely drawn from the staff and students of MIT, CMU, Yale, Stanford, and other elite pools of computer engineering development talent — was proving more than up to the task of solving. 

By the spring of 1986, the first Connection Machine — the CM1 — was being manufactured and was available for sale. The machine’s chips each contained 16 customized 1-bit VLSI processors. Thirty-two of these chips would be printed on a circuit board and mounted next to each other — eventually making up four separate quadrants of 16k processors. Each quadrant could stand alone as a cheaper, scaled-down model of the 64k processor machine. In spite of the novelty of the truly parallel design of the architecture, each of the components remained quite safe and reliable to use — as the gray coding approach intended. The 1-bit processors were basic and slow compared to the state-of-the-art processors at the time, but they also produced so little heat that the machine did not require much cooling. Installation of the machine on a buyer’s site was easy and could be readily set up with a LISP machine on-site — which customers knew how to use. However, customers did have to learn new, Connection Machines-workable, versions of C and LISP to make full use of the machine’s capabilities while programming. The user did not need to fully understand how memory assignment and other aspects of the machines worked with these languages, but learning the languages was a necessity.

With the project going well, in the beginning of 1986 DARPA funded TMC for an additional $3.8 million over two years to find ways to exploit the CM1 and CM2 on various scientific and military applications. While the Connection Machines were still in the process of becoming truly useful machines, this particular generation of DARPA’s computing programs carried a strong political emphasis on pushing successful research projects into application mode as early as possible. This iteration of DARPA funding for TMC included covering work on projects to further improve software development practices for the machine, develop the machine so it could be used as a network’s server, increase storage capabilities to aid in work on data-intensive problems, and developing a training program to help others learn to use the machine. Beyond this $3.8 million, DARPA would also serve as TMC’s biggest buyer in the coming years. 

With this DARPA funding, the work continued on into more mature prototyping and scaling stages. As of 1987, around twelve CM1s had been purchased from TMC — about half of them purchased by DARPA for its contractor community. The machines were typically purchased for around $5 million a piece in 1987 (~$14 million in 2023). As the TMC team sought new applications for its machines, under Hillis’ leadership, they eagerly sought out projects with many in the research community. This included partnering with physicists, astronomers, geologists, biologists, and chemists to understand the details of their applications. Someone like Feynman worked in application areas at TMC such as problems in database searches, geophysical modeling, protein folding, analyzing images, and simulated evolution. Establishing partnerships with researchers proved, unsurprisingly, quite natural to the organization. It should be remembered that, during his graduate work, Hillis designed the machine to be something like an optimal tool for researchers studying artificial intelligence — as well as other complex research problems. People like himself, his former lab mates in Minsky’s lab, and other researchers from similar environments were the kinds of people who staffed the company as well as its initial, intended customers.

The company did not prove nearly as eager to establish profitable partnerships with industry as it did partnerships with researchers. Of course, not all of the problems the company explored applying the machine to were research related. For example, Feynman also worked on one application area related to reading insurance forms. However, this sort of application work did seem to be a less natural fit for Hillis and the company.  Hillis, after all, saw the Connection Machine as a machine for science. In practice, TMC very clearly preferred not to make compromises to the goal of pushing science forward in search of increased revenue. For now, DARPA was TMC’s guardian angel when it came to buying machines, helping broker sales with research consumers of the machines that DARPA didn’t buy itself, and funding the company’s R&D work.

The CM1 — which TMC largely relied on DARPA to facilitate purchases for — did not only have limited market potential outside of the DARPA universe because of the price tag of its machines, but also, simply, because of what they just couldn’t do. From a consumer standpoint, the first major deficiency was that the machines could not yet run the standard computer language of the scientific community: FORTRAN. The second major deficiency was even though the CM1 could perform ten times as many computations per second as many contemporary, commercial supercomputers — 2.5 billion operations per second vs. a few hundred million — the 2.5 billion operations it could do per second were not floating-point operations. And floating-point operations were vital to most computationally intensive research problems at the time. Nevertheless, DARPA likely felt comfortable facilitating these purchases because much of the SC program emphasized folding other SC-developed component technologies into projects sooner rather than later. At the beginning of SC, DARPA had also facilitated the purchases of the young BBN Butterfly computer to some of its contractors in order to ensure they had the most up-to-date technology available to them and their development work. So, TMC was not the only contractor in the architectures portfolio to have received this sort of favorable treatment from DARPA.

With the deficiencies of the CM1 in mind, TMC got to work incorporating the clearly needed functionality into its CM2. Surely this lack of functionality, at this stage in the development process, was not considered a failure by any means. The first successful prototype — the CM1 — lacking much needed functionality from the user’s perspective was likely just a symptom of the gray coding approach. The project was likely unfolding as Squires hoped successful architectures projects would — in successive generations. By this time, the architectures program, headlined by TMC, was clearly proving that parallel computing could be very workable. From a technology development standpoint, that was no small thing, even if the tech was not the most powerful or user friendly yet. 

The CM2 was brought to market in April 1986 and could run FORTRAN as well as do floating-point operations. This version of the machine was much more usable to the scientific community and the machine's very novel, massively parallel architecture still required its customers to learn to use special software and new programming techniques. Using these programming languages proved difficult for even the machine’s quite savvy researcher-customers and far more trouble than it was worth for almost all commercial customers. An additional factor that made the machine more trouble than it was worth for many corporate customers was that the machine's rate of breakdowns and subsequent downtime, while not an issue to academic work schedules, was far more than corporate customers were comfortable with.

Some in the research community found useful applications for the new machines in areas such as fluid flow dynamics and large database searches. Yet, in spite of these successes, the machines were not yet considered very usable to their potential customers. Years later, Dave Waltz — the head of TMC’s AI group — recounted that most of the CM2’s early customers were not using the machine correctly. In his eyes, TMC had built a machine that worked far more like a human brain than a sequential computer like the Cray — the most successful commercial supercomputer company at the time — but, he noted, that people actually knew how to write programs for a Cray! Many of the customers were running  their computations in such a way that they used the floating point processors in the CM2s in a standard way and largely ignored the machine’s novel, 64,000 single-bit processors. But, with the technology continually progressing, DARPA continued to help TMC facilitate sales. As one of TMC’s research directors later recounted,  "Our charter wasn't to look at a machine and figure out the commercial profit. Our charter was to build an interesting machine."

As the Cold War waned and the acute need for military supercomputing and near-term AI applications lessened, the Bush administration and the president’s main science advisor — nuclear physicist Allan Bromley — turned its eye towards helping solve “grand science challenges.4” These challenges included problems related to climate modeling, analyzing protein folding patterns, mapping the human genome, earthquake prediction, and learning more about the underpinnings of quantum mechanics. Bromley seemed to believe that these problems required enormous computing power rather than artificial intelligence. From the modern perspective, smart people could argue whether or not Bromley was right in that idea. But, surely, massive increases in computing power paved the way for specific solutions in those problem domains — even if novel AI methods were what pushed some of those things across the finish line. Regardless, that general ethos was what drove the government’s new High Performance Computing and Communications (HPCC) program. DARPA was the lead agency in this work and received an additional budget of several billion dollars through 1996 for the program. One of the primary goals of the new program was a computer that could achieve one trillion computations per second.

Naturally, DARPA turned to TMC and gave the company some of this funding to help the company expand on its work that was so successful thus far. DARPA granted the company a $12 million initial contract to produce a scaled-down machine that could theoretically hit the trillion computations per second benchmark by 1992. It was in this period when TMC was working on the CM5 — TMC CEO Sheryl Handler skipped “CM3” and “CM4” in the hopes that the naming curveball would make espionage harder for would-be infiltrators — that the music stopped for the young technology darling.

In this period, as TMC was hitting its objectives and in a great place from a technology development standpoint, they began to spend as if they were having equivalent levels of commercial success — which they were not. Company executive Sheryl Handler had been brought in to co-found the company to bring a level of business maturity to the operation, but that proved to be a mistake. Her corporate spending habits were, putting it charitably, a bit decadent. This habit showed itself early on in the company’s lifespan and continued to worsen as the 1980s wore on. As early as 1984, the very young TMC moved into the top two floors of the expensive Carter Ink building and began spending money on modern, luxurious tech company-like amenities. A couple of these expenses included shelling out for a custom design plan and a gourmet chef. In 1989, as TMC’s work on the CM5 was getting underway, Handler signed a whopping 10-year lease with the Carter Ink building which cost the company $6 million a year — the $37 per square foot for the lease was reportedly 4.5 times higher than Lotus Development Corp. was paying just down the road. Additionally, management rapidly increased headcount by around 40% around this time — resulting in a staff of around 400. The company’s spending was more in line with the company’s growing status in the computing world than its actual financial standing. As Stephen Wolfram described the company’s status at the time, the company was “the place that foreign trade delegations would come to visit to see where American business was at these days.” An IBM computer scientist put it a little differently when he described the company as having cornered the market “on sex appeal in high-performance computing.”

While TMC was in the process of developing machines that were on their way to doing things that no other computer could seemingly do at the time, they were not producing machines that the market wanted as measured in total dollars of actual market demand. With the launch of the government’s HPCC initiative, the company’s research emphasis shifted even more strongly into building the most powerful computer possible by 1992. As it did this, the company continued to put off making the developments it needed to make to find some level of product market fit. The company had produced a machine that cost millions, but was almost exclusively sourcing its customers from the research community. The number of customers and the budget per customer in this community was far more limited than it was in the corporate sphere.

As work on the CM2 and later CM5 progressed, it does seem that the company had at least some opportunity to slowly ramp up efforts in the commercial sector. For example, in the late 1980s TMC sold two Connection Machines to American Express as database mining for improved customer analytics was a growing trend at the time. There was some internal debate about whether or not to start a business supercomputing group at the company — which would be an obvious “yes” for most young and growing companies with shareholders — but nothing much came of it. The company, in practice, clearly preferred to serve the research vision that drove its founder and employees to pursue computing over maximizing shareholder value.

As the work got underway on the CM5, progress was not as smooth as it had been on the previous two generations of machines. The first CM5 would not be completed until October 1991 — and even then it was not quite as powerful as promised because there was a delay in producing chips that fit the machine’s design. However, throughout this 1988 to 1991 period, TMC was peaking from a revenue standpoint. In 1989, thanks to the company’s good reputation within DARPA’s walls, the company achieved its first profit — $700,000 profit on revenues of $45 million. Then, in 1990 the company saw profits of $1 million on revenues of $65 million. This would be the peak of TMC’s profits.

Sadly, for TMC and the world of early 1990s parallel computing, that was as good as things would ever get for the firm. The company’s introduction of an early version of the CM5 in late 1991 was lauded by Hillis as possessing the highest theoretical peak performance if you added enough processors to it — but that particular version of the machine was less powerful than the CM2. It’s very possible that with more DARPA funding and further work upgrading the prototype, this branch of the technology base would have continued to progress from within the expensive walls of the Carter Ink building, but that never came to be.

In August 1991, in the midst of the HPCC initiative’s leaders beginning to make plans for how to deploy most of its sizable budget moving forward, the Wall Street Journal released a piece diving into DARPA’s subsidies of at least two dozen Connection Machines in addition to the machines it had bought. While this in and of itself was not necessarily illegal or even nefarious, this particular political era was one in which anything that looked like “industrial policy” or government interference in markets was extremely scrutinized. DARPA buying its contractor pool BBN Butterfly machines and some machines that resulted from CMU’s Warp project does not seem to have received negative press in the same way. This disparity in PR outcomes might be because buying machines that might soon become obsolete due to technology in development within your portfolio did not come off as bad as subsidizing machines that were not yet considered to be truly finished, market-competitive products. 

Regardless of why the negative press came to be, the Bush administration made a point of ensuring that “industrial policy”-like help to TMC did not continue, and the company did not fare so well when it was abruptly thrust out into the competition of the commercial market. In 1992, TMC reported a $17 million loss. Massive layoffs and spending cuts began around this time. A new CEO was brought on who attempted to broker some kind of merger or partnership deal with Sun Microsystems and IBM, but TMC’s balance sheet was more toxic than its tech was impressive. To give the reader an idea: TMC owed $36 million in rent to the Carter Ink Building over the next six years for its ill-advised ten-year lease. And that was just one of the apparently many dark spots on its balance sheet. A deal like the one the new CEO sought could surely have been brokered around 1989 or 1990, when the company still seemed several years ahead of the field in the rapidly developing field of parallel computation and was beginning to show profits. However, the company had rejected offers for mergers and acquisitions at that time.

TMC filed for Chapter 11 bankruptcy in late 1993 and re-emerged from this as a small software company attempting to sell programs to run on their former competitors' parallel computers.

Connection Machine’s Results

Thinking Machines had been capable of hanging onto the dream of building a machine that helped raise all boats in the research community when it was fueled by DARPA funding. With DARPA funding, it was able to largely ignore the commercial market and commit itself to pushing the technology base forward with DARPA’s relatively rare form of risk-tolerant, farther-sighted capital. And they were doing a good job of meeting DARPA’s needs. However, TMC was spending its privately raised capital as if they were doing an equally good job courting commercial customers as they were hitting computing benchmarks. And that was absolutely not the case. Of the 125 machines the company sold in its entire history, only ten percent were to commercial customers. So, when the DARPA relationship took a turn, the company inevitably had to file for Chapter 11 bankruptcy after some rushed attempts at solvency.

The company surely succeeded in its primary goal: producing a machine that proved truly parallel machines could work. Also, the firm did this while developing its technology, step-by-step, in a way that fit with DARPA’s preferred approach to moving the architectures program’s technology base along at the time: gray coding. From a technological standpoint, for the first five or six years it seems the company did an effective job of moving technological progress forward — in the form of a machine with steadily increasing functionality — on a consistent timeline that DARPA was very happy about. From a commercial perspective, however, the company was clearly a failure. Hillis himself admitted as much. At the time of the company’s bankruptcy, Hillis was asked how he felt. He replied, “Sad, that the people who put so much in — the employees and the investors — won't necessarily be the ones who benefit. And sad that I haven't been able to build a technology success into a business success.” DARPA surely got more out of the program for what it spent than the company’s investors did. 

In retrospect, DARPA’s general thinking on the problem area seems to have proven quite prescient. A major application area like image processing was exactly what they believed building a truly parallel processing machine like the Connection Machine could do. In the seminar series that Feynman encouraged TMC to put on, they even had a speaker from a not-so-popular field called neural networks come and talk. This story is surely proof that the DARPA ecosystem was, in many ways, functioning quite well — even if no commercial success came from TMC. Hillis and individuals at DARPA were in contact, talking about securing money for the idea contained in his Ph.D. thesis, at least as early as 1983. DARPA’s awareness of his idea and the young company almost from the moment of inception — as well as its recognition of Hillis’ ability to lead such an operation from a technical perspective — should surely be seen as a feather in DARPA’s cap. This awareness, of course, was possibly facilitated by Hillis’ prior connection with Minsky’s DARPA-funded lab at the heavily DARPA-funded MIT. But these kinds of interpersonal networks at key universities that were very good at certain sub-areas of research — such as MIT and CMU in AI research — were what DARPA PMs made it their business to maintain. That sort of relationship management, first and foremost, will always be one of the primary pieces of the puzzle that determines whether or not ARPA-like organizations succeed or fail.

A final note on TMC’s results: it is very possible that even a change as small as Hillis partnering with a much more responsible, business side co-founder than Handler — who proved adept at fundraising, but deficient in many key areas that TMC required — could have been enough to shepherd the company to a longer lifespan. From a commercial strategy standpoint, an individual like this may have helped TMC find commercial uses that fit in with the company’s goals and culture. From a financial standpoint, a better CEO might have ensured that the company either did not raise money from misaligned capital pools or did not spend the money it did raise as poorly as it did. 

Connection Machine’s Lessons Learned (and Caveats)

One quote that encapsulates several of the PM-relevant lessons from the Thinking Machines Corporation’s work and death was said by Hillis as the company was filing for bankruptcy, “The real money is in handling Wal-Mart’s inventory rather than searching for the origins of the universe.” From the perspective of a pure venture capitalist, this quote might read as the complaint of an academic wishing that the financial markets rewarded what researchers found important. Of course, that is not the ideal perspective most venture capitalists would not want in their portfolio founders. But the quote can also be read as Hillis calling attention to what might be a very major market inefficiency. Research companies like Thinking Machines are great vehicles for building new, exploratory technology, but there is often a very salient tradeoff between a company like this moving the technology base forward and profit-maximizing for shareholders in the near term.

Let’s not forget, the ILLIAC IV project did not only run into difficulties because its team was attempting to build a machine with too many novel components at once. Slotnick’s “battleship in a bathtub” remark was explicitly describing how ill-suited modern universities had become to building big, new machines. Hillis did not make the identical mistake of trying to build the Connection Machine from within a university, but he still did make a mistake in choosing his organizational setup. TMC ran into the conundrum of raising money from private investors — who would naturally prefer the company do things like build a computer to handle Wal-Mart’s inventory — as well as a funder (and initial customer) in DARPA that primarily cared about building a computer to move technology forward. In most cases, a company is going to make one of its two (rather different) funders unhappy in a situation where their incentives are only partially aligned.

Had Thinking Machines only pursued DARPA funding and spent more frugally — as an entity like ISI, which ran DARPA’s MOSIS program, would have done — it is very possible the company would have continued to progress as the darling of DARPA’s architectures program for the foreseeable future. If the company would have taken this approach, it could still have raised money from private markets at some point, just later on in its existence when it had a product that was closer to market-ready and had a more natural customer. Additionally, the company could have attempted to find a variety of customers so it would not have all its eggs in the DARPA basket the way BBN (covered at length in a coming piece) was known to have done in the 1960s and 1970s. BBN, in its early existence, did not seem to be purely profit-maximizing. Rather, it sought a variety of contracts — from DARPA, private philanthropies, engineering industry, and more — that its researchers found new and useful, but that still paid enough to facilitate a rather profitable operation.

Of course, none of this alternative history brings any assurance of success either. The political winds may have blown against TMC one way or another regardless. It should be noted, though, that Hillis’ Wal-Mart quote was predictive in some sense. The company that did make possibly the biggest parallel computing breakthrough in the late 1990s — NVIDIA with its new GPU — found a way to fund its breakthrough research by servicing a market that required high-tech solutions and was surprisingly large given how unimportant it was in the grand scheme of things: gaming graphics. 

It is also important to note that, in the first five or six years of TMC’s existence, it seems that the gray coding approach was working just as Squires had hoped. TMC was often putting out versions of Connection Machines that did not have the functionality or usability that many users would have wanted. But that could be considered a marketing issue as much as anything else. In bringing new functionality to the machines in this steady way, this gradual decrease in user dissatisfaction seems par for the course. The rate of progress of the Connection Machine seems, generally, far preferable to the messy process that was the ILLIAC IV development. Of course, individual PMs will know best when either approach is most suitable to a given project or portfolio. A variety of factors can impact the decision and there is no one-size-fits-all solution. One factor that could impact a decision like this, for example, is the higher the probability of failures or surprises in developing components in a given project, the more gray coding might be the clearly responsible approach. However, there are obviously also cases in which multiple components need to be changed between iterations of a machine for anything truly new to come of the process. When to apply either approach is, as always, best left up to a smart PM.

While the legacy of the Connection Machine is a complicated one, it was still the gem in an architectures portfolio that accomplished what it sought to do. As Roland and Shiman put it:

The first stage of the SC architectures program proved unquestionably that parallelism was viable and did have promise in solving many real-world problems…According to Professor John Hennessy of Stanford University, a respected authority on microelectronics and computer architectures, computer designers before SC, in their quest for greater speed and performance, kept running into the wall posed by the fundamental physical limits to the improvement of conventional systems and components. Each architect would try, and each would join the pile at the base of the wall. It was Squires, he said, who showed the way, and, through parallelism, led designers over the wall.

Hopefully the collection of history and lessons in this post prove of some use to PMs thinking through decisions like those that arose in the ILLIAC IV and CM1-CM5 projects.

Share

Subscribe now

Stay tuned for next week’s pieces! They explore DARPA’s ~1980 work in speech and computer vision applications, expert systems, and its programs for moving the field of chip research forward.

• Coming Sunday

  • MOSIS: The 1980s DARPA 'Silicon Broker'

• Coming Monday/Tuesday

  • The Autonomous Land Vehicle, Pilot's Associate, and Battle Management: Three North Star Application Projects from DARPA's Strategic Computing Portfolio

Specific Links:

ILLIAC IV

• Slotnick’s The Conception and Development of Parallel Processors: A Personal Memoir

  • An extremely useful source. Short, but quite thorough. Dives further into Slotnick’s journey from grad student to eventually running the ILLIAC IV project and his perspectives and thoughts on the events as they happened. 

• Chapter 17 of Van Atta’s DARPA Technical Accomplishments: An Historical Review of Selected DARPA Projects, Volume 1

  • A 1990, third party perspective on the planning, operations, and lessons learned from the ILLIAC IV project

• Chapter 5, pages 160-164 of Strategic Computing: How DARPA Built the Computer Age 

  • Particularly emphasizes the DARPA internal memory and different interpretations of the ILLIAC IV projects at DARPA during the 1980s.

• Chapter 6, pages 264-268 of Transforming Computer Technology: Information Processing for the Pentagon, 1962-1986

  • Particularly emphasizes how IPTO first got into parallel computing and the plans and early project planning of the ILLIAC IV project from a high level

• Chapter 2 of Transforming Computer Technology: Information Processing for the Pentagon, 1962-1986

  • Particularly dives into much of the early administrative history of IPTO — including things like budgets, norms for discussing and sourcing projects, etc.

• Slotnick’s Scientific American article, The Fastest Computer

  • A 1971 article outlining the project’s goals and achievements for a public audience. 

  • The article also contains the performance metrics and specs of various aspects of the machine as of 1971. 

  • The article also spends several pages walking through an example problem from mathematical physics, walking the reader through how the ILLIAC IV — or any parallel processor for that matter — would perform its operations and solve the problem faster than a traditional computer at the time. 

Connection Machine

• Chapter 5, pages 158-184 of Strategic Computing: How DARPA Built the Computer Age 

  • Provides more detail on the history of Squires’ approach to the architectures program and other projects that made up the portfolio. Some details about how the projects were run can also be found there. 

• The Rise and Fall of Thinking Machines

  • A thorough article diving into the lifespan of the company from a business and cultural perspective. The piece covers Thinking Machine’s corporate decision-making and relationships from its early years until its bankruptcy.

• Richard Feynman and the Connection Machine

  • This Physics Today article, written by Hillis himself, not only discusses Feynman and his work with the company, but also paints a picture of what went on in the early days of the company when it was a bunch of technical researchers working out of an old mansion outside of Boston.

• Thinking Machines Rather Than Markets

  • This Washington Post article, written in the midst of TMC’s demise, provides additional facts about the situation, but will be primarily useful to readers of this piece in helping them understand how reasonable-seeming help provided by an actor like DARPA can be portrayed depending on the given political climate.

General Links:

• Strategic Computing: How DARPA Built the Computer Age

• Transforming Computer Technology: Information Processing for the Pentagon, 1962-1986

Luckily, it turns out those who run research orgs and fund research believe administrative histories like these help them do their jobs better. Some in the community have even reached out to me asking if I could expand my work to cover a specific sub-area of particular importance: ARPA history.

In the coming months, I plan to do jus that. I will be writing administrative histories on specific DARPA/ARPA projects throughout history just as I have been doing with industrial R&D labs. Each individual write-up will be a bit shorter and there will be a bit less storytelling, but there will be many more of them to make us of. The hope is that these will prove useful to PMs at organizations similar to DARPA as well as the broader science and engineering research community. The work is being done with the help of former ARPA PMs, making full use of those who have written books containing details on specific ARPA projects, and in my role as a Fellow with the Good Science Project.

Please subscribe to follow along with the work! And, of course, I would be happy to talk through any of the write-ups with practitioners who have questions. Later this month, I’ll be releasing the first batch of pieces covering the history of 8-10 DARPA projects in about 40-50 pages of writing. This first batch will largely cover computing projects from the late 1970s into the early 1990s. The following batch will largely cover projects from DARPA’s earlier decades of computing projects. As the project wears on, I hope to cover projects from all eras and general subject areas of DARPA history.

This project will take a deep look at projects that are considered unequivocal successes as well as those that are remembered more for their lessons learned than scientific outputs. The goal of the project is, first and foremost, to produce materials that help those who manage research funding portfolios and labs to do their jobs even better. The tone of the administrative histories on this Substack has always been a positive one and I intend to keep it that way even when writing about projects that did not work out.

Please subscribe to follow along! If you know any great ARPA alumni who I should talk with or projects from history that I should look into, please ping me and let me know! Pointing me to any useful reading materials is always appreciated as well. My Twitter is right here and my email is egillia3 | at | alumni | dot | stanford | dot | edu.

Please subscribe and follow along! Thanks for reading.

Subscribe now

Share