Why Does DARPA Work? | Hacker News Reader

Why Does DARPA Work?

benjaminreinhardt.com

347 pointsMKais90 comments

A big part of the "DARPA formula" that doesn't seem to get much treatment here is consistency. DARPA is going to spend a few billion this year, and it spent a few billion last year, and it spent a few billion twenty years ago, and you can feel confident they'll spend a few billion next year. That means that if an academic researcher or similar can position their lab in an area that DARPA likes, they have a good shot at reliable funding for the length of a career. With all the economic uncertainties around research, this is tremendously appealing and will cause a lot of smart people to re-orient their entire research programs around DARPA-friendly subjects [0]. A funding organization that might disappear in a few years, by contrast, will get plenty of grant applications, but won't attract nearly the same level of researcher devotion.

(As a side note, I want to pull out a quote I thought was really nice, hidden a ways in:

"DARPA funds wacky things that go nowhere. DARPA programs have a 5—10% success rate and have included things like jetpacks, earthworm robots, creating fusion with sound waves, spider-man wall climbing, and bomb detecting bees. You can’t cut off just one tail of a distribution.")

[0] Which may, incidentally, go a ways towards explaining why DARPA heads describe their projects as "idea-limited"

creating fusion with sound waves (or shock waves, which is essentially what sound becomes at high enough energy) is one of the crazier ideas I've heard..

It's literally one of the ways they make nuclear bombs so it isn't that crazy.

Well yeah, if your shock waves for fusion are created by nuclear fission chain reaction that technology has been quite ready and mature since the 1960s.

Sonofusion [0] was a neat idea, and some of the relevant experiments (at least the cavitation part) can be done on a bench top with a hobbyist level budget. Last I checked, the research had been somewhat tarnished by academic fraud. The physics of jet formation during bubble collapse near a solid boundary remains fascinating [1].

[0] https://en.wikipedia.org/wiki/Bubble_fusion

[1] https://authors.library.caltech.edu/48933/1/On%20the%20Mecha...

My impression is that it's not at all that consistent. The total spending is constant at a few billion per year, but what that gets spent on depends entirely on the taste of the program managers, which in turn depends on the darpa director. The program managers only serve for a few years at a time, and the director for a bit longer, so it can change drastically. I think the trends are typically last a decade: in the 1980s after the success with the F-117, they were funding all kinds of airplane projects. Also in the 1980s they funded a lot of AI research, until they suddently didn't (the "AI winter"). In the 1990s, they were into networked combat simulators and command & control systems.

Starting around 2010, DARPA suddenly began to fund a lot of computer science research on formal verification. I was a phd student at the time, and it was pretty striking, programming language theory used to be a really fringe subject, and now suddenly it was awash in cash and all the professors could hire huge herds of postdocs. But some of those professors also told me that I should not expect it to last: sooner or later there will be new people in DARPA and all the money will instantly stop.

Certainly it's up to the vagaries of fate and such, but when I was in school I worked in a lab that for several years got fairly reliable DARPA funding for NLP work, and while they never built the department around it or anything it was nice to have that consistent year over year assumption that one or two projects would be interesting to the DARPA folks.

For them (us?) it was more a matter of "well this doesn't necessarily merit or reach the requirements for quote-unquote 'real' research, but it's really interesting and you know some investigation couldn't hurt". Those got pitched to DARPA, they asked "can you do it with Arabic and Chinese too?" and we said "no" and they funded us anyway a few times out of ten. Not bad, really.

My favorite project was the MIT silicon wafer etched jet engine. Like full jet engines the size of a pencil eraser. They also made solid rocket engines etched out of silicon.

Super interesting stuff.

That sounds cool. I found this article[0] that I think is about the research you're describing. It must work a little differently than a conventional jet engine since I don't see how they could have bearings on the shaft for something that small. Any idea how they solved that problem?

[0]: https://news.mit.edu/2006/microengines

I do think there is something to this. Not just that they spend allot of money consistently but consistently stick with a project to see it to completion. In organizations that I have seen lack of progress they jump from one fire to the next. This means that they work on A for a few months and then suddenly drop it to work on B because B is higher priority. Only to later realize that A was more important and switch back. DARPA having a large budget also allows them to see through projects to the end.

That hasn't been my experience at all!

DARPA contracts are structured with a lot of milestones and go/no-go decisions. We had teleconferences with the program manager and staff every two weeks and it was very clear that the money could abruptly stop if they ever felt that we were not going to deliver. This wasn't an idle threat, either: The program-wide PI meetings definitely got smaller and smaller as time passed. As a result, there was a lot of...reprioritization to make sure that the PM stayed happy.

This is very much unlike most other funding agencies. Once an NIH or NSF grant starts, you can be confident that the money will be there for the duration.

IMO this analysis gets caught up looking for answers in process when the real problem that DARPA solves is a political-economic one.

ATT didn't invent the internet not for lack of process, they were extremely innovative. They held back on innovations because they were making huge sums of money overcharging long distance mainframe-mainframe data link rates in the 50's-70's. DARPA succeeded as a trust buster.

They have the resources and legality to make whatever they want, regardless of patents (because National Security/patent law). They can make technology a reality, then develop political support with working prototypes. They build things to show how the Government is getting screwed over by some giant defense corp, because of a lack of competition in certain types of contracts.

I think they are becoming less relevant because the corporate-political landscape is becoming more trust-based.

Author here. Good point about DARPA serving a political-economic role. It's a different lens on the coordination role.

Is the corporate-political landscape becoming more trust-based than the Military-Industrial Complex that Eisenhower warned about?

I don't understand the comparison in reference to the question of 'Why Does DARPA Work?' or my comment.

Military-Industrial Complex is where defense industry profit motives drive unwarranted political influence that shapes defense policy, foreign policy. Military-Industrial Complex drives defense spending up.

Trust is about the erosion of market competition, which results in decrease product value-price ratio. In the context of defense product, it means that the products that the consumer (here military/govt) can buy from the market become increasingly expensive and weak. Which creates an opportunity for consumers to do-it-better-yourself-and-for-less, which to me, explains DARPA.

To a certain extent the MIC as you define it would explain the erosion of the market wouldn't it?

Also I've read your comments a few times now and I'm not sure if I'm completely understanding your view of what is becoming more "trust-based". Are you saying that DARPA is more trusted by the consumer (the federal government) than the market entities (the contractors)? If so, isn't that exactly what government entities of all types are created for? These agencies exist as part of the government working for the government because it's difficult to have an effective private market created for these similar purposes.

I think it makes the question about "what makes DARPA successful" so much more interesting from if we acknowledge the limitations of the sector. Many agencies, entities, or initiatives of various types have failed when given similar mandates and similar market conditions. That's even if you believe DARPA is truly as effective as is claimed at achieving it's mandate of innovation in the Defense and National Security sector...

Back in the early days of the Internet, I met some of the Bell Labs people working in that area. What they didn't like about an IP-based network was the jitter. That was totally unacceptable for voice. They wanted something with reliable clocking, and had come up with Datakit. That sends all packets for a given call over the same path, in order. You don't open a virtual circuit unless all the nodes have enough bandwidth for it. Telcos still use Datakit, and Asynchronous Transfer Mode is a successor to it.

I never dreamed that people would accept the degradation of telephony to 1 second of delay jitter with random dropouts and echoes.

Acceptance of the degradation couldn't be done in a day. It required societal acclimatization.

Other networking protocols coming from a telco background, in particular ATM and ISDN, were all circuit switched, and had suitable resource reservation for QoS. Acceptance of telephony degradation was probably driven by cost: VoIP was free and that made a difference, especially for international calls. In my experience, in 2020 the VoIP calls I make are really high quality, even better than 1980s-style ISDN calls, and the main cause of audio quality degradation are people using "hands-free" setups with their laptops.

I remember when cross-oceanic long distance calls required both parties to shout as loud as they could into the phone -- and often still not be heard well enough to make out what they were saying.

Modern IP telephony often has very high quality voice reproduction (my wi-fi to wi-fi Fi calls sound fantastic), in exchange for some timing issues. Echoes usually get solved in software (usually), and dropouts seem to be the main complaint.

In exchange, my wife can call her family in South Korea for approximately nothing, using the same data backbone as we use to watch movies and read web pages.

You can have ip over time multiplexing data link layer, like the IP over ATM. I believe they were still used in the core voice networking.

IP won because they are more flexible, and open. And the Internet cement the win because of that.

That's way planning ahead too far works less and less successful for bigger and bigger project. Too much dynamic is embed in the long wiring process. It becomes impossible to have a plan work out correctly.

1 second of delay jitter with random dropouts and echoes are typically caused by problems at the ends (poor wifi/cellular signal, too much load on cpu during a high res video call, laptop overheating, etc). UDP protocol allows to make a call even in the presence of such issues, whereas circuit switching with reliable clocking would simply not work at all.

These days, I suspect there's a lot of ISP last mile issues with everyone working at home.

You raise an interesting point that the author shies away from addressing.

Can you elaborate on the phrase "becoming more trust-based" ? Do you mean that decision makers in this landscape rely more on the strength of their connections than their individual understanding?

That's a good point. University labs are more nimble than a military contractor. The PI at a university directly does the work with their students (in some big labs, it's arguable that the PI manages the students that directly do the work). Whereas the contact at a military contractor is an accounts person, with lawyers and managers between them and the people doing the work.

Cost-wise, universities usually charge an indirect cost rate of 50-65% (which many people improperly think it means is the percent of the grant money taken by the university), whereas the military contractor charges an indirect cost rate of 120-200%.

So basically, about 70 cents of every dollar spent at a university directly goes towards the work (and trains students as a side effect; and as a triple bonus the cost paid towards student's tuition has no indirects on it at all), while only 30 cents of the dollar spent at a military contractor goes towards the work (and probably much less towards the people actually doing the work).

On the other hand, is one "unit" of student effort equivalent to one unit of staff scientist work?

I would say no: early-PhD students in particular are still learning and have other responsibilities (coursework, quals, etc) to boot. You can also interpret your numbers as suggesting that an experienced person is 2-3x more productive than a trainee, which seems....quite reasonable, actually.

I think that's the other "secret": the combination of large budgets and tight timelines encourage the use of more experienced staff and smarter division of labor: hire a programmer for this, contract out to a machinist for that. This is very different from the model used on most NIH grants, where grad students are used as jack-of-all-trades.

With all due respect to the incredible amount of intelligence, knowledge, and insight in display on HN, your comment, with it's culminating sentence, may be the most insightful thing I've read on HN.

I don't care if this comment doesn't meet HN guidelines. I needed to celebrate your comment.

How are they becoming less relevant?

The DARPA grand challenges have bootstrapped:

1. Self driving cars and got them working as a prototype. [1] 2. A rapid Orbital Launch Program (Still in progress after only one competitor left)[2] 3. A subterranean challenge to map and search underground environments [1] 4. A way to use social networking to find something that is time critical [2] 5. Onion routing which is used in Tor. [3] 6. Cyber Grand Challenge to automate the exploitation and securing network systems [4]

Note that competitors also have open sourced what they have done. [1] https://en.wikipedia.org/wiki/DARPA_Grand_Challenge

[2] https://en.wikipedia.org/wiki/DARPA_Network_Challenge

[3] https://en.wikipedia.org/wiki/Onion_routing https://en.wikipedia.org/wiki/DARPA_Grand_Challenge_(2007) [4] https://www.darpa.mil/program/cyber-grand-challenge

How are they less relevant? Well it's a bit left-field but I believe part of the reason why the political climate has become more pro-trust within the last twenty years is that international tech dominance is a national security boon. DARPA is not going to be asked to create ground-breaking tech in search, tracking, social media, tech generally, because the tech giants are already providing more than defense strategists could possibly have hoped for, on the world stage.

So instead we have these contests, the insights of which are handed out to existing companies to help them make incremental improvements. But the Onion browser isn't going to be rolled out to secure political autonomy throughout the world, or small business competitiveness because the data tracking is too valuable for surveillance programs. The national security/intel value of big tech companies has given those companies so much good will that govt officials/politicians have been unwilling to consider innovations that would undermine them.

I’m not arguing applications I’m arguing innovation. They do the initial research and then companies monetize .

> The DARPA grand challenges have bootstrapped:

I have a contrarian opinion about the impact of DARPA and related programs (ARPA-e especially). To some extent, there is a confusion of cause and effect. DARPA is good at identifying possible nascent technologies and funding them. They then have a flag planted, and can claim "we helped invent X", which is hard to disprove. But was it because of DARPA involvement, or was DARPA merely there at the right time? They have definitely funded plenty of projects that go nowhere.

ARPA-e makes this strategy explicit. They'll give money to an up-and-coming startup between rounds and then claim all later investment as "follow-on funding". They then report "follow-on funding" as a success metric to congress, which seems easy to game. ARPA-e has been around for over 10 years and I don't think they can point to anything coming out of the program as a legitimate game changer.

I do think the applied research programs do some good and has some impact, mostly as a way to convene the broader technology community. But for the most part I would rather have the government focus on funding basic research instead of applied research.

There is a feedback loop here. They help fund existing ideas, get more people/companies involved, raise awareness on certain problems, and motivate more participants to do relevant research.

So do they "invent"? ...well that's a semantic argument. The point is they absolutely help make things happen.

My question is... do they really? They happen to be around when tech moves, but would it have moved without them? It's a historical counterfactual, and hard to argue about. I'm skeptical and I'm more skeptical of ARPA-e.

What's the difference between "helping invent X" and "identifying possible nascent technologies and funding them"?

Nascent technologies aren't a fait accompli. Without funding, something expensive like a launch system isn't going to get anywhere no matter how clever an idea someone has on paper.

At least in the self-driving car case, it's hard to say they didn't have a big impact. For universities to allow, fund, and support students to form engineering student teams, there usually needs to be some kind of official competition that the team can compete in. In this case, those university teams made major progress on what was at the time seen as an intractable problem. Google and its ilk then headhunted all of the students in the top performing teams to launch their self-driving car division, and now here we are.

DARPA is effective, but this document paints a rosy picture that is far from the truth. I've been a part of DARPA projects both commerical, university and on the government side.

The real process is:

1. PM gets picked due to knowing someone or being a former employee.

2. The biggest test of a program isn't if it's doable or a good idea, but if its able to be transitioned to another government agency with deeper pockets.

3. Most contracts are lost before you begin writing, as people have insider information about what the PM wants. This is done through just talking with each other (remember that most of the PMs come from the same companies), and not through any other formal process.

4. DARPA has some really cool stuff, but fails to transition it well enough (leading to 2.)

DARPA is not without it's problems, but has a better track record then NSF (NIH has them beat). What is funny is that you quickly realize how much bunk there is in scientific research and how many papers are not replicated.

this aligns with my experience. NIH has a better track record than DARPA, which has a better track record than NSF and the worst is the DOE. The DOE knows this and is trying to cargo-cult DARPA through stuff like ARPA-E. Predictable results have ensued.

What does a better track record mean? More papers? Products? Social impact? Why is DOE the worst?

Ummm.. The DOE run National labs have pioneered fundamental science. A lot of modern materials science technique development originally happened at some DOE National Lab.

Judging basic science by productivity is the absolute worst metric to choose. DOE Labs are going down that path, and are becoming glorified university research groups all in the name of productivity.

What is a better track record? I don't know what that means? More papers? More consumer products? More social impact?

Projects achieving their initial goals. Government keeps metrics on all grants, projects and contracts. The reason NSF is low is because they rely on universities who have poor quality control standards as they don't pay their researchers enough.

A bill, the Endless Frontiers Act, has been introduced in both houses of Congress to make a section of the National Science Foundation that works more like DARPA, while massively increasing its funding. The idea is to extend the DARPA model to many more non-military areas, just as suggested in this article.

https://www.sciencemag.org/news/2020/05/us-lawmakers-unveil-...

Certainly a timely article. I don't know whether it's a good thing or not. The DARPA model has certainly been productive, but it isn't suited for every research topic or subdiscipline.

As discussed in the article, currently NSF is the most open to basic science of funding agencies, and gives grantees the most latitude in what they work on. That is a valuable thing to have in the ecosystem.

If Congress expects the majority of publicly-funded research even from the NSF to be on short-term grants for specific visions and technologies, it will rule out working on a lot of important things.

There is also already ARPA-E, which is a similar mandate to DARPA (i.e., risky projects with high payoff) for energy technology. How effective it has been thus far is a matter of debate due to a variety of factors, but I like the basic idea.

https://en.wikipedia.org/wiki/ARPA-E

Author here. I actually wrote a discussion of the Endless Frontier Act: https://benjaminreinhardt.com/the-endless-frontier-act/

I agree that the ARPA model is not suitable for all (or even the majority of research) and the act doesn't inspire confidence that shifting the NSF towards a DARPA-like model would do well.

Just wanted to say, thanks for writing the article about DARPA, and this analysis of the endless frontiers act. From reading both articles, obviously you're very pro-DARPA and anti-NSF.

I have two thoughts that might erode your thesis a bit, that 1) I think you're overestimating the amount of time that faculty spend on writing grants. We like to complain about it, but I've tracked my time to the minute over the past 7 years, and grant writing (both the proposals and the reports) is about 2% of my work time. 2) You might be misunderstanding indirect costs like many people, where you say "Universities can take more than half of grant money as administrative overhead". Indirect cost math is funny, but in order for administrative overhead to be more than half of the grant, the indirect cost rate would have to be over 100%. I can explain more if you're interested.

Obviously we're both biased by our job, but it's still useful to read your perspective.

Appreciate the correction about indirect cost math - I’ve applied for several grants but never did lab accounting so I interpreted it incorrectly when professors told me “50% overhead.” I’ll fix that when I’m at a computer.

I would characterize my position as less “pro DARPA and anti NSF” and more “I think on the margin the world needs more DARPA-like activity more than it needs more NSF-like activity”

Small anecdote, but good experience with DARPA and SBIR both. One thing I liked - they seemed pretty low overhead / low hassle and outcome / objective focused.

That contrasts VERY strong with most govt contracting which is under allowable cost or cost reimbursement setups where the absolute most important criteria is to bill enough costs to draw contract in the right cost buckets (which can be super annoying if local agencies require super complicated budget mod processes).

In the cost based contracts, the focus really focuses on the accounting for the costs and other compliance related items. Ie, did you send someone to a conference with govt money, how can you prove you didn't use the govt money in this way or that way etc etc. Bam, welcome to personal activity reports with fund codes that no one understands (ie, major universities have an insane number of codes), and all the nightmares that follow including a fair bit of rule pending that even normally ethical folks find themselves being asked to to get through the paperwork.

Seriously, you deliver the product at 50% of cost? You will get a nasty note from the head of your agency saying make sure you draw full contract because agency budget depends on the indirect portion of this award (ie, 30% to overhead) and even the govt agency supervising (who also budgeted based on a cut of full contract) AND other folks for whom leftover money makes life difficult (harder to close contract etc) AND because a lot of govt funding is on the repeat what we did last year model so drawing down everything avoids a cut next year when you may really need it.

You determine it would be cheaper to do x vs y and that required a budget mod? Wait 1-2 months for commission approval if you even bother trying to fight it through (changes < 10% often ignored thankfully).

I don't know how the accounting for SBIRs etc work, but somehow those projects always seemed more results oriented (so a LOT more fun to work on, focus is on getting a solution going).

I used to work for a company doing SBIR work. We finished a contract underbudget returned the surplus money and got audited because of it.

I don't know all of the accounting details, but SBIR has some leeway (because its supposed to be research) but it's not perfect. However, you are allowed to use surplus money to fund development of things which are related to what you actually put in the proposal/contract. So if you have a $100k contract, and meeting the requirements only required on the $75k, you should and do spend the rest of the $25k on extra features. But you have to spend all the money.

I think the real thing about SBIR money, is that three people get phase 1 money and only one or two will get phase 2 money so you really have to deliver in order to get selected. Then if you get the phase 2, there basically isn't phase 3 money, so you better get to a product that is ready to be sold. Finally if can't show on paper a return on investment (e.g. nonSBIR funding or revenue from sales/acquisition) they cut you off from the SBIR spigot. So the incentives align with getting work done.

I worked at company in SBIR phase 2 and one quirk I found strange was that the money could be used for R&D not things deemed “production”. Endless 3D prints were ok, random software features were ok, tooling for injection molding was not. It wasn’t a particularly onerous requirement, but I think reinforced the tendencies of our former academic founders to spend a lot of time on research and none on development.

Part of it is figuring out how to classify "production" as R&D. Well we need to test a prototype and in order to that we need it made as the same was as production due to structure property process relationships.

Or manufacturing defects is going to be a huge source of bad thing so we need to do some research into the manufacturing process.

The irony is palpable:

> I would rather this be read by a few people motivated to take action than by a broad audience who will find it merely interesting. In that vein, if you find yourself wanting to share this on Twitter or Hacker News, consider instead sharing it with one or two friends who will take action on it. Thank you for indulging me!

I'm glad of course it was shared here. As a distillation - I think the author's theme lies with enabling more _researchers_ rather than business people to take moonshots and do foundational knowledge building and discovery that redefines a field and then focus on commercialization from a birds eye view by technically capable visionaries.

This is the SBIR [1] model (also a US Gov requirement to fund small business research for any federal agency with >$100M in funding), the Bell Labs model (which yeilded amazing foundational work like UNIX and the transistor and was a direct result of AT&T's monopoly and excess resources), and perhaps even the YC model (though that one is obviously focused on a shorter horizon and more on commercializing existing tech and more rarely on foundational research).

I've personally thought about this problem a lot and done this at a small scale and would love to expand upon it. https://augmentedlabs.org Would love to hear others experiences and thoughts.

[1] https://en.m.wikipedia.org/wiki/Small_Business_Innovation_Re...

Does DARPA work(anymore)? The first example of how it works was it's "badass program managers", but all of the program managers I've met have been, to put it lightly, idiots. One of the worst was the PM in charge of the entire subject research division. It turned out, they had a graduate thesis that was entirely observing an instrument artifact (showing bad jugement). Not too long after I interacted with them, there was a minor scandal in the community because postdoc who struggled with reproducing the effect and complained about the results was abused and railroaded by the PI in charge of the project. Shortly thereafter the PM didn't seem to be a PM anymore.

...so your best anecdote about why DARPA doesn't work is that they fire incompetent PMs?

Yes, after being promoted to the head of the division with several people speculating that they were being groomed for even higher posts. And it took a scandal to get rid of them. Person should never had been a PM in the first place.

You never hear people say "it's proof that Congress works because they ejected that pedophile (sending lewd messages to the pages) from their seat".

Hate to break it to you, but "an incompetent person rising through the ranks quickly before crashing and burning" is something that happens in pretty literally every large org.

Any 60 year old organization with a large budget is going to have multiple instances of this happening.

Perhaps I have a flair for the dramatic, but consider my statement that I haven't met a competent pm in the division I was in (n~5). The incentives are not that well-aligned. Think about it this way. If you're really smart, why do you become a DARPA PM, and not a PI?

Yes there are a few good reasons, but the population it applies to have largely been selected out by the postdoc phase (where I would say you have had sufficient experience with crushing scientific and engineering failure onself and watching others to be effective) and the pool of candidates therefore is vanishing.

> If you're really smart, why do you become a DARPA PM, and not a PI?

Reasons are myriad.

100% of the PMs I've worked with had tenure and successful labs prior to becoming PMs, and went back to their institution & restarted their labs after leaving DARPA. The reason for leaving your lab to be PM is fairly obvious: controlling the funding gives you a lot of leverage for shaping the priorities of the field, in a way that merely running your own lab doesn't.

Maybe your division sucked. I've only ever worked with highly competent PMs, and all of the programs I've worked on ended in commercialization.

Anyways, prosecuting individual cases doesn't seem like a particularly good way of evaluating the effectiveness of an agency.

Well, it's pretty obvious that DARPA has different standards in different divisions (division I was in was relatively new), then. Which still makes me wonder wtf they are anymore.

> division I was in was relatively new

That makes sense.

What does PI mean in this context? I've worked in software projects for 20 years and not once have I stumbled upon the term.

Principal Investigator, basically the lead researcher

It however matters quite a lot what it takes to removed them from such position and how often does that happen. It also depends whether there is a system that can identify such people people and minimize damage they cause. In this case apparently, it took a lot and there was no system.

Facebooks building 8 was run by a former darpa chief and went up in smoke. Also anecdata but it was enough to make me question how good their process really is

Keyword is "former". She probably left because of this:

https://en.wikipedia.org/wiki/Regina_E._Dugan#Potential_conf...

I don't think a single person that no longer works there can represent the entire process of an agency.

If anything, Facebook's building 8, only reflects badly on her and Facebook.

That’s a fair point. For what it’s worth the complaints I heard about her management from PI at building 8 were strictly about her style of extreme micro management which seemed to be pretty different from the rest of Facebook, he never mentioned any of the corruption allegations. For example laying out years of research with what she called “inch stones”of weekly expected progress in the initial proposal. I’ve never worked with DARPA myself so I can’t say whether that was something unique to her.

> I’ve never worked with DARPA myself so I can’t say whether that was something unique to her.

Up to the PM. DARPA is definitely more heavy-handed than other funding orgs, for whom annual reports are pretty common. But monthly check-ins is on the extreme end of things even for DARPA. Quarterly/six month are more common. Never heard of weekly, and I doubt many PIs would put up with that for very long.

Oh to make things even funnier, last I checked, said ex-DARPA pm was working on a biotech startup around diagnostics frommicro-blood draws. You can't make this shit up.

Author here. It's an important question. I would argue that DARPA does not work as well as it did in the 1960's but that your experience with PMs isn't representative.

Even though it's less than other government orgs, DARPA still has more process than it used to. Additionally, the opportunity cost for people who would make excellent PMs has become steadily higher over time.

It's worth viewing DARPA from their customer's perspective. What DARPA brings to the DoD in a crisis is a stable of highly competent engineers and scientists who want to push the envelope and are smart enough to not let a crisis go to waste. I'm on the demand side right now and it's been awesome to see all these crack folks swarm down on my requirements.

If you think the DoD is behind the power curve, um, well, I have seen things that don't officially exist show up, obviously already engineered, so fast you can calculate from the shipping label how long it took someone to get off the phone and get it onto a plane.

So, yes, there are absolutely the long game 5-year efforts (which are often really sprouts of 20 or 30 year efforts). But it's a bit like YC: the alumni network is amazing.

What does the power curve mean in this context?

I'd like to write a comment but I generally prefer to have read the entire article before I do... see you in a week or so.

Jokes aside, I agree with the author (I think) that DARPA has been a surprisingly effective org in an age of frequent failures of other orgs with similarly lofty goals.

DARPA is effectively what things like the SoftBank Vision Fund should've been (wanted to be?). It would certainly be interesting to see what it would look like to have a privately run clone of DARPA if you injected as much cash as the Vision Fund did. Per the article, about $400M/y is spent on actual R&D by DARPA, where as Vision Fund has injected that amount into single companies many times over.

Vision Fund is there to throw huge amounts of money at a business in order to ramp it up quickly and capture large, global markets.

Think Uber: disrupting the Taxi business (or just regulations ...). Once something like Uber starts to work, a 'Vision Fund' takes this fledgling thing and backs it with billions to conquer the world.

It's not about moon-shots, it's about market power and speed on a global scale.

Like hyper-supercharged 'Round C or D' - instead of doing an initial public offering for cash, you take on massive cash from Vision.

Wether or not it will work is something else, but there's logic there.

Oh yeah, absolutely. I just kinda wish we had that level of private interest/investment with similarly lofty goals. Not just "eat the world".

My more top level view is that DARPA and such are just side channels to sneakily funnel public money into research.

In theory free market should take care of the economy. It's really good at optimizing manufacturing processes and getting stuff as cheaply as possible into hands of as many people as possible.

However there's one thing essential to the economy that free market sucks at. It's research. And that's not because companies are bad at doing research. It's because companies are bad at funding research. Research is inherently risky and no sane capitalist will invest in anything beyond tinker level research because he will loose. And he's not supposed to loose.

In theory you could fund research overtly with country budget, but no one is going to support that. Why spend money on eggheads playing with useless stuff if people are hungry and streets are dirty?

What nobody opposes is giving more money to the military. And what anyone can't oppose is military spending money in whichever way they please. So they can spend it on research. Most will be wasted, very few will have actual military potential. The rest can be graciously dumped into the economy for companies to tinker, optimize making of, manufacture, market and sell.

IMHO military is the core of USA success (or even success of global capitalism), not through might, but through ability to get plenty of public money and ability to allocate it into things that would never get funded in any other way.

Great comment.

I agree that companies would much rather do the exploiting and leave exploration to the government.

Well, the public does support money going to universities.

And I believe there are quite some people, who opposes more money going to the military.

When you say that ‘DARPA’ “works”, what precisely do you mean by that, and how do you measure success? It seems to me that it primarily works by throwing large amounts of money at academics and then claiming credit for anything they invent, whether or not it contributes to whatever DARPA is trying to accomplish.

If it were possible, a closer reading of history might provide additional lessons. For instance, ARPA took a hit in the early '90s under Bush. Some silver linings, but that's a transition which might illuminate process tradeoffs.

There was (at least back then) an ongoing meta discussion about how to do better. So it might be useful to explore not just the organizational designs which were realized, but also the space of things considered. For instance, before Bush hit, there was discussion of tiny "fix that!" grants. Like there's one person who is an outlier in understanding how to do X, and their book just isn't getting finished. So rather than society waiting years on diffusion and reinvention (which is what ended up happening), it might be worth paying someone to sit outside their office, and stand on their desk, and be a forcing factor on making the book happen.

At least back then, with a failed attempt at a commerce ARPA clone, it was thought important to have a clear metric to prioritize projects. "What's better for DoD?", rather than the far less tractable "what's better for the commercial economy?".

Perhaps I missed it skimming, but a major issue has been the death valley between research and commercial impact. And attempts to address that making things even worse (ie, researchers encouraged to think commercialization, sacrifice impact by holding things close, and then commercialization generally fails, so there's no offsetting benefit). And there's the unfortunate pipeline from research to patent to unsuccessful startup to dominant company having yet more anticompetitive ammo. I wonder if it might be fruitful to broaden focus to the research pipeline rooted in ARPA? Because a successful clone would presumably again face this difficulty. And there might be some other design point that is less ARPA-like, but does impact better.

How well ARPA works comes and goes. It's not a stable equilibrium. So instead of asking how to create a successful clone, perhaps one might ask how to create something viable in the vicinity of success, and separately, how to increase time spent less distant from success?

The difference between an old-school autonomous PM, rolodex and checkbook in hand, showing up on your doorstep and saying "I've heard you interested in doing X - what would you need?", and say NSF exploratory grants of "groups with the following characteristics, may submit grants addressing the following issues, with a deadline of mumble, and the following logistics"... is really really big.

Darpa is incredible because it's run by scientists and not managers. A lot of small tech is also run like that, it's so refreshing.

There's an organization closely resembling DARPA but with exclusive focus on computing. Wish more gets written on it →

https://en.wikipedia.org/wiki/Intelligence_Advanced_Research...

IARPA is specifically mentioned in the article.

I've done some work around the edges of IARPA. It's not exclusivly computing BTW.

It's pretty much DARPA with smaller budgets. Some pretty interesting projects - the one about increasing human intelligence with magnets is pretty far out!

"the one about increasing human intelligence with magnets is pretty far out"

Sounds very esoteric to me. Is there a link somewhere? I would like to know, if there is substance to it, or that someone was able to talk quite good.

It's actually both magnetic (transcranial magnetic stimulation) and electro-magentic stimulation (transcranial direct current stimulation): https://www.theatlantic.com/magazine/archive/2014/09/prepare...

There's a reddit dedicated to people trying it on themselves: https://www.reddit.com/r/tDCS/

I wouldn't say the results are exactly conclusive, but there did seem to be some results showing that pain can be reduced in some circumstances.

And while "magnets" sounds like some kind of crystal therapy or something, ignoring those aspects it's actually exactly the kind of way-out things that *ARPA should be funding.

Thanks for the links.

"but there did seem to be some results showing that pain can be reduced in some circumstances"

Even though that sounds not more deep than placebo.

Apart from that I am also open to crystal therapie. You know, the world is quite made up of quants and crystals can focus and polarize quants .. so if someone has some claim with more substance than that and wants to do a scientific research about it, why not. But so far I have seen only esoteric "research".

It seems that this would be a much better way of funding research in a general sense, since that seems to be the general process of their problem solving. The only thing is that to apply this to other fields, the barrier for getting people that are equally motivated and intelligent about analyzing if the research is 'going' somewhere is high

I would say the key difference between DARPA funding and other research functing is that DARPA PMs have a crystal clear map of the leading edge in a field that's ready for advancement and can target it perfectly with both money and real competition.

I think it comes down to the fact that the military has a lot of money and the projects have goals that are somewhat directed but also long term. Don't think it's more complex than that really.

it works because there are no google bros

This is correct. The privately run version of this setup Alphabets Project X was wrecked by lousy hiring decisions. Who the heck recruits people like this?

“ As the Times reported, DeVaul, whose title is “director of rapid evaluation and mad science,” told a young female job candidate during a 2013 interview that he was in a polyamorous relationship. Later, when he saw the woman at Burning Man while she was still waiting to hear back about the job, DeVaul asked her if she would take off her shirt for a back massage.”

https://slate.com/technology/2018/10/google-x-sexual-harassm...

Reply on news.ycombinator.com