Hacker News Reader

Comment by jakewins | Hacker News Reader

My wife is doing her PhD in molecular neurobiology, and was amused by this - but also noted that the question is trivial and any undergrad with lab access would know how to do this.

Watching her manage cell cultures it seems the difficulty is more around not having the cells die from every dust particle in the air being a microscopic pirate ship brimming with fungal spores set to pillage any plate of cells they land on, or some other wide array of horrors that befall genetically engineered human cell cultures with no immune system

View on HN

I was thinking this as well.

> Due to the sensitive nature of this model and of the biological threat creation use case, the research-only model that responds directly to biologically risky questions (without refusals) is made available to our vetted expert cohort only. We took several steps to ensure security, including in-person monitoring at a secure facility and a custom model access procedure, with access strictly limited to the period of the experiment. Additional considerations regarding information hazards and security protocols are detailed in the Appendix.

This is well past cringe. A lot of what the AI community puts out feels like they're LARPing scientists, similar to the valid question of whether Software Engineering is actually engineering.

Truth. AI researchers need to work with actual scientists or they ended up with the same brain-in-a-jar problems that their models have...

The difference between a software engineer and a programmer is similar to the difference between a mechanical designer and a mechanical engineer. After several years doing the same job in a well-established organization, they're not going to perform all that differently.

Pretty much this. Having worked in biopreparedness...the instructions aren't the hard part. Both creating and deploying a biological threat are wildly more difficult.

I don't fully agree. Things are becoming quite simple, which is scary.

There's a well-intentioned lobby in the UK advocating for some minimal control, i.e. checking DNA / RNA / protein sequences.

For instance, cheap mRNA synthesis as a service has lowered the barrier of entry for any malicious actor. It's frankly ridiculous there are no checks in place right now.

> minimal control, i.e. checking DNA / RNA / protein sequences

They were already doing this 15 years ago.

> It's frankly ridiculous there are no checks in place right now.

It's frankly ridiculous that you think this.

That's your opinion, but I've seen firsthand that there's a lack of controls in this area.

My experience, not my opinion. I personally tripped over the checks many times, back before they got institutional authentication smoothed out.

I regularly hear bio researchers claim firtshand experience that there are no checks, but then it turns out they were using institutional names, credit cards, and shipping addresses the whole time, so what they actually experienced was a lack of false positives rather than an absence of checking.

>> minimal control, i.e. checking DNA / RNA / protein sequences

> They were already doing this 15 years ago.

I work in this area. Some synthesis providers check, others don't. And the checking isn't great.

(Some of my coworkers work on https://securedna.org which is trying to make this screening more robust.)

TL;DR: I don't understand how adding homomorphic encryption makes a cloud virus scanner for physical pathogens a better idea

> Only authorized researchers should be able to obtain DNA permitting them to assemble pandemic-capable agents.

Imagine this becomes legally mandatory. What happens when they get breached? For example:

* Does it enable denial-of-service attacks by returning false positives hospitals legally can't ignore?

* Does it return false negatives as part of another attack? Example: printing known or novel virii which will be introduced in a specific lab?

* Does the attacker selectively perform these behaviors as part of an action against a specific person or area?

Metadata seems like it would enough to target the physical parts of an attack usefully, even without plaintext.

The best security & privacy approach I've seen in hospitals is keeping critical services on-prem, or at least strictly intranet-only. The latter ends up being forced by physical scale once a medical complex grows large enough to span multiple buildings.

The firmware embedding mentioned on their site seems better than cloud, yet still seems misguided. How will you debug problems, especially if you never have access to the plaintext? Formal methods and verified programs? Provably equivalent encrypted and plain operations?

> I don't understand how adding homomorphic encryption makes a cloud virus scanner for physical pathogens a better idea

The reason to use a cloud service is that you can check whether someone is trying to synthesize something hazardous without distributing a list of the hazards. There are a lot of subtle ways to cause harm with biology, and you don't want to tell people where to look.

Then the reason to use homomorphic encryption is that otherwise the cloud service learns which sequences people are trying to synthesize. Biotech companies care a lot about keeping their in-progress work private, so they'd reject a cloud system without this.

> Does it enable denial-of-service attacks by returning false positives hospitals legally can't ignore?

Aside: I'm confused why you're saying "hospitals" -- medical providers don't synthesize things, it's researchers at universities and biotech companies. Maybe you're using "hospitals" as in "research hospitals" which is fine, but maybe you're using it because you're under the impression that synthesis is part of medical treatment?

To answer your question, though, if an attacker managed to add something normal and harmless to the DB then yes, a benchtop synthesizer would refuse to synthesize the sequence. The lab would escalate, it would be sorted out, there'd be a postmortem etc.

> Does it return false negatives as part of another attack? Example: printing known or novel virii which will be introduced in a specific lab?

Yes, if an attacker managed to remove a harmful sequence from the database then it wouldn't raise any flags if they or a confederate tried to synthesize that sequence.

Both of these cases come down to "it's important that you have good controls around what's in the DB, and a secure process for making changes."

> Metadata seems like it would enough to target the physical parts of an attack usefully, even without plaintext.

I'm not sure what you're saying here.

"Aside: I'm confused why you're saying "hospitals" -- medical providers don't synthesize things, it's researchers at universities and biotech companies"

During at least two pandemics I've worked on, I've known hospital labs synthesizing their own PCR primers for diagnostic tests.

If someone really, really wants to do it they'll buy a second hand oligonucleotide synthesizer and the reagents, and do the larger scale assembly the old fashioned way.

Still, it's worth keeping the barrier of entry higher than just being able to order it.

As per the homomorphic encryption discussed earlier, you can easily avoid the 'need' for this by having federal governments operate the screening tools.

> it's worth keeping the barrier of entry higher than just being able to order it.

Exactly! Keeping groups with lots of resources from synthesizing harmful things isn't a battle we can win (and in a decade or so biological design tools will make scanning synthesis orders a very weak precaution) but raising the bar means fewer people are in a position to cause large-scale damage.

(Long term we also need to make society more robust to biological attacks, with better monitoring, better PPE, and faster countermeasures.)

...what do you think a vial of mRNA actually does?

I'll give you a hint: if you touched it to your finger, it would be completely destroyed.

You don't want RNA anyway, you want to turn the virus's RNA sequence into a DNA sequence (reverse transcription), then do whatever editing you want once that's in your computer, then synthesize the DNA, and grow up the virus in culture by introducing the DNA. The cells in the cell culture will turn the DNA into RNA for you and then assemble the viruses.

Also the whole "testing" part. Unless you're just culturing an already known virus (in which case, why do you need to synthesize anything? You have it in a test tube or something) then there is not in fact any real way to know whether or not some particular genetic modification translates to any type of favorable property. Even gene-linked desirable traits in one species may not correspond to that trait in another, particularly when your metrics are things like "transmissivity" and "lethality" and not "I need slightly more peas per pod".

When we did this kind of thing in high school we had huge problems with contamination, but I don't agree that it is so hard.

I think the barrier is really that up until now exactly zero who want to do biomedical research have also wanted to kill huge numbers of people, with the exception of some idiots in the past who worked on state bioweapons.

Reply on news.ycombinator.com