Hacker News Reader

Comment by sys32768 | Hacker News Reader

Fable and now Opus 5.5 won't answer my college student's prompt about Alzheimer's and immune response.

ChatGPT 6 Pro answered it without issue.

I am honestly still confused about this limitation. I can understand cybersecurity, because mass "hacking" can be automated and Claude itself can help you do it, but biology...? Is it that easy to manufacture and distribute viruses and whatnot?

I imagine some terrorists in a cave with a lenovo laptop manufacturing bio weapons with some flasks and Claude

Right next to the hypersonic missile vibecoder

You joke but it's really easy to develop viruses at home. You can order everything you need online, and it's not expensive nor does it require a particular skill set.

God I wish people on HN would just shut up with snarky takes about things they don't understand. This place used to be better.

Considering there are many high-school competitions in genetic editing, some listed at [0] as well as a whole biohacker culture, and labs providing gene sequencing as a service e.g., [1,2], we can reasonably assume it is not beyond the reach of some garage lab to accidentally or deliberately spread a deadly pathogen if it can find the right sequence.

So, yes, having an unconstrained frontier AI doing the searching and analysis to find the right (i.e., wrong and deadly) sequence would massively increase the odds some garage biohacker or small aggrieved nation-state starting the next pandemic.

[0] https://www.sciencebuddies.org/projects-lessons-activities/g...

[1] https://www.genewiz.com/public/services/sanger-sequencing

[2] https://plasmidsaurus.com/

You can order genes online, and some say you can assemble using stuff cobbled together in a home lab rather easily. For the last 20 years, I've personally felt that bio-terrorism is the highest possible risk, well above nuclear, or chemical warfare. But it does take training, expertise, or, it did.

Then stop selling genes online?

It’s like saying you sell ammonium nitrate and fuel oil online and then saying it’s too risky to let people have computers in case they use them to make ANFO. They can only make bioweapons because you’re selling them bioweapon components! They can’t make genes at home!

A fair response in general, but it's not as if I advocate it, or run a company doing it. I'm simply stating fact.

And I will point out that "stop doing that" can be applied in both directions, towards AI and towards material supply.

And that this is one way, not even remotely the only way, that gene editing at home is easy.

Again, these are just facts.

Some questions...

Is it fair to restrict AI, or fair to restrict 1000 industries?

And if it is fair to restrict 1000 industries, OK, but there should be time to do so, probably? A transition period?

And if you do restrict, many such industries just make needed chemicals, which are used by endless other, non-threatening industries.

What of them?

The companies selling genes online already scan the orders and do not fulfill anything considered a possible hazard (presumably unless it is to a known and certified lab at a serious organization).

And, this is not the only way to make genes at home.

It is a complex problem.

I'm pretty sure one reason is to influence public opinion about LLM regulation. Open-weights models cannot be restricted as effectively and they want to ban those for obvious reasons.

See real world example in https://www.anthropic.com/threat-intelligence-report-septemb...

"Here, we present five case studies of actors using our models in ways that could support biological weapons development."

And capabilities continue to improve.

HN doesn't give a shit about AI safety. The people here might care after thousands of people die, but they'll probably just call it a "marketing exercise" or blame the company - certainly won't blame themselves for cultivating an environment in which safety isn't taken seriously amongst technologists.

I'm convinced everyone here thinks of engineering ethics as some sort of joke.

Reply on news.ycombinator.com