Hacker News Reader

Comment by josh-sematic | Hacker News Reader

Must be nice to be in a position to sell both disease and cure.

That's exactly not what they're doing. They aren't creating operating system vulnerabilities. They're telling you about ones that already existed.

Well, in a slightly indirect manner. Claude is writing a ton of code, and therefore creating a lot of security vulnerabilities.

That's not what's happening here. This announcement is about the velocity with which Claude finds vulnerabilities in already-existing software.

Software already exists that has been written by Claude. They absolutely are selling the means to write software, and the means to securing the insecure software. At least for the time being. In the future Mythos will probably just make it possible to prompt good software from the start.

Ok. But mostly its entirely the old software, not the new software, that the bugs are being found in.

Maybe because there’s no critical and widely used software written by LLMs so far? Which says a lot about LLMs are failing to even approach the level of capabilities you would expect from all the hype? The goal has always been, even before LLMs, to find something smarter than our smarter humans. So far the success at that is really minuscule. Humans are still the benchmark, all things considered. Now they’re saying LLMs are going to be better than our best vulnerability researchers in a few months (literally what an Anthropic researcher said in a conference). Ok, that might happen. But the funny part is that the LLMs will definitely be the ones writing most of these vulnerabilities. So, to hedge against LLMs you must use LLMs. And that is gonna cost you more.

So today, most of the vulnerabilities being found by these tools are in code written by humans. Your hypothesis is that down the road, most of the vulnerabilities will be in code written by LLMs.

What seems more probable is that the same advances that LLMs are shipping to find vulnerabilities will end up baked into developer tooling. So you'll be writing code and using an LLM that knows how to write secure code.

I don't think claude wrote openbsd but to be honest that was before my time so I'm not sure

If it’s very good at finding security vulnerabilities, I would assume that the code it generates is much more hardened than anything your average developer can put out.

Dario is big on beating china, and no doubt he believes cyber security is how to do that. You can tell, but anthropic is sht at everything else. Nobody uses it for real research.

Mythos aside, frontier LLMs can already be used to find exploits at faster pace than humans alone. Whether that knowledge gets used to patch them or exploit them is dependent on the user. Cybersecurity has always been an arms race and LLMs are rapidly becoming powerful arms. Whether they like it or not LLM providers are now important dealers in that arms race. I appreciate Anthropic trying to give “good guys” a leg up (if that is indeed their real main motivation which I do find credible but not certain). But it’s still a scary world we’re entering and I doubt the fierce competition will leave all labs acting benevolently.

Yeah, I'd pretty pissed at my doctor for finding cancerous cells that probably wouldn't have been a problem for quite some time, either. Ignorance is bliss, security through obscurity, whatever.

The doctor analogy is more like you're grateful that your doctor found cancerous cells before they became a problem, but at the same time his other business is selling cigarettes.

You may joke, but this is a genuine issue in certain screening tests. e.g. most cancerous cells found in PSA prostate screening are so slow growing that they never cause any symptoms during a person's lifetime, so the treatment is almost always worse than the disease. It's similar for some sorts of thyroid and breast cancer tests. This is why a lot of countries are heavily reducing these sort of tests

Reply on news.ycombinator.com