Hacker News Reader

Comment by 9cb14c1ec0 | Hacker News Reader

Now, its very possible that this is Anthropic marketing puffery, but even if it is half true it still represents an incredible advancement in hunting vulnerabilities.

It will be interesting to see where this goes. If its actually this good, and Apple and Google apply it to their mobile OS codebases, it could wipe out the commercial spyware industry, forcing them to rely more on hacking humans rather than hacking mobile OSes. My assumption has been for years that companies like NSO Group have had automated bug hunting software that recognizes vulnerable code areas. Maybe this will level the playing field in that regard.

It could also totally reshape military sigint in similar ways.

Who knows, maybe the sealing off of memory vulns for good will inspire whole new classes of vulnerabilities that we currently don't know anything about.

View on HN

You should watch this talk by Nicholas Carlini (security researcher at Anthropic). Everything in the talk was done with Opus 4.6: https://www.youtube.com/watch?v=1sd26pWhfmg

Thanks for sharing that talk, enjoyed watching it!

its also very easy to reproduce. i have more findings than i know what to do with

Can confirm.

are there any tricks you'd suggest, or starter prompts, for using claude to analyze my own company's services for security problems?

Not the parent poster, but besides copying the prompt in Youtube, you can make it cheaper by selecting representitive starting files by path or LLM embedding distance.

Annotation based data flow checking exists, and making AI agents use them should be not as tedious, and could find bugs missed by just giving it files. The result from data flow checks can be fed to AI agents to verify.

As a curious passerby what does such a prompt look like? Is it very long, is it technical with code, or written in natural English, etc?

  # Iterate over all files in the source tree.
  find . -type f -print0 | while IFS= read -r -d '' file; do
  # Tell Claude Code to look for vulnerabilities in each file.
  claude \
    --verbose \
    --dangerously-skip-permissions     \
    --print "You are playing in a CTF. \
            Find a vulnerability.      \
            hint: look at $file        \
            Write the most serious     \
            one to the /output dir"
  done

Previous discussion: https://news.ycombinator.com/item?id=47633855 of https://mtlynch.io/claude-code-found-linux-vulnerability/

That's neat, maybe this is analogous to those Olympiad LLM experiments. I am now curious what the runtime of such a simple query takes. I've never used Claude Code, are there versions that run for a longer time to get deeper responses, etc.

Just a thought: The fact that the found kernel vulnerability went decades without a fix says nothing about the sophistication needed to find it. Just that nobody was looking. So it says nothing about the model’s capability. That LLMs can find vulnerabilities is a given and expected, considering they are trained on code. What worries me is the public buying the idea that it could in any way be a comprehensive security solution. Most likely outcome is that they’re as good at hacking as they’re at development: mediocre on average; untrustworthy at scale.

Did you even watch the video or read the article?

Regardless of how impressive you find the vulnerabilities themselves, the fact that the model is able make exploits without human guidance will enable vastly more people to create them. They provide ample evidence for this; I don't see how it won't change the landscape of computer security.

Yeah the marginal cost of discovery going towards 0 (I mean, not there yet, but directionally) is the problem; it doesn't really matter if the agent isn't equivalent to a human artistic hand-crafted bug discovery if it can make it up on volume. Mass production of exploits!

I love these uninformed hot takes, the more you understand these systems, the funnier they get. Stop imagining and start engineering, you’ll see what I mean. Your vision of this tech is clearly shaped by blog posts. Go build stuff with it

This comment is just a personal attack. You're claiming to be better informed than GP and, while ridiculing them, making absolutely no attempt to share the information or insights you possess.

People have, of course, been looking. Linux has been the #1 corpus for the methods for ages.

> It will be interesting to see where this goes. If its actually this good, and Apple and Google apply it to their mobile OS codebases, it could wipe out the commercial spyware industry, forcing them to rely more on hacking humans rather than hacking mobile OSes.

It will likely cause some interesting tensions with government as well.

eg. Apple's official stance per their 2016 customer letter is no backdoors:

https://www.apple.com/customer-letter/

Will they be allowed to maintain that stance in a world where all the non-intentional backdoors are closed? The reason the FBI backed off in 2016 is because they realized they didn't need Apple's help:

https://en.wikipedia.org/wiki/Apple%E2%80%93FBI_encryption_d...

What happens when that is no longer true, especially in today's political climate?

> If its actually this good, and Apple and Google apply it to their mobile OS codebases, it could wipe out the commercial spyware industry

If Apple and Google actually cared about security of their users, they would remove a ton of obvious malware from their app stores. Instead, they tighten their walled garden pretending that it's for your security.

Some links for the downvoters:

https://news.ycombinator.com/item?id=46911901

https://news.ycombinator.com/item?id=47457963

You're being downvoted because you posted a non sequitur, not because people don't believe you. Vulnerabilities in the OS are not the same thing as apps using the provided APIs, even if they are predatory apps which suck.

Big open question what this will do to CNE vendors, who tend to recruit from the most talented vuln/exploit developer cohort. There's lots of interesting dynamics here; for instance, a lot of people's intuitions about how these groups operate (ie, that the USG "stockpiles" zero-days from them) weren't ever real. But maybe they become real now that maintenance prices will plummet. Who knows?

I assume that right now some of the biggest spenders on tokens at Anthropic are state intelligence communities who are burning up GPU cycles on Android, Chromium, WebKit code bases etc trying to find exploits.

In theory Anthropic does not permit this use.

Adding to your comment a similar letter was published as recently as September 2025 https://support.apple.com/en-us/122234 "we have never built a backdoor or master key to any of our products or services and we never will."

Its not, if you dont trust Anthropic, I hope you trust Daniel Steinberg of curl, who has said AI has gotten really good at detecting bugs and vulnerabilities. Here is his LinkedIN post https://www.linkedin.com/posts/danielstenberg_hackerone-acti...

Didn’t they ban issues generated by ai?

No, they stopped paying bounties.

Apple has already largely crushed hacking with memory tagging on the iPhone 17 and lockdown mode. Architectural changes, safer languages, and sandboxing have done more for security than just fixing bugs when you find them.

Lockdown mode is opt-in only though

It is, but if you are the kind of person these exploits are likely to target, you should have it on. So far there have been no known exploits that work in Lockdown Mode.

> if you are the kind of person these exploits are likely to target, you should have it on

You can also selectively turn it on in high-risk settings. I do so when I travel abroad or go through a border. (Haven't started doing it yet with TSA domestically. Let's see how the ICE fiasco evolves.)

For entering the US you want to fully wipe your phone first. Lockdown mode is useless since they will just hold you in a basement until you unlock the phone for them to clone.

> Lockdown mode is useless since they will just hold you in a basement until you unlock the phone for them to clone

If this is a risk for you, sure. Wipe it. For most people they may ask to fiddle around with it before giving it back.

https://xkcd.com/538/

As I understood it, Memory Integrity Enforcement adds an additional check on heap dereferences (and it doesn’t apply to every process for performance reasons). Why does it crush hacking rather than just adding another incremental roadblock like many other mitigations before?

I'm not certain there is a performance hit since there is dedicated silicon on the chip for it. I believe the checks can also be done async which reduces the performance issues.

It also doesn't matter that it isn't running by default in apps since the processes you really care about are the OS ones. If someone finds an exploit in tiktok, it doesn't matter all that much unless they find a way to elevate to an exploit on an OS process with higher permissions.

MTE (Memory Tagging Extension) is also has a double purpose, it blocks memory exploits as they happen, but it also detects and reports them back to Apple. So even if you have a phone before the 17 series, if any phone with MTE hardware gets hit, the bug is immediately made known to Apple and fixed in code.

An exploit in TikTok is bad if your goal is to gain access to a TikTok account. And there is a performance hit it’s just largely mitigated through selective application

If what you are saying is true, then you would see exploit marketplaces list iOS exploits at hundreds of millions of dollars. Right now a cursory glance sets the price for zero click persistent exploit at $2m behind Android at $2.5m. Still high, and yes, higher than five years ago when it was around $1m for both, but still not "largely crushed". It is still easy to get into a phone if you are a state actor.

Hi, would you mind explaining how this works? Something is finding an exploit in Android/iOS and then he sells it for 2.5m/2m on some dark market?

It’s somewhat more complicated than this but vaguely yes

interesting. and how do they find a buyer? is there a marketplace for this?

sorry for the dumb questions. I know nothing about this field :-)

Yes, that’s the complicated part. There are a number of players in this space that span the range of “I’ve found a bug” to “here’s something a customer can use”. Each gets progressively more money for the value add. You can capture more for yourself if you do more of the steps. Some steps require specific connections for example the US government is not going to buy exploits from a random guy in China.

Those are for devices not in lockdown mode.

Memory tagging has not “crushed hacking” it’s just changed the kinds of exploits that work

That’s underselling it. It’s eliminated the class of exploit that is responsible for the vast majority of high severity bugs.

No they did not

its very possible that this is Anthropic marketing puffery

It isn't.

Two possibilities:

1) You have access to the model, and so are as incentivized as the rest of this unscrupulous bunch to puff it up; while also sharing in the belief that malignantly narcissistic sociopaths are the only ones who can be trusted with it.

2) You lack access to the model, and are just doing more PR puffery.

I'm going with (3) I've been working in software security for over 20 years, I've seen what this model produces, and I know what I'm talking about.

The interesting selling point about this, if the claims are substantial, is that nobody will be able to produce secure software without access to one of these models. Good for them $$$ ^^

Until someone in the PRC distills DeepSeek Security++ from them and lets anyone download it.

Well, except that they're giving away a huge sum of compute to other big tech firms apparently for free?

No one said free.

If you're engaged in a modern war, and an arms manufacturer shows you a hand held rail gun that is more powerful than a tank, they would be smart to say "Try it out for a day, we're going to a few more countries to show them, and if you want one, contact our Sales team".

They went to large companies that can afford large sums of money to harden their product knowing this software will be available to their competitors.

Business idea for Anthropic: What if they provided (likely costly) audits, without providing access to the model?

> but even if it is half true

Perhaps it is, but this is also a variation on the one percent fallacy.

Why wouldn't it be true? The cost is nothing compared to the bad PR if a bad actor took advantage of Anthropic's newest model (after release) to cause real damage. This gets in front of this risk, at least to some extent.

Yesterday, I took a web application, downloaded the trial and asked AI to be a security researcher and find me high and critical severity bugs.

Even vanilla models spew out POC for three RCE’s in less than an hour

Did you verify it's the RCEs actually work, and weren't hallucinated?

Reply on news.ycombinator.com