Hacker News Reader

Comment by throwaway71271 | Hacker News Reader

[flagged]

There is a bigger chance that a toddler smashing a keyboard finds a bug than gpt5. LLMs can't understand intent, so they literally work like `grep` with little to no understanding of the context, so most of the time it will false flag good code.

There are already a lot of tools already to find bugs, like fuzzers, but I am sure that LLMs won't be one of them.

Llm powered / guided fuzzer would be pretty cool though.

https://github.com/google/oss-fuzz-gen

they don't need to understand intent, they just need to find exploits. they don't even need to do it by reading code alone - give them a vm running the code and let them throw excrement at it until something sticks!

Writing an exploit is usually much more difficult than patching the underlying bug.

Half of the work in fixing a bug report is getting a reproducible example. Nay, more than half.

If there was a magic AI which could generate exploits, I'd imagine there would be an equally magic AI patching the holes right out.

Maybe but keep in mind that there’s often a substantial lag in practice between a fixed vulnerability and its deployment into production.

That said, I’m quite skeptical there’s any AI’s on the horizon that can autogenerate exploits from CVEs.

It’s definitely nowhere near capable of doing that.

Is "with a sufficiently smart LLM" the new "with a sufficiently smart compiler?"

"imagine feeding this into an LLM/ChatGPT" is the new "imagine a Beowulf cluster of these"

grendellm?

Apparently GPT-4 has some capacity to conduct exploits this by "reading" CVE reports. I don't know if it can autonomously create exploits though:

GPT-4 can exploit vulnerabilities by reading CVEs (theregister.com) 81 points by ignoramous 60 days ago | hide | past | favorite | 29 comments

https://news.ycombinator.com/item?id=40101846

which links to a Register Article[0], which links to a paper[1]:

"In this work, we show that LLM agents can autonomously exploit one-day vulnerabilities in real-world systems. To show this, we collected a dataset of 15 one-day vulnerabilities that include ones categorized as critical severity in the CVE description. When given the CVE description, GPT-4 is capable of exploiting 87% of these vulnerabilities compared to 0% for every other model we test (GPT-3.5, open-source LLMs) and open-source vulnerability scanners (ZAP and Metasploit). Fortunately, our GPT-4 agent requires the CVE description for high performance: without the description, GPT-4 can exploit only 7% of the vulnerabilities."[1]

[0] https://www.theregister.com/2024/04/17/gpt4_can_exploit_real...

[1] https://arxiv.org/pdf/2404.08144

Yes, that sounds about right. LLMs aren’t quite good enough to find novel bugs and exploit them like a human would.

Yeah, that works for web vulns where the vuln description is practically the exploit anyway. I could write a perl script that parses out variable names and writes sql injections for it.

For comparison, in the native world program is considered vulnerable when someone finds arbitrary write primitive (even without leak), use after free, and even double free. There is a huge gap between these and actually having a working RCE exploit. Most CVEs in this space are given without a working exploit ever written.

Have you used GPT-5?

If you aren’t using GPT-6a then you are years behind.

GPT-69 is already far ahead of 6a.

Been using 73 for months now.

you need to wake up at 4am and have a cold shower!

12 before 12, 12 cold showers before 12 allowing GPT-7 to take care of my daily needs.

GPT-5, maybe not. But somebody somewhere is building something that can do that. And if they can't do it _now_ they have a plan that tells them what's missing. TLDR; it's coming, soon.

Writing exploits is a bit of an art-form. Current incarnations of GPT have trouble writing code at a level more advanced than a junior developer.

and lots of people are spending lots of time and money on AI Coding Assitants... which is more or less the knowledge base you need.

If they could use that structural training to answer queries like "Is there any code path where some_dangerous_func() is called without it's return value being checked"...

You can do this today by querying the AST output by a compiler. Regardless, the parent comment was talking about exploits, not vulnerabilities/bugs. Vulns are a dime-a-dozen compared to even PoC exploits let alone shippable exploits.

Ok, so add "and generate a C program to exploit it" to the prompt.

You're either being sarcastic or wildly underestimating how hard it is to write an exploit. I haven't written about exploit dev publicly for a _long_ time, but I invite you to read https://fail0verflow.com/blog/2014/hubcap-chromecast-root-pt... for what I consider to be a pretty trivial exploit of a very "squishy" (industry term) target.

XNU isn't the hardest target to pop but it is far from the easiest.

There's nobody more confident in the world, than HN poster wiring about a topic they have no experience with.

There is a huge gap (in the binary exploitation world) between identifying a problematic code pattern and having a workable bug (a reproduction), and even larger one between a reproducible crash and a working exploit (because we're not in the 90s anymore and complier/hardware mitigations are literally always enabled). Current LLMs can cross neither gap, and are not even close to bridging the second one.

> Like when you can just send one icmp packet with `+++ath0` and just disconnect someone's modem

Oh, I remember the "XDCC SEND KEYLOGGER 0 0 0" exploit from IRC era ~2010... dumbass middleboxes would yeet anyone whose packets crossed them.

the real win will be when it can also generate the codename for the exploit. FATEFATAL

Reply on news.ycombinator.com