Hacker News Reader

Comment by quyleanh | Hacker News Reader

> We also tested Astra on SRE-Bench [15], a benchmark that measures whether models can reverse engineer software binaries to understand its core logic without access to raw source code. Astra solved 88.0% of tasks in a single attempt and 99.2% within four attempts, compared with 55.9% and 68.7% for GPT‑5.6 Sol, respectively.

So the closed source application should open its source in near future?

[15] https://arxiv.org/abs/2608.11469v1

View on HN

Not if OpenAI considers reverse engineering an offensive cybersecurity skill.

Surprisingly, I've had really good luck with reverse engineering on frontier models (without being part of CVP or similar). It's more the exploit development/PoC that it locks up on, which, as I use `pi`, I just switch the model to Kimi K3 to finish up making the PoC.

Ironically, due to the stringent guardrails on American models that exist to avoid giving adversaries a leg-up in cybersecurity, I end up feeding dozens of 0days straight to the CCP lol

I do RE mostly, and they do lock up for me. But what works for me is: warm up context with non-RE things with American models > switch to GLM 5.3 with actual request, let it fill context with some RE work > switch to American > switch back to GLM 5.3 and keep switching back and forth if anything gets flagged.

Solves the guardrail problem - check Feeds China data to US - check Feeds US data to China - check

But I think OpenAI has caught up with this, and it's why they're pushing so many things server-side and encrypt it (subagent messages, compaction, new context history thing).

I was listening to the Lex Fridman / DHH podcast last night [0], and DHH was saying that this is a new era for open source software. I'd agree, and also extend it open hardware.

Recently I've seen quite a few posts from people using AI to reverse engineer the Bluetooth protocol or such on devices that need a proprietary app. The same thing for firmware is surely coming, which is great as you can get lots of fun hardware from China, but it often has shitty firmware. Once that becomes the norm there's no reason not to make it open in the first place.

[0] - https://open.spotify.com/episode/45lhw2Adbrsw0xSCOgIeg3?si=r...

Open source is about freedom, not just about being able to read the source code.

How is that relevant?

This thread seems to confuse source-available and open source

Not really? Nobody is talking about source-available code with a restrictive license. People are talking about having LLMs write up protocols and libraries and firmwares from scratch and making those freely available, as in open source.

Reply on news.ycombinator.com