This Should Not Be Possible | Hacker News Reader

This Should Not Be Possible

ghuntley.com

14 pointsghuntley12 comments

This is a troll, right? The strace for "ls" on my system consists of 1) checking if selinux is enabled (which their re-implementation does not do), 2) openat -> fstat -> getdents64, 3) writing out the information those syscalls gave you.

Jumping from "oh my gosh you can (incorrectly) reimplement ls from a trace of syscalls" to "this means the end of binary blobs" is a level of detachment from reality I've most recently seen from Geoff Lewis. Never mind that the author mixes up eBPF tracing and strace.

This is a weird post as the generated program doesn’t even output results.

I tried a similar test using Python and Claude Code and it generated something that outputted a sorted list of directory contents, like ls.

However, unlike the author, this doesn’t lead me to think this is AGI. How many posts are there out there that these models would be trained on have people explaining strace outputs and comparing code to them?

Why is this impossible? If the LLM has effectively seen all the code that could lead to that trace, all it has to do is to pick the bits out that has the highest chance to map to it right?

Is it a new marketing strategy to start by saying you're "incredibly cynical" about something that you're going to say the exact opposite about, perhaps to mask arguments with little rigor?

A case of https://knowyourmeme.com/memes/you-keep-using-that-word-i-do...

I think you can significantly save on your claude tokens if you just give the llm `head -n 1 TRACE`

      execve("/usr/bin/ls", ["ls"], 0x7fff7e969540 /* 110 vars */)

Some how it figures out how to implement `ls`. That indeed shouldn't be possible. It puts an AI skeptic into believing we're in post-AGI territory here.

Edit: tbf, I did not notice they replaced `ls` with `lol` in the trace file to avoid this, but still. To think the strace of ls was never part of the training data of the LLM then to think that is enough to somehow reverse engineer what binary blob is is insane thinking https://github.com/ghuntley/strace-to-application/blob/trunk...

If you would, please explain why it shouldn't be possible (I am not a software engineer).

I was being sarcastic. The author asked an LLM to implement `ls`* and was so surprised when it did that he thought maybe he can use that insight into reverse engineering firmware blobs. Complete delusion, but chatbots have that effect on some people.

* if you don't know what `ls` is, just think like implementing the simplest possible application that's 2 or 3 lines of code.

To be fair, the author claims he removed any mention of "ls" from the file. But also to be fair, the post is annoyingly click-baity. I'd be impressed if the LLM could generate more than 2 lines of code from a strace.

The LLM also failed to implement it. The rust code produced doesn’t actually result in the same syscalls as the strace contained.

It doesn't even print the results lmao

This guy's such a weird writer. Speaking like he discovered YOLO mode and called it the Ralph Wiggum method.

This was underwhelming, but I like his websites styling.

Why does this guy keep using words like “skeptical” and then demonstrating the opposite attitude?

Do I have to do all the skepticism myself? FFS!

Reply on news.ycombinator.com