Hacker News Reader

Comment by om8 | Hacker News Reader

Not surprising, llama.cpp code is a mess.

It's sad that hacked things that emerge first are way more popular than properly done projects that come later.

In fact the llama.cpp codebase is well-developed and actively maintained. It has undergone iterative security hardening, intensive low-level security checks have been implemented in both the core inference engine and RPC components.

This standard of security is what made the exploitation such challenging and rewarding.

Thanks for the writeup! Was a very interesting read! I've subscribed and I am looking forward to your next exploits! ^_^

It’s actively maintained but I wouldn’t classify it as a clean codebase. Neither the abstractions it has within ggml, the structure of llama.cpp, effective use of modern c++ etc. it can’t even really make up its mind as to whether it should be c++ or c and there’s a lot of dirt because of that. Heck instead of using a submodule they’re copying ggml between projects making it very difficult to keep track of what’s actually happening where and what the ground truth is. It’s sloppy engineering. Parts are better designed for sure.

None of that is meant to take away from your effort or the success of llama.cpp, but I have spent quite a bit of time reading and working with the internals across layers and have a good eye for quality c++ patterns.

Is there a comparable open source thing "done properly"?

llama.rs, of course /s

Reply on news.ycombinator.com