Hacker News Reader

Comment by stymaar | Hacker News Reader

> It took him two hours of passing errors to Claude for the endpoint to start working

What? It's literally three actions and you're good: download llama.cpp, download the model on Huggingface, and run it with.

I have no idea how it's supposed to take two hours (unless you have a slow connection and the model download takes this much time, that is).

View on HN

I'm not saying this is the case in the particular example, but these days a lot of people aren't bothering to read a 1 page README, and are instead letting claude "just fucking do it already". I see it at work.

Claude has got better at "just fucking doing it" by asking if it's ok to go read the latest github issues and pull the README, which means that people will likely get lazier and lazier.

To me laziness has nothing to do with it. I'd rather focus on my limited time on things that actually matter, e.g. the design of an important product feature or the root cause of a bug and the proper fix. I delegate everything else to Claude. It's really nice to be able to say to Fable "see if you can get this running locally" and then come back 10-30 mins later and read the results.

I've discovered a lot of neat tooling this way that I otherwise would not have bothered tryin to set up because that can take up a lot of time. And even when it's fast, you're suffering from context-switching penalties. You framed it as "read the latest github issues and pull the README" but anyone who has worked in tech for an appreciable amount of time knows that that stuff can turn into a deep rabbit hole.

FWIW laziness is a strength, not a weakness in software engineering :)

My experience with Claude is that it suffers badly from “not invented here” syndrome. So probably it rebuilt something like llama from scratch and then 2 hours suddenly seems reasonable (if you don’t question the approach). And that’s the thing, someone with no experience isn’t going to question it.

Not necessarily rebuild llama from scratch, build attempting to build it without cmake and manually invoking all the build commands would be quite in character.

I wonder if this is an artifact of RL, where the training heavily emphasizes codegen. It may be that the model is just better at generating code than reusing libraries, so it prefers the lowest cost approach.

I also wonder if this manifests much less in contexts where the libraries/frameworks are a large part of the training set. It may be that the model doesn’t generalize well so it’s always better to use knowledge in its training set vs attempting to understand how to use a new, potentially never before seen (from the model perspective) api

In the README there is usually a section where it explains which llama.cpp parameters to configure (sampling, penalties, context size, etc) to get better results.

Your mileage may vary. I tried this a couple months ago and spent a full day on it just not working before giving up. Anything I sent, it wouldn't run.

Reply on news.ycombinator.com