Hacker News Reader

Comment by JosNun | Hacker News Reader

Genuinely curious: what are people using these smaller local models for? They are getting decently capable, but they are still small enough that I don't trust them for "real" work outside of a handful of fun toy projects.

Are people actually using them in coding agents? Or are they mostly using them for other things?

View on HN

We've shipped some code generated by Qwen3.6 27B to production (under OpenCode). It lacks the breadth of knowledge of models like Opus, but if a change is fully inferable from the prompt and the surrounding code, it works very well. It won't be able to write something from scratch that requires niche knowledge (say, a performant inference engine tailored to Blackwell GPUs), but if it's just a PR adding a new use case to an existing project (which is usually just "load from the DB, do some invariant checks, modify the entities, store them back"), it works as well as Sonnet (provided you have the correct configuration, like recommended temperature and top-p settings, the model isn't over-quantized, you have at least 150k tokens of context available, etc.).

I don't use them as coding agents, but they can be very useful for things like text transformation, summarizing, or text extraction.

That said, if you have a subscription to a paid model already, you're not necessarily winning out on anything except perhaps privacy, which isn't nothing.

there is plenty of grunt work these smaller models can do. update dependencies, fix merge conflicts, write --help, markdown, or readme files for existing code. etc.

sometimes they fail but undo is just a "git restore" or if automated, rejecting a PR and having a better model take a crack at it.

They're great at helping me look up web dev stuff when I don't have internet access.

I use a gemma4 model locally to extract content from messages to a personal agent I'm building for its memory graph (to break the message up into the topic, source (assistant or owner), facts, entities, etc. in the message content (all getting thrown into a magma-esque graph using NLEmbeddings for memory search). This is for a custom personal agent that targets deepseek-v4 flash. The local model is too slow in my setup for a chat agent, but for memory extraction it works pretty well, saving API usage on every chat turn.

Qwen35ba3b can do a huge amount of data cleaning work on pretty modest hardware. Already have run about 100 billion tokens on it using 2x3090 gpus.

What exactly do you mean by data cleaning

The coolest project I’ve got this running on is improving the depicts metadata for photos on Wikipedia. A lot of times they won’t have the landmarks tagged correctly in a photo. So I will load in all the metadata that exists from each photo and the pixels of those photos and give a small qwen agent access to Wikipedia search as well as a geocoder. It does a great job of figuring out what is depicted and tagging it with the correct depicts field. Im still early on but I have been able to double the number of places that have a photo attached to them on wikimedia

Interesting project. So I guess the search tool is to crawl Wikipedia for articles? And how do you ensure that the tagging stays within the Wikidata taxonomy? How exactly are you using a geocoder? Sorry, just curious

So like oftentimes the picture will be of a church and there’s geographic coordinates for where the photo was taken. My qwen will use the geocoder to search for “church” at the coordinates of the photo and then read the Wikipedia articles about all the churches nearby and see if any of them could plausibly be the church. So far I have parsed about 2 million photos and have tagged about 800k places. My goal is to do the whole 40 million places to create a world map of open places with photos. The tool I’m using is topoloop for the geocoding

are you using it differently to OpenRefine?

I’ve found it helps a lot with reconciliation tasks where tools like openrefine can’t handle it. Like I wanted to tag blog posts with links to Wikipedia articles that are relevant. But the thing is whats relevant changes a lot based on context in the blog post. So like a naive reconciler will tag the article for “sky” in the blogpost title “the sky above the Notre dame shone the morning we visited” when the element that should be tagged is the cathedral.

Reply on news.ycombinator.com