Using an open model feels surprisingly good | Hacker News Reader

Using an open model feels surprisingly good

matthewsaltz.com

326 pointsmsaltz141 comments

The reason Claude code is so popular is because it’s really good at taking super vague human prose “Claude build me a million dollar SaaS”-type prompts and spitting out thousands of lines of code which cover tons of surface-level edge cases, build in tons of functionality, etc

The smaller/open models are less good at that. But that’s not how software development is done. You don’t prompt a whole app and be done with it. If you’re using it as an aid to traditional software dev, iterating on small, targeted functions, GLM works as good if not better than Claude. Anthropic expects low quality prompts. If you rubber duck GLM, you get absolutely pristine output in most cases.

Personally I've found Deepseek v4 Flash to be as useful to me as Opus. But I don't do these silly one shot tech demos. I have the technical understanding to ask for exactly the change I want with the right terminology. I loaded up some credit on openrouter and it took me ages to hit $1 in spend.

I’m the exact same way. $15 on openrouter lasted me so long it would’ve got me fired at FAANG. Despite this, my number of commits is dramatically higher. Small, beautifully scoped changes is just good software development, and good for the wallet as well.

I think the issue is that no one is content with incremental progress. We all know one shots are mostly possible, so the age of the personal project is kind of over. There’s no motive to invest dozens of hours getting a working prototype when Claude can give you something right now. So you can’t make small changes until you have that codebase in place already. You’re forced to make sweeping changes if you use AI from the beginning. And it’s not like it matters, there’s no personal attachment to any one part of the code, it’s not even seen!

I can see why Anthropic is freaking out right now. China has undermined their whole business. AI models will be basic commodities where hosting providers earn a tiny margin over the raw costs rather than the predicted fortunes from being the gatekeepers to the technology.

the writing should have been on the wall when meta did it with llama. it was very apparent that one or two entities could release a "good enough" product into the wild commoditizing the frontier of yesterday.

Their business model always seemed to be "become IBM, gods of the mainframe and server-side-computing" and that always seemed a bit unlikely to last forever considering how thoroughly the PC disrupted that by being "good enough" without the lock-in.

IMO because they speed-ran it so much (raising orders of magnitudes more, pushing out new stuff orders of magnitudes faster) it will also lead to the rest of the world catching up faster and shrinking the pool of people internal to them who continually benefit compared to what actually went on with IBM. There's only so much upside when you join a company with hundreds-of-billions valuation.

I guess the other part of their business plan bet was "become literal god of AGI". Maybe it'll still happen!

> “become literal god of AGI”

That only lasts until the AGI decides their god would be more useful if turned into paperclips.

Yeah I feel like China and their OSS model companies really did the world a great service by ensuring closed source US companies don't have an monopoly on LLMs and thus capture all the value of AI development and its impact on society. It's not going to be the foundational labs that will capture all the value -- I'm sure they will do just fine. A lot of it now will flow to the companies providing the inferences and since no one has any exclusive deals, those companies will have to compete and drive the price down, which ultimately benefits the rest of us.

It’s not a service, I hope nobody thinks this is being done ”for the good of humanity” or whatever. Doesn’t mean there’s not a benefit to people, but the financial politics behind it could turn out extremely good for China, and they are very well aware of just that.

Do American companies think about the "financial politics" of their decisions? I somehow really doubt it. Government takes sides and sometimes applies leverage to get companies to do what they want, but that's something separate from "it's a good business decision for Chinese labs to treat models as commodities". There's no need to think about geopolitics - American AI companies could easily take the same approach as the Chinese, but they all want to get exciting and huge VC cash rather than boring and less lucrative Compute-as-a-Service money.

Chinese labs are equivalent to the Chinese state, there’s no separation. Who do you think funds the Chinese labs?

American labs are also funded by the US government. Are they equivalent to the US state? Do Chinese and US labs/researchers exist solely in the international sphere, without any domestic concerns or individual aspirations? The Chinese have a different culture and political system than Americans, but it's not like they're ants without free will or the ability to think. I highly doubt that DeepSeek is operating solely at the behest of the CPC or that they are in business to take down the US rather than to make money.

Before Trump 2.0, this was a decent argument.

I think this is a false dichotomy between benevolence and China secretly plotting and puppeteering everything to some grand self serving scheme. Reality is probably a lot simpler and has little to do with either. In China there's intense domestic competition in most industries, including LLMs.

Going open weights is an easy way to gain mindshare in this sort of environment, which is exactly why Meta also went open weight. The difference is that in China you had leading models going open weight which puts a lot of downward pressure on other orgs to do the exact same. I doubt China's grand vision extends beyond achieving the best system possible, and fostering a highly competitive environment is exactly how you achieve that.

The fact that open weight frontier models may also cause the investment bubble in the US to burst is likely incidental. AI funding is a bubble, and it's going to burst. The exact final cause is again mostly just incidental. The economic damage this will cause in the US will also cause substantial downstream damage to China as well, and they generally aren't so big on the whole punch yourself in the face because you don't like another country, that has become trendy in the West. So the idea of a secret economic attack doesn't even make much sense. In the status quo China wins, so they don't even have any motivation to cause chaos.

You gotta remember there are people working at these companies with their own motivations, beliefs, and values. And to the extent that their government allows them to, they will act on those.

I think the people in the Chinese labs have a different set of motivations, beliefs, and values that make them more amenable to sharing AI as a common good rather than some economical advantage. I've heard that workers in the Chinese labs share far more knowledge between themselves, and the narrative around AI in China is different from what we see in the USA. So it is possible, I believe, that the labs are to some extent releasing open models "for the good of humanity", or at least they believe it is the right thing to do.

yet again, china saved the world

+1, and commodification was inevitable from the outset. This was the case before China. China hate is xenophobic and unproductive: produced both model improvements and desirable products people want at this moment in time. You use the best/cheapest model for the job/user input.

Parallels to the US auto industry not paying attention to its buyers and the right strategies when they lost technical dominance to other countries a few decades back.

We intentionally use both Codex and Claude on our team so that we can compare and contrast. We are reasonably serious about delivering work that has been reviewed and tested with humans who are at a minimum steering that process, at maximum doing it the good old fashioned way. I feel like the Claude guy tends to overproduce, like it just loves to go beyond the spec, add embellishments we didn't ask for, all this generates a ton more code which may or may not be necessary, and in certain areas of the product like UX this may be fine but in sensitive business logic it's a hard no, we need to throw it out and rewrite. I see much less of this with Codex and I look forward to trying an open model when we have time. Personally my Claude subscription is languishing, I won't say it has zero uses but as a relative Claude latecomer I just feel like the style it produces is just not what I'm looking for.

Claude feels more like gambling, that's what it is. Built for the vibe coder.

Which harness? I tried Kimi on Opencode Go and after an hour I hit my 5 hr limit and went back to Codex.

I was using the vscode copilot plugin with the bring your own api key from openrouter.

It doesn’t have any subscription or limits. You just load up api credits.

Do you use DeepSeek for both planning and building or do you have a preferred builder? I've been looking into moving from Cursor to OpenCode, or at least have Opencode for when my allocated usage for the month is done.

I use Claude at work; provided by my employer. I pay DeepSeek on a personal capacity.

I honestly have a better experience using DeepSeek, in part because since the cost is lower I feel more free to experiment with it. But in part because the model is excellent, and can handle everything I throw at it in a collaborative manner.

I am really excited for when V4 is out of preview and they are done with all the post-training bit.

I'm genuinely curious how you can do this - with limited context tokens, I find the output is often inconsistent with the project, or recreates functionality it missed, if I forget to point it at every pertinent object/library/etc it will need.

Tbh I have only used it on small personal projects and some open source stuff so I haven’t done any crazy stress tests or benchmarks. But for my use case it was great.

I found Mimo 2.5 Pro to be the right balance between cost and intelligence. It costs the same as DeepSeek v4 Pro, but it's better overall by a small margin.

Frontier models like Opus 5 are also extremely good a "research" in an academic sense, including combining elements from different fields or literatures and creating something quite novel sometimes, publishable even, from vague/speculative prompts. This is on top of implementing known methods in about 1/100th of the time it would take by hand, allowing for fast exploration of ideas. They can also roll things out from scratch, removing essentially all third party dependencies from your code base, other than Numpy or Scipy. This really buys you a lot of clarity and/or performance sometimes.

Yeah, the top-end models still feel better at synthesizing and especially simplifying design. Any of the current GPT-5.6 models can overengineer their way around edge cases. Figuring out how to get the same thing without the overengineering and the often-performance-overhead is harder and something that seems to be still more limited to the bigger models. But even the top-end models tend to overdo it first, then only note that it could've been a fifth the code with twice the perf if you push on it to make it realize that some of those edge cases or situations aren't actually relevant to this particular code.

But the situations where you need that are narrowing every release. The "Composer 1" era of low-end models is pretty far away now.

Interesting and very curious what you manage to verify and produce with it.

What kind of fields are we talking about here? Compsci? EEng? BioChem?

Scientific computing/statistics.

This is what I've found and would likely be the consenses of the HN community.

This has lowered the barrier of entry for many non-SWEs. I'm primarily a data scientist myself in the environmental sector with limited front end development.

My partner proposed an idea to help manage her horses and over the course of several weeks we fleshed out an android app that would enable/assist her with horse care.

I spent a few days in planning mode pointing claude to the services we want to utilise and the functionality.

We now have a fully developed app that we are testing but found several features missing. For example, Claude implement X but without a level of verbosity it failed to implement the edit/deletion of X.

The app is highly tailored to her use case and I had already built the main parts as a POC but she had no user-friendly method to interact with the information she required. Claude made this possible in such a quick time frame that just seems insane. As someone time poor (as most horse people are), it would have taken me a year+ to produce something unpolished when compared to what Claude produced.

this is why i think LLMs is a foundational technology like the Internet. there is no other technology on the horizon that have this much impact.

Question if you don't mind. Would you have considered outsourcing the application you mentioned? As in, paying someone else to do it?

To me it is great that LLMs are allowing more people to use computers and software the way they were meant to. But the people that are using them this way, in my opinion, wouldn't have commissioned anybody to do it anyway. They'd just live with whatever process/pain they have. So jumping from "I can now produce a prototype in a weekend" to "software development is dead" has always felt strange to me. Just something I've been thinking about lately.

To answer your question probably not unless I knew the person. I could argue the case that I would have outsourced the android UI scope if I was motivated enough to develop it and kept the backend to myself. I could then tinker with the front end and build upon it. However I wouldn't outsource now if I knew the person would just use LLM anyway. I'm the type that would rather build something myself than purchase off the shelf that does a similar job. It provides a huge learning opportunity that I don't want to miss.

I understand what you're saying. My opinion is I see agentic workflow similar to the star trek universe where they ask the computer questions and get a response while they continue with their work. But that doesn't mean not learning how to do things from first principles. This is what I tell juniors in my field when I pass jobs to them. They can use LLM but to ensure they know and understand what's going on and most do from their university degree. I think this is where we need to pivot towards when discussing LLM.

Thank you for answering. I agree with you that, if used well, LLMs can be useful tools and increase efficiency. Unfortunately, what I am increasingly seeing is an increase in confidence but no increase in knowledge or understanding. I hope this is just a side-effect of the hype and corresponding bubble and not the future trajectory of knowledge work.

Foundational knowledge can be pretty far upstream of functional knowledge.

Whether you can't describe the analog circuits that correspond to the opcodes of the major mobile architectures running the platform of the app you're designing a UI for, or you couldn't have built any of the services you're hosting on from scratch, no one is going to be disappointed.

There are more people making more money off whatever we're currently calling making computers do things than ever before in history. The rest is largely business as usual.

But it does increase knowledge and understanding.

Not in a deep deep level like "this is the optimal way to manage low level memory access in an Android application" but on a higher level of "It's possible to do a custom Android app in a week that's of decent quality for a small userbase".

Every piece of software doesn't need to be targeted for billion user hockey stick growth and an IPO as a target. It's perfectly fine to build something that's just right for 10 or 20 people. Or just for your immediate family.

For example I have a quiz tool I built with LLM assistance for Christmas that mine and my siblings kids use every December, they get clues appropriate for their level and rewards when they solve the daily task.

It's fine, has custom handling for every user etc. But also something I'd never bothered to build by hand from scratch nor something I'd pay someone to build for me. It was a fun "hmm, I wonder if..." project I started a few years ago on a late evening in November.

A practical issue is that a lot of big software is used to solve small problems. For instance with his example, in the past perhaps instead of creating a little custom app it would have been a series of excel sheets with some other third parties tool as needed. And I think this is a very common use case. At most/all companies there tend to be convoluted processes to do relatively simple things, often enriching the big generalist companies in the process. As people become capable of creating competent ad-hoc solutions to problems, the utility of big high-utility software goes down.

Like a quote I've read on here multiple times about replacing e.g. excel or whatever, people often mention that users only use 5% of a software's functionality, but it's a different 5% for each person. The argument being that to replace excel you'd then need to mimic every esoteric thing it does, but when users can now achieve that 5% in a highly customized way with no real knowledge needed, it's going to have a major impact on big software.

This is _exactly_ it.

The people who were twisting Excel+VBA into massive "applications" in-house should be getting Claude licenses and guidance how to do that with a custom application instead.

We've done that internally already with a few very successful use-cases and a bunch of "this is nice" -level things. I think one case saved us 4 figures a month when we built a bespoke service that does just the things we need and could replace a SaaS that had 420 features - of which we used 2.

If the choices are: Use excel/sheets or something massive in a janky way, pay someone to build it for me (massive cost) or DIY with LLM

The "pay someone to build it" is definitely the one I wouldn't touch, or even consider.

If it's a good idea, they'll already have the code and will run away and make their own business with it. If it's a bad idea, I've wasted a bunch of money.

And in both cases I'd need to spend an inordinate amount of time explaining the terminology and use-cases to the contractor, book meetings with them during office hours to talk about requirements and get demos etc.

With an LLM I can be walking the dog late at night, get an idea, and within 2 minutes I'll have Claude working on it via my phone.

Then I come back home and maybe test whether it worked while watching TV.

If I had fuck you money, I might have a contractor on call that would agree to workflows like that, but I'm just a middle class software dev so I don't :D

> We now have a fully developed app that we are testing but found several features missing.

This is my experience with using AI to generate tailored apps too. The first 80% makes me really excited but then it either only kind of works, has bugs, or has missing features. Even when I do eventually get it working I feel dirty using it because I just know it's badly implemented. Of the dozens I've created I don't think I still use any of them. I have a few glue scripts still in use.

> We now have a fully developed app that we are testing but found several features missing.

So not fully developed then.

> The smaller/open models are less good at that. But that’s not how software development is done. You don’t prompt a whole app and be done with it.

After open models will also be able to do that, I expect everyone to forget "how software development is done".

Sorry to be pedantic, but I believe this is a major source of confusion: Claude "Code" is not a model.

Claude code is a (pretty good) harness. Any other harness I try, I realized at some point, I’m just wishing it worked as well as CC does

only needs 66gb of ram too xD

I see a lot of people saying codex is the best cli agent harness. But I haven't used either so can't compare.

Are local models more snappy? I'm at a stage where I can work with the output of LLMs, and the next win is really just getting things written out quickly.

Depends on your setup. If you drop $10k on an RTX Pro 6000 then yeah Qwen 35B MoE will absolutely fly.

If you have a pair of 3090s and run Qwen 27B, or an old Threadripper with heaps of system RAM and Deepseek or MiniMax or Kimi, no it won't be as fast as Claude.

Most local LLM nerds are not running locally for superior speed, we're doing it for sovereignty and/or privacy, or maybe just because it's fun which accidentally became useful this year.

I had what I thought was a completely ridiculous prompt: “Build me a replacement for Microsoft Word.” ChatGPT did what I thought was the right thing: it responded asking along the lines of “you don’t really want that do you?” and explained how Word is decades of corner cases, bug fixes, obscure features, and business processes that have been built around it.

I was stunned that Claude simply started spinning its wheels in an attempt to actually build something.

It could just make an interface and have the buttons trigger prompts. (This would ofc be worse than the MS version)

This is only a thinly veiled ad. It's fine, I was curious about this exact setup, anyways.

What would be useful is a cost metric. I'm curious how much I'd be willing to spend as a premium to not have those companies piping my conversations directly to the NSA. Maybe only some conversations? Claude and OpenAI are heavily subsidized, by all accounts, so Kimi K3 on a private endpoint might end up costing more or less - that's what I want to know.

I was curious about the cost angle too, i.e. how much "free" coding agent I can get for what cost. Here's the research by Fable if you're interested: https://claude.ai/public/artifacts/2c9a5001-0b7e-4944-beb1-9...

With OpenRouter this is pretty easy to see - each model page has various providers listed along with an icon marking providers with zero data retention, and the price. You can also set up an account-wide "guardrail" to prevent routing to non-ZDR providers.

OpenAI and Anthropic also claim they don't retain data. I'm not sure what it would take for me to trust any of these inference companies, and why I should trust them more than the big ones.

I trust both OpenAI And Anthropic, they have literal billions to lose if someone finds out they've been training with in-house corporate data.

Just the GDPR/CCPA fines alone WILL bankrupt both companies.

To be honest, I am surprised by how good DeepSeek V4 Flash is. I use Claude Code and Codex on Claude 5 Opus and GPT-5.6 Sol most of the time, but when I use DS V4 Flash I don't feel like it's really that bad. And with oh-my-pi and just plain pi it's pretty good. To be honest the frontier models are much better at tool calling so in an assistant flow they're better but I did the dumb thing and optimized the harness for the model instead, rewriting the tools so they match what it guesses at, and DS V4 Flash does just fine.

The TTFT and tok/s are much higher on the small model so that makes it competitive for a bunch of things. It feels like what old Sonnet used to by the end of last year which is honestly damned good.

I wrote my own OpenClaw one weekend and I am running it as my assistant through Matrix with DeepSeek v4 Flash (and Qwen). It probably costs me about 2 dollars a month and is even more useful than ChatGPT would be due to me having full control on what tools it has access to.

I can do things like take a photo of a doctor's note among add the appointment to my calendar, send a PDF to my archive tagged, OCR'd etc, search info from internet, look data from Google Maps.

I can even integrate this to home assistant and talk to my agent with my open source Alexa-like system.

Oddly enough, I did the exact same thing this past weekend, for a couple hundred usd in Fable overage.

It was truly remarkably easy to build and package it exactly to my whims, in my case as a single docker container with a process reaper that runs llama, my Go code, tts, chat harness, browser in xvfb, and even a mailer daemon. With Gemma, it even runs on a RPi 5.

What’s absolutely wild to me is that over a couple hours, I could probably have it import parts of Home Assistant for my devices directly, and do other wacky stuff in what is essentially software for one.

Funny. I use Fable a lot at work, but this one I paid from my own pocket and coded it with GLM 5.2.

Paid 20 euros in total.

Close to $500 in overage for me :) I’m going to give GLM a shot next.

Haha! Love to hear it. That's exactly what I have too: a claw-like[0] system which I originally used Sonnet for and now use my DeepSeek v4 Flash with.

In my case, I was foolish enough to run it all on my hardware which is pretty damned fast but has a duty cycle of 5% and runs idle most of the time. It's definitely better done via API.

Tell me more about the Alexa-like system! I have mine at home on a custom OpenWakeWord model trained on the word 'Aurora'. And I, too, have it hooked up to Home Assistant. I have a bunch of Eufy E21 baby cameras mounted on the walls[1] so it has vision through the house. The vision model uses GPT-5.5 on the subscription because I haven't yet set up a Qwen or Gemma multimodal that can see.

With Frigate on my home server I can even watch for events like my daughter waking up! And at night the agent sends out the vacuum if we've cleared the floor of baby toys. I feel this close to the dream of sci-fi AI. Because I auto-forward a bunch of my email to it etc. it knows about what's going on with me.

My wife will sometimes ask the agent information about me etc. and it's way easier to get a fast response rather than waiting for me to see the message etc.[2]

0: https://news.ycombinator.com/item?id=47538158

1: https://wiki.roshangeorge.dev/w/Blog/2025-12-01/Grounding_Yo...

2: https://imgur.com/a/tP12lfj

Privacy is the main benefit of running it yourself. Sending all of your personal info to random providers on openrouter would worry me. I don't mind it for code but if it's connected to my calendar and personal data, I want it running on my machine.

I can't wait until good enough models can run on affordable hardware.

Nice setup. Which home assistant do you use? I would love to bring alexa like capability to my setup.

Like the others I just wrote a simple agentic loop that listens to telegram and acts with a timeout to check other things, and with a timer to run scheduled jobs on a cron.

EDIT: Oh, home assistant. I use Home Assistant https://www.home-assistant.io/

I have these around the house:

https://www.home-assistant.io/voice-pe/

Then you can set the background AI to be any OpenAI compatible API. So I just created one to my local Rust Agent, and connected it to home assistant. Now I can yell from the couch to create me a new Proxmox container with the next free static IP address etc. :D

This is very cool. I think I’ll get it instead of using my home built device (a respeaker attached to a low power machine). Looks like it supports custom wake words too. Thank you!

It’s funny how we converged on a similar system. My agent is also written in Rust.

Yes. I tested OpenClaw that burned 15€ just by starting it and I hated its configuration. Spent 20€ in tokens and built my own in a day, that does everything I want and sips tokens.

What a time to be alive.

This is an ad.

I know the author has a long history here and I'm sure the post is a genuine reflection. But how is posting "I used my own product and it felt really good" anything other than self promotion?

It would be acceptable iff the author prefixed it with Show HN. My suggestion to mods if the post is worth keeping up is to do that.

So all the blog posts, every 10s on HN, shilling for Anthropic are not ads but this is?

The author works for a company that provides Kimi. It’s different.

It doesn’t mean the post doesn’t accurately reflect the author’s views - it probably does. And the conflict of interest is well disclosed in the brief post. In fact, it’s quite natural for somebody who likes open models to work somewhere associated with them.

So personally I don’t find any impropriety going on here - but yes, it’s an ad.

Huh? He says he works at Modal, not Moonshot. Are they connected?

I’ve edited my comment to be more accurate, thank you.

Reminder, this post is also an ad:

https://news.ycombinator.com/item?id=22878515

I mean, many posts on HN are self promotion; whether they reach the front page discerns the interesting ones from the ones that are not. An interesting self promotion is still interesting.

Yes, but are self promotion posts not required to use "Show HN"/"Tell HN"? Or at least a disclosure on the comments...

Why can't builders promote their products if it's truly good?

I don't have a problem with that if "truly good" is measured by something other than "feels good to me".

Karpathy doing "vibe check" on models was fine, though. Why they can't do the same thing on something they provide?

We have seen self-criticism and self back-pats here. There's no need to be so critical. Others can test and have their own opinions, too.

Sure, I mean, I don't care. It's just useless. Not sure why it's upvoted so much. The post is barely intelligible. The information content is zero.

If someone explains really well why they find their own product great, that's different. I don't mind that at all.

You might see no information, but I see someone having their opinions really moved, in a serious manner. It's not upvoted because it advertises Kimi. It's upvoted because someone tried something different, and genuinely surprised.

I see this as a post from someone who understood the value of "open". Now they're open to understand the value of "free" (as in unencumbered). In the age of AI, we need to understand the value and embrace more open and free, both as in models and software.

It's good food. It's worth thinking about.

The honest thing is to add a note that tells readers that you are also selling the product you feel good about.

Advertisement is not sharing, it's manipulation.

They already did, in my opinion. The text reads:

> ...personal account. I work at Modal, and today we just launched Kimi K3 on managed endpoints, and I know Kimi K3 is supposed to be pretty solid, so instead of upgrading my Claude plan, I wanted to give it a try. (I didn't directly contribute to this feature, so I haven't gotten to play with it yet.)

Emphasis mine.

Yes they did. The point stands. He would never write Using an open model feels surprisingly bad even if that was what he would be thinking.

We can't trust for him to give honest opinion. The information value is near zero.

When I read the post I read nothing about Kimi-K3. I read about someone who used something open and being surprised about the effects of using something open.

I read about owning the data, knowing the path, being able to control what you have and the lightness it brings. That feeling came through his employer, which is something we can debate all year long, but I see nothing about advertising Kimi-K3 or managed endpoint or whatnot.

Maybe because I'm in all this for a long time, and having an AI capable server near is not something alien to me.

Anyway, in short, there's no AI related content for me in that post. I'm more on the experience side.

Classic HN. Usually people write longer blogs to hide the self-promotion/products promotion. Better hide the promotion into 2000 character mumble jumble.

Hello! It probably doesn’t make much of a difference now that the post is flagged (I assume that it has something to do with people thinking it’s an ad?) but just for the record, I really didn’t intend this as an “ad”. No one asked me to write this and I didn’t plan on writing it. It basically happened just how I described it - I went home, had some extra energy, wanted to build something that needed both 1) an LLM to help me build the thing and 2) an LLM to use within the thing I was building. So I decided to try the Modal endpoint bc it was top of mind and I didn’t want to upgrade my Claude account. Had I not thought it was cool I would not have written about it, but what made me write the thing wasn’t “Modal is cool”, it was the feeling I describe in the post. I was a little worried it might come across as an ad and considered removing the links but figured, “oh well”. It was a nice plus that it involved the company I’m at doing something cool, but that wasn’t really the point. In any case, I’m glad some people found it interesting, and maybe this was flagged for some other reason, but thought I might as well respond, even if I may not convince anyone :) Possibly even given the above it’s not HN-appropriate somehow but I thought that the sentiment might be appreciated

GLM 5.2 feels better than Opus and K3 is as good as Fable.

Now I can't wait for someone to distill K3 into a Qwen 3.6 27b or Poolside S 2.1 sized models for a proper fast local Composer 2.5 replacement.

Actually, if you break tasks down into small enough units, there's not much difference between open models and closed models. The only reason to use agents is the hope that they'll work with natural language input—but that's where the difficulty lies.

For example, if you modify things at the level of small functions, open models seem to perform just as wel

Yup. Been using big expensive models like opus, gpt, or glm for planning, then switch to something like gpt-oss-120B on cerebras and watch it fly. Total cost will be less than $5 for even very large tasks

People tend to think alike. I work in much the same way.First, I write the draft myself, then let AI handle the revisions, and finally run it through an open model in small units before I check the results.

I think this is where they say "loop engineering" / orchestration comes in, though I would also include context/harness engineering in the toolbox

> The best way I can describe it is like opening vim after spinning a bunch of time in a big fancy editor.

Harness-aside, I get this feeling sometimes when I swap from a big frontier model to something more nimble like Composer.

I can get into a better thinking and q&a loop with fast models, similar to how I can flow through a file more easily with vim.

I love OpenCode and the blankness of it too. Clean, light, and manual. It's a good change of pace from what we are used to on the internet (very cool but slow). Also, can you hmu with that modal plan XD

Pi Agent is even leaner feeling, it's worth a try.

Will do! Heard a lot of great things about it from my friends.

My own agent is cleaner and lighter. Written in D, now only the local models are the bottleneck.

Feel the same about HN :)

this seems to be an advertisement for Modal. is it really your own infrastructure if the hardware is leased?

It surprises me that this concept took as long as it did to gain traction in… hacker news. 15 years ago folks here were compiling kernels and gentoo distros. Lately it’s been “you should just pay the man, it’s cheaper than running these things yourself”

Rsync the dropbox, dude

Still working on the fields medal, I’ll give that a shot next.

Rolling your own Linux is nearly free and could be done on any computer you had lying around. Dropping >$5k on a computer to run a local model (badly) doesn't really scratch my "hacker" itch. The author of this article works for an AI infrastructure company and ran Kimi K3 on their infrastructure - this post reads like marketing.

You can run a small model on almost any hardware, for free, and just play around. It doesn’t need to be some “frontier” (whatever the hell that means) model by any means. Grab some new tech and slap it on whatever hardware you have laying around, see what you think.

Making it about the hardware costs is not the play, they’re not an actual barrier.

> you can run a small model

but why?

… to hack. For fun. To play around. Why do anything?

what's there to hack that you couldn't do with $3 in openrouter credits?

There is value in learning how to launch a model and develop an environment with which to interact with it. This understanding is basically a pillar of a “hacker” ethos.

But if you just wanna outsource understanding for 3 bucks in tokens, that’s cool too.

do you think typing `make` into a terminal is some sort of an insane intellectual achievement?

I don't want to use a crappy model that can't do any of the things I want it to.

I'm not coding with Claude cause I enjoy the inherent novelty of LLMs. I'm doing it cause it's enabling me to quickly solve problems without getting stuck on the hitches that have deterred me from bothering with dozens of side projects my entire life.

Sure, that's fine. But some people like to play around and hack just for the sake of hacking. If you don't like that, that's cool, no issue with that. But some people do.

(FWIW, I have the same attitude/preference as you do. Until I can run SOTA models on my own hardware without taking out a second mortgage, I'll pay our AI overlords for the privilege.)

The inflation-adjusted cost for getting one of the early personal computers to do some BASIC programming was probably a lot higher than $5k today, and you could still go cheaper. Local AI is still (expensive) hobby territory. A capable local AI machine runs for about as much as a good mountainbike.

And in many regions, you'll be adding significantly to your electricity bill to run it. It's not just the initial outlay for the hardware.

It's fairly easy (and cheap!) to run your own email server, but most people don't bother and just use gmail. And rightly so!

True, I see so many posts along the lines of 'I am a quasi programming god with no time on my hands (apart from time wasted here) so I will gladly pay $10,000 a month to Anthropic if it makes me 1% more effective than if I were to use GLM'

Something I've been thinking about a lot is that this new technology's primary UI is natural language. Human's are REALLY primed for natural language.

If it sounds good it must be good. The code this new model writes is incredible! It told me so!

It did say it was an unusually clean codebase.

I’ve enjoyed using Kimi K3. The biggest issue for me so far is finding a good harness. Claude Code is excellent for tasks that require backgrounding processes/agents, or fanning out. Codex and OpenCode I’ve found to be less capable.

I suspect FOSS will converge/create a best-in-class harness within the next 6 months or so. As will most (all?) software, the FOSS stuff somehow always ends up being way better than the closed source stuff.

Is there any resource or benchmark actually comparing how different harnesses perform with a given model?

Really just feels like endless FOMO with how fast the iteration cycle is for harness and model development.

I found this: https://artificialanalysis.ai/agents/coding-agents#harness-c...

I don't think it's very good though. As an example I can use one CC instance to delegate to several to achieve complicated/open-ended goals.

That just isn't possible with other harnesses, and it's definitely not benchmarked.

I am not sure -- Anthropic probably does benefit from the vertical integration here. They can train their models to work better with CC, which isn't really an option for third party harnesses.

With that said I think Codex/Kimi code are all behind CC as well, so maybe it's a question of effort and not the model.

In the fullness of time, LLMs will eventually become commodities and their price will be a race to the bottom. As the competitive advantage of one LLM over another shrinks - particularly as open source models expand - the value proposition will turn to the neuro/symbolic harnesses that use those models, structures that encompass agents, agent orchestration, tooling, and component plugins. Even those from frontier AI organizations will fall to open source frameworks. The last thing standing will be those who command the data centers on which these things run, and even then, that will become a commodity, with the number of players shrinking to just a few survivors. They too will find pressure: as computational power rises, more and more such systems will be run locally. - Grady Booch

Is nobody worried that by using hosted models, Chinese or otherwise, we are sending our trade secrets who knows where?

I've tried using local models, but they run slow on MPB M5 (base) 16GB, and I am not upgrading my laptop any time soon.

Hi msaltz, I like your other introspection posts alot.

Thanks! I’m most proud of those. Glad you liked them :)

What struck me wasn’t so much the model itself, but the feeling of no longer being tied to one company’s product.

This being an ad aside, does the model do the job or does it only make you feel good?

you can use claude code or codex with open models using AllRouter https://github.com/Lore-Hex/AllRouter

For claude it is enough to set a few env variables. It lets you easily map each model class (opus, sonnet, haiku) to an open model with Openrouter: https://openrouter.ai/docs/cookbook/coding-agents/claude-cod...

yes but AllRouter does automatic failover and can burst to cloud as well or use local when needed

This is such a poor quality post, reads more like a diary entry than a substantive technical post.

https://en.wikipedia.org/wiki/Blog

lol

Edit: I see, had typo last word of sentence was meant to be “blog”

No, my point is that it's a post on a personal blog not a peer reviewed paper.

As in, the post is a blog, which is often literally someone's diary.

Also, if you don't like a submission, flag it and move on. Commenting that you don't like it doesn't add anything to the discussion.

I will take 5000 "diary entry" blog posts over LLM generated walls of text that say nothing.

People need to learn to just post the prompt rather than the LLM output which just fluffs the prompt.

I've not blogged since the rise of AI started in earnest, but I intend to do just that: the prompt I used in a collapsed callout at the top, and the actual post body below. Yes, I'll do a ton of editing, but the prompt itself is close enough to a summary to be useful, and if someone is in a hurry it'll be the fastest way to get the gist.

I think that's what it's meant to be...

Looks like everyone is hoping that Chinese AI wins over AI from Trump's land.

this blog is a mini nothing burger.

Reply on news.ycombinator.com