jwr

Hmm, perhaps I should switch to an MLX version… problem is, it took quite a bit of work to get llama-server (with llama.cpp) to serve my model(s) and allow requests in non-thinking (default) and thinking modes.

But 30-40 tokens/s would make a big difference.

> Eventually I used Gemini 3.5 Flash for pairwise evaluation

But, but… wouldn't that be… (gasp) DISTILLATION?

Fun project!

These are very good!

I'm hoping for speed improvements because the only problem running the 27B model on my Macbook pro (M4 Max) is the speed: 20 tokens per second. I benchmarked and MTP actually makes things slower, so I disabled MTP altogether. I'm hoping there will be some breakthroughs or optimizations that will allow me to run this at 30-50 tokens per second, which would make a big difference.

I would suggest careful benchmarking. I actually tested and benchmarked, and the new Qwen3.8-27B model is actually slower with MTP on my M4 Max. MTP only gains anything when generating long code sequences, which is very unlikely as the model spends most of its time thinking, not generating code, even if you use it for coding (which I don't).

I get 20 tokens/s on an M4 Max (larger GPU).

The EU has been using a single mandatory tire-label system that has an "efficiency" category, among others, so that you can see the tradeoffs, since 2021.

But of course, the EU is over-regulated, let's laugh at the EU :-)

Freedom -- that's what it is. There are now more and more cases where we don't have to ask for permission.

Unfortunately, as I look around, I see most of the hardware vendors around me as gatekeepers. They prevent me from writing custom code in various ways. Apple restricts access to NFC or their UWB positioning, Supernote Manta does not let me access a bluetooth microphone, the list goes on. Denon at least has an API in their amps/receivers: buggy, but it's there.

> "gives site owners actionable performance data they would not otherwise have"

As I understand, the next step will be to inject ads, to give CloudFlare actionable monetization strategies they would not otherwise have.

Here we go. First, get everyone on the internet to use a free service (nameservers and CDN caching, which BTW is mostly unnecessary for pretty much everyone out there). Then, inject analytics, to "give site owners actionable performance data they would not otherwise have". Then, inject ads.

In the meantime, also implement "bot protection" and get everyone on the internet to outsource the decision on who can access their website. With no appeal.

I'm increasingly worried about Cloudflare.

They are certainly much better than the competition. That said, do not believe the marketing hype. If you want to accept various kinds of payments, you will eventually run into the stupid limitations of the payment system, and Stripe APIs will not abstract this for you in spite of what they advertise.

For example, redirect-based payment methods (iDEAL, Bancontact, Sofort) are a huge source of complexity, and handling all the additional states a payment can be in is far from obvious. Stripe mostly just drops them in your lap and says "deal with it".

Yes, there definitely is vendor lock-in. I've been watching Stripe billing fees being raised slowly over the years, and thinking about exactly this. It is very difficult to migrate to a different subscription system.

This is a typical comment I get from people who don't use E-bikes. Also from people from the US, who get sold POWERFUL E-bikes that are not pedal-assist type. You click a button and you ride, like on a scooter. That's not what I mean by E-bike.

Let's set aside how this statement does or does not make sense in a competitive environment. Look at this from a different angle: at the end of the day, would you rather have those hundreds/thousands of dollars, or not?

Because if I could "pass the fee to customer", I might as well (raise prices), and then still pocket the hundreds/thousands of dollars.

SaaS owner here (solo founder). I have been calculating this manually on a regular basis, as I've been wondering if it makes sense to migrate from Braintree to Stripe and what the cost difference would be. I finally moved, in spite of Stripe being slightly more expensive (but it's nearly a wash in my mix of currencies/accounts), mostly because of Braintree's catastrophically bad handling of 3D Secure (mandatory in the EU) and abysmally bad support.

However, people often say that Stripe's integrated solution outweighs the price (see other comments here). I disagree here. Sure, if you're just building an MVP, you want things to be up and running as quickly as possible, and you don't much care about processing fees. However, in the long term, if you run a sustainable SaaS over many years, these small amounts add up quickly. So, as an example, I do not use Stripe billing; I have my own subscription system. Because I looked at the amount I would be spending with them yearly, and realized that I prefer to have my own, building my own also gives me more control and flexibility. And independence, which matters quite a bit.

Biking does not have to equate to cardio! You can commute to work on an e-bike with your heart rate similar to walking quickly (what I do). And you can go exercise later.

Not sure how these two things got tied together in the common culture. Something I hear all the time is "but you have an electric bike, what's the point, it doesn't get you any exercise" and people saying that commute to work sitting in their cars.

Once you try a (good!) electric bike, you don't really want to go back. Life is just so much better. For urban commuting, you no longer think about heat (getting sweaty), distance, inclines, wind, etc.

Obviously I'm talking about good electric bikes, with mid-mounted engines that detect pedaling strength. These feel great, unlike electric motors strapped to bikes, which feel like, well, electric motors strapped to bikes.

FWIW, Qwen 3.8 27B is actually slower with MTP enabled, unless you generate a lot of code with it (and even then it's not obvious, because you'll likely spend most time thinking).

What would be really interesting is if I could connect a cable to two ports on a Mac and have the Mac test the cable.

I have an M4 Max (unfortunately 64GB). I have been running the Qwen 35B A3B one for a while now, after testing and benchmarking a number of models. That one was consistently the best in class for tasks like despamming, E-mail classification, OCR and dictation post-processing. It was also really fast (90 tokens/s).

I'm benchmarking the 3.8 model now, it seems it is better (near-perfect score on my E-mail spam filtering benchmark, best of any model I tested, ever). But it is slow.

One thing I would recommend is keeping an eye on MTP parameters. I tested and benchmarked extensively, and I use `--spec-draft-n-max 2` with llama.cpp. Longer sequences actually decrease overall performance.

As for running, I ended up using llama.cpp and its llama-server, with a bunch of scripts written by AI, because I got tired of LM Studio not implementing the image-related parameters which made gemma4 useless for OCR.

Me too. 35B A3B runs really fast on my MacBook Pro (M4 Max) and is suitable for real-time tasks like dictation post-processing. The dense model is not.

I've been doing a lot of benchmarking for a long time now with a number of local models for the purposes of spam filtering. The major observation is that there is a lot of variance in model performance. This should not be surprising, as these are probabilistic machines based on random numbers, so your performance will vary from run to run. But this also means that any sort of evaluation of benchmark with a sample size of 1 is essentially worthless for the purposes of model comparison.

In my benchmarks, I started insisting on having at least 5 runs.

This also makes me very suspicious of some of the posted benchmarks that compare models. If the results don't say how many benchmark runs were performed and what the variance was, there is really no basis for comparison. You are just guessing.

That seems like comparing apples to oranges. I switched to Mastodon in order to avoid the inevitable enshittification of every service that has anything to do with a corporate entity. I learned my lesson and I am not about to repeat it. And having spent considerable time on Mastodon now, I'm quite happy with the value I'm getting from it and the people I follow. I do not need everybody in the world to be on it. In fact, I actually prefer that many people not be on it.

I got used to doing this over many many years with the tcsh shell. I switched to Fish recently and I continue to be annoyed by its refusal to implement this feature. Yes, I tried using all the newfangled history-search thingamajigs. They all kind of work, and they all have their issues. But the !, together with in-place expansion, is something I still dearly miss.

This. Good stable foundations allow for quick software development. Even if you use AI for coding, working with Clojure as an experienced developer produces extremely good results quickly because of the great foundations.

I love the expression "the hammock time has already been spent": it is actually very accurate and I'm very grateful to the Clojure team and community for spending that time!

Caveat: this kind-of-works. There are plenty of subtle bugs around this. For example, it's hard to have a default model, and clients need to select the model explicitly in every API request, which makes it difficult to work with several models and multiple clients.

Does everything have to be "news"?

Removing spam from email, extracting attachments (invoices) from emails, and otherwise managing E-mail. Those are my use cases, although I suspect this model will be too small for that. I currently use 27-35B models (MoE mostly).

Now could you please sell this to all airlines to replace those terrible, terrible in-flight maps that they show?

> and I imagine Claude did the v4 rewrite

HN detectives, hard at work. I wish there was a way to automatically hide all the comments saying "but I think LLM was involved in this software". These days it seems like most of the content here on HN.

This is especially galling when we discuss software that has been around for >20 years and that people (actual users, not HN haters) universally love. See other comments around here.

I have been using SuperDuper for the last 20 years.

It works. It does its job. It doesn't pester me with popups. It didn't get bloated. It didn't get rewritten using Electron to support New Enterprise Features. It didn't get a built-in sync client for my pictures and videos so that I can share them with my team. It didn't get ads. It didn't get enshittified in any of the hundreds of possible ways that other apps did.

I find it ridiculous that all I paid for this experience was… $27.95 back in 2006 (I just checked the receipt). That's $1.40/year. I have no idea how Dave can make a living this way, and if he ever introduces subscription pricing, I'll be happy to subscribe (cue the HN hater crowd with their knee-jerk reaction "subscriptions BAD!" and downvotes).

If you don't like the enshittification of everything around you, support software that doesn't do that.

Thank you for the fantastic app, Dave. I will happily pay for the upgrade and will happily keep paying, because good apps are worth paying for!

The worrying thing here is that so many people accepted outsourcing the decision on who can see their website to a large company (Cloudflare). If the company decides that a certain user should not see the website, the user will not see the website, and no one will know about it, and the user will have no recourse. That is not the open web that I would like to see.

A second side effect of a knee-jerk reaction to bots crawling websites is that if you try to fight all bots, you also end up hurting real users that use "bots". If I run a local script or an LLM that needs to fetch a web page from your site and you block that script or LLM as being a bot, you hurt me, the user. I wanted the information and I have no way of getting at it without investing my own time and effort in going to your website. That might or might not be what you expected, but it's worth taking into account.

And finally, something worth noting is that there are so many websites whose owners complain about bots, but the real problem is that the website is poorly built and should be improved anyway. Bot traffic is not necessarily bad.