omneity

Two phenomena at play, correct spellings tend to be the most common on aggregate in a large enough dataset so there’s a bias, and the finetuning step (Instruct SFT) helps the model hone down on what it should use from the set of all possible formulations it saw in pretraining.

This is why LLMs can still channel typos or non/standard writing when you ask them to write in such a style for example.

For the anecdote, David Byrne’s “Like Humans Do” was included by default in Windows XP’s Media Player. I kinda like that song, but many people I knew hated it or were quite indifferent.

CUDA is equally proprietary and not an industry standard though, unless you were thinking of Vulcan/OpenCL which doesn’t bring much in this situation.

So how long until software in cars is treated with the same seriousness and rigor as the software in airplanes?

There aren’t many but they do exist. OLMo for example.

Oh thanks! It was a bug in handling browser light mode. I just fixed it.

Thanks! Let me know if you give it a shot and I’ll be happy to help you with anything.

This is basically our approach with Herd[0]. We operate agents that develop, test and heal trails[1, 2], which are packaged browser automations that do not require browser use LLMs to run and therefore are much cheaper and reliable. Trail automations are then abstracted as a REST API and MCP[3] which can be used either as simple functions called from your code, or by your own agent, or any combination of such.

You can build your own trails, publish them on our registry, compose them ... You can also run them in a distributed fashion over several Herd clients where we take care of the signaling and communication but you simply call functions. The CLI and npm & python packages [4, 5] might be interesting as well.

Note: The automation stack is entirely home-grown to enable distributed orchestration, and doesn't rely on puppeteer nor playwright but the browser automation API[6] is relatively similar to ease adoption. We also don't use the Chrome Devtools Protocol and therefore have a different tradeoff footprint.

0: https://herd.garden

1: https://herd.garden/trails

2: https://herd.garden/docs/trails-automations

3: https://herd.garden/docs/reference-mcp-server

4: https://www.npmjs.com/package/@monitoro/herd

5: https://pypi.org/project/monitoro-herd/

6: https://herd.garden/docs/reference-page

On this very topic, this might be one of my most favorite videos ever single-handedly changing my understanding of holograms: https://www.youtube.com/watch?v=EmKQsSDlaa4

I think it’s an expectation issue. AI does make juniors better _at junior tasks_. They now have a pair programmer who can explain difficult concepts, co-ideate and brainstorm, help sift through documentation faster and identify problems more easily.

The illusion everybody is tripping on is to think AI can make juniors better at senior tasks.

My gripe with an approach like this is the lack of any grounding to these generated topics. Hallucination accumulates like error in this case so every generation that is conditioned by a previous one (the recursive "hierarchical topic exploration" in TFA).

I suspect most of the "leafs" are unusable.

Thanks! Personally I found FFT is not necessarily a strict improvement over (Q)LoRA as it can sometimes more easily lead to instability in the model, hence the bit of extra scrutiny.

Curious to see your thoughts and results whenever you get something out.

Congrats for launching and awesome work, I will try it out soon!

Question, I have my own persistence and auth services, all well documented, that I would like to build apps against. Is there a recommended way to do this other than pasting the docs in the prompt each time I create a new app?

I explored revolt with a group of friends earlier this year, along several other solutions such as Matrix Element, Telegram and the new TeamSpeak.

Neither Revolt nor others are unfortunately at the right level of maturity to be adopted seriously. The team is doing a great job, but it’s still extremely basic.

Discord with all its warts is still the best way to have group calls in a casual setting.

Interesting, do you have benchmarks on FFT vs QLoRA for RL?

Perhaps less about DSPy, and rather about this: https://github.com/OpenPipe/ART

For anyone else curious, I found this [0]. Not a full answer but a starting point.

0: https://en.wikipedia.org/wiki/Lp_space#The_p-norm_in_finite_...

This is also pretty subjective. I’m a power user of both and tend to prefer Claude’s UI about 70-80% of the time.

I often would use Claude to do a “make it pretty” pass after implementation with GPT-5. I find Claude’s spatial and visual understanding when dealing with frontend to be better.

I am sure others will have the exact opposite experience.

What does `n` correspond to here? And why is it “ours”? (although the second I understand as euclidean space corresponds to n=2 and we seem to live in a locally euclidean space)

Surely it’s not dimensions, since all of these examples were two-dimensional (x and y). So I’m a little lost here.

You are assuming the only threats can come from outside.

Defense in depth is essential in an age of unreliable software supply chain.

Here’s a quick sanity check before you embark on the real thing:

- Tokenize your entire corpus with a few off-the-shelf multilingual tokenizers like Llama, Qwen and Gemma and calculate the ratio of letters to tokens. The higher the better, ideally in the 3-5 range

- Manually produce or select sentences that are similar in meaning but not in writing (embedding models also leverage graphemic overlap not just semantic similarity), and then check if similar sentences show consistently higher cosine similarity than dissimilar sentences. This is for embedding models like XLM RoBERTa not for LLMs, but it has a similar insight potential.

If both of these tests are promising then you likely don’t need custom implementations for these.

Personally if I was you, I would just take Qwen 0.6B Base (not instruct, since you want text completion) and continue pretraining it on the data that you have. It is very likely to work decently out of the box.

Yeah it’s a pretty crazy decision to be honest. Flashbacks to MongoDB and ElasticSearch’s early days.

Fortunately it’s an easy fix. Just front it with nginx or caddy and expect a bearer token (that would be your api key)

The first thing that comes to mind when reading “custom tokenizer” and “slice off the embedding layers” is that this sounds very much like pre-training from scratch, for which 2GB is far from enough.

Assuming you do get the data though, for a model at the sizes you’re evaluating you’re looking at weeks on a Colab A100-40GB most likely.

My recommendation would be to approach this with a smaller model and with a different training method that doesn’t involve a new tokenizer or new embedding layers because that’s what’s causing the cost x time to balloon beyond feasibility.

Why would you go through openrouter in this case?

Strongly seconding Roo Code. I am using it in VSCodium and it's the perfect partner for a fully local coding workflow (nearly 100% open-source too so no vendor is going to pry it from my hand, "ever").

Qwen Coder 30B is my main driver in this configuration and in my experience is quite capable. It runs at 80 tok/s on my M3 Max and I'm able to use it for about 30-50% of my coding tasks, the most menial ones. I am exploring ways to RL its approach to coding so it fits my style a bit more and it's a very exciting prospect whenever I manage to figure it out.

The missing link is autocomplete since Roo only solves the agent part. Continue.dev does a decent job at that but you really want to pair it with a high performance, large context model (so it fits multiple code sections + your recent changes + context about the repo and gives fast suggestions) and that doesn't seem feasible or enjoyable yet in a fully local setup.

Qwen3 32B is a dense model, it uses all its parameters all the time. GPT OSS 20B is a sparse MoE model. This means it only uses a fraction (3.6B) at a time. It’s a tradeoff that makes it faster to run than a dense 20B model and much smarter than a 3.6B one.

In practice the fairest comparison would be to a dense ~8B model. Qwen Coder 30B A3B is a good sparse comparison point as well.

Consumer GPUs in theory and by a large margin (10 5090s will eat an H100 lunch with 6 times the bandwidth, 3x VRAM and a relatively similar compute ratio), but your bottleneck is the interconnect and that is intentionally crippled to avoid beowulf GPU clusters eating into their datacenter market.

Last consumer GPU with NVLink was the RTX 3090. Even the workstation-grade GPUs lost it.

https://forums.developer.nvidia.com/t/rtx-a6000-ada-no-more-...

It takes ~17-20GB on Q4 depending on context length & settings (running it as we speak)

~30GB in Q8 sure, but it's a minimal gain for double the VRAM usage.

Conversations are always "reprocessed from scratch" on every message you send. LLMs are practically stateless and the conversation is the state, as in nothing is kept in memory between two turns.

Would running vLLM in docker work for you, or do you have other requirements?