kristjansson

Delaware law is not so much favorable as it is well-trodden. Of all places here we should understand the value of the network effect.

> The key mistake people made was thinking in terms of their own workflows and own local stacks

We need a dunning-kruger for empathy: people least able to imagine situations not their own trying hardest to influence others.

The user cares about the distribution of outputs. That distribution is structurally determined by the distribution of inputs (i.e. prompts), the weights of the model, and (these days) the dynamics of the harness guiding successive generations.

The only way to characterize whether a choice is 'right' is to characterize the output distribution (i.e. evals)! Changing the underlying weights necessarily invalidates whatever characterization may have been done. One may assert that one's harness regularizes outputs back toward the desirable distribution, or one may hope the different weights induce a sufficiently similar output distribution.

But no, one should not be completely agnostic to the choice of weights just because there's some nondeterminism.

Well, time to set down the glass beads and dive into a an alpine lake.

With MTP and FP4 I max out at 30ish t/s on mine. Without MTP or in regimes where the drafter performs poorly it’s about 10 t/s. FP8 is about half that

They made the coding plan a bit better toward the end, but it was pretty tough to use throughout.

Seems like an Amdahl’s law of inference economics? there’s so much compute relative to SRAM on the chip and shoreline bandwidth onto the chip that caching buys ~nothing? The contended resource is SRAM and a given token of context needs just as much as another.

> one TLD does not operate at the most fundamental level like the others;

contrary, this is one TLD _ceasing_ its idiosyncratic mode of operation.

Of course there's some trim that has to go into the grinder, but we could do a much better, more respectful job of using the animal as it was instead of homogenized lumps of 80/20 or 90/10. More beef cheek in the butcher case and homemade cabeza, more lengua, more shank and round and plate, more heart and tripe and liver, etc etc etc. Buy a subprimal and grind your burgers at home from a single animal.

This is more a culinary and ethical argument than efficiency or safety. One should use the whole animal, in all its varieties, because they're delicious. Grinding otherwise edible parts of an animal killed on one's behalf because one disfavors (or is repulsed by) them seems to me much worse than the killing. Eat - and enjoy! - all of it or don't kill it.

I don't know if you're a bit out of the loop, but ... meat is very much not at issue in this case.

> grinder

There's your problem. Ground products are completely dispensable.

> LLMs could

This is going to end up like the Law of Headlines, isn't it? "Do x, y, z Cure All That Ails You?" ... no but we got you to read the article. "LLMs _could_ x, y, z" ... but they don't because they're programs, not magic.

It seems impossible for this to be a true statement. and yet.

What’s next? $20m series B in ByBy, a horizontally integrated trash disposal, concrete shoe manufacture, lye bath, and pig farm operation?

The operative part of schoolwork is … not the work part? Of course school work is bullshit, in the sense that the teacher already knows what 7x9 is or who fought at Waterloo or whatever. The point is not the work product, it’s that the student produced it and has been incrementally improved by the experience!

hf download maintains each downloaded version in a commit-hash-addressed snapshots dir, with symlinks to the underlying files content-addressed in a blobs dir. It’s not quite git, but it is a stable reference path.

Just quantizing takes seconds-to-minutes, llama.cpp provides a nice tool[0]. Improving quality is then a matter of picking specific tensors to maintain at higher accuracy, checking on representative data, and repeating.

[0]: https://github.com/ggml-org/llama.cpp/blob/master/tools/quan...

I'm generally happy with the tests I ask it for, some of which are PBT. It's just the insistence upon memorializing every single change with a test. Maybe encouraging / forcing PBT will dissuade it?

And the tests. Oh god the tests. Personal recent favorite: I asked for some changes to a Dockerfile, which it did ably, and then promptly tested by writing a pytest module that traversed up to the root, read the Dockerfile, and checked that the added lines were present.

A big differentiator is whether one imagines an LLM in-band with most/all future software. If there is, and we’re deferring until very late parts of a program that world have been load bearing, and we’re able to programmatically ands reliably squint and say “eh i know what you meant” … then yeah formalizations seem superfluous-to-counterproductive.

OTOH if LLMs are to write, but not supplant, much of software, then boundaries, delegation to deterministic layers, good compilers to bonk miscreant models on the head with error message seem essential.

At one point it would have been shocking to assert that the compiler would live in-band with the program too. and yet JS eats the world. It seems shocking today that we could have a universal prior over the world operating in the ms/us nJ/pJ range required. And yet … ?

the norms of issue trackers are strongly opposed to “bump”. these autoclose bots may yet change that

harnesss == thing that calls LLM API, acts on response, and maybe does that again.

> Deepseek v4 pro 0813

Which itself released yesterday? You're writing, reading, and evaluating enough software in a ~36 hour period to form, reject, and form another opinion about which model makes better _architectural_ choices?

Oh shoot it had a nice countdown timer to ~8a PT when I submitted it

How they have any credibility after they told everyone they could run Deepseek R1 on their laptop (by giving the Qwen 7B distill the `deepseek` moniker)...

> atem

also perhaps taking some small joy from the lexical similarity to aten[0] namespace that lies at the heart of pytorch

[0]: https://github.com/pytorch/pytorch/blob/main/aten/src/README...

including Anthropic[0] and OAI[1]

[0]: https://platform.claude.com/docs/en/agents-and-tools/tool-us...

[1]: https://developers.openai.com/api/docs/guides/tools-programm...

As a relatively recent herdr user, I’ll say (1) congratulations and (2) the thing that was most attractive was the seeming orthogonality of the tool. It sits apart, interfaces with subordinate programs in a few clearly delineated ways, and integrates them only as much as the user desires. Critically, “no integration just show me the terminal output of my program” is an option, and a useful one!

I appreciate that discipline and respect for your user in the current tool, and hope to be a paying customer of a similarly values-aligned business you’ll build on top of it!

Also:

your messages causing your LLM (harness) to run CLIs on your computer? charming, thrilling, great fun.

other people’s messages causing your LLM to run CLIs on your (cloud) computer? terrifying, awful, sickening, no fun at all

Apropos of nothing: https://en.wikipedia.org/wiki/Marcus_Licinius_Crassus

> An orange frisbee is harder to find in grass then a red one

same for ungulates. hence, tigers

seems like they missed a zero somewhere. its a dollar of usage for a penny and change.