cmrdporcupine

This is a great post, thanks. I've been following prototype systems since I first logged into lambdamoo in 91, and Self is an old intellectual muse of mine but this is the first time I'd seen Korz.

You might be interested in my lang/environment "mica" which attacks this same dimension of problems from a relational/datalog angle and then makes object/prototype identity hopefully emergent out of that: https://github.com/timbran-project/mica -- relational dispatch feels like a broadening of multiple dispatch.

I feel like I may have ended up at a similar place as Korz without realizing it. I will need to dig into this paper.

Speaking also of multiple dispatch and prototypes, there's also this paper from an old acquaintance: https://www.cs.cmu.edu/~aldrich/courses/819/salzman-pmd.pdf which I used to find interesting (and built my first version of "mica" back then around the idea).

yeah, substantially agree. But I also think if only Atari and Commodore teams could have just been one company... it would have been glorious. Somehow. The market couldn't afford to be split like that, the pie wasn't big enough.

Also I was an Atari ST user back then, and loved it. But the Tramiel's didn't "get" software in the early days. They failed to make the right investments to keep TOS/GEM actively developed (e.g. as a Mac alternative) until it was too late (they hired Eric Smith to work on MiNT and then MultiTOS in the early 90s but they were already done as a company by then).

And then three fundamental hardware mistakes with the ST were:

1. not shipping it with the Blitter from the start so it couldn't compete on games, obviously with the Amiga but even with 8-bits of the time

2. forcing the ROM size down to 192kB so they had to size optimize the crap out of GEM before shipping (waste of precious development time), especially including tossing away all of GDOS (which had support for proportional fonts and printer drivers etc). A 256kB or 512kB ROM would have solved a lot of problems with what I'd think wouldn't have been an insane BOM cost bump.

3. Minor but long term annoying -- tying the cartridge port to only have 128kB of address lines. If they had given it 256kB or 512kB address ability their later expansion issues (OS upgrades etc) would have been much improved.

As for the Amiga, great machine but a) it cost too much at the start so failed to catch immediate fire (the 500 rectified this to some degree) b) interlace.

The alternative to "open weights" at this point is "American controlled."

And Dario and Sam have already made it clear that it's America First.

The rest of the world isn't going to accept a regulatory regime which imposes American hegemony. Maybe when Silicon Valley was playing all utopian like they used to. Not now.

Open weights is the most reasonable counter-power we have.

guess i'm old cuz i still feel like it's weird and antisocial

I mean, those of us with Marxian biases define it that way -- not right wingers -- and see "social democrats" in the 21st century as being the people who successively turned the term to mean instead anything-where-government-does-stuff. That's effectively what it came to mean in the US.

Notably, the definition given by Marx and other 19th century socialists (worker controlled production) is doesn't even have to involve government or the state at all. It just came to mean that once proximity to power came into play. The "soviet" word merely meant local worker councils, not a state form, for example. But once the Bolsheviks took state power, the power of the actual soviets was successively demolished in favour of a centralized one party state.

So, notably: Self-called "Marxist-Leninists" in the USSR, and Social Democrats (or "democratic socialists" in the US), and Right-wingers all far prefer the term to mean "government controls things." The latter because they can then point fingers and call things authoritarian (but somehow exempt corporate power from critique).

But it is not at all clear that 19th century socialists .. from Proudhon through Marx etc -- who coined the term -- meant this at all.

Also this hard line between "communism" and "socialism" is fictitious and very schematic. All communists call themselves socialists, and almost none of them agree on what that means.

Social democrats trying desperately to sound respectable clearly want a nice hard line there. But they also have no intention of bringing about socialism.

Update: It's markedly worse than 5.6 Sol. It costs far less money because it's far far stupider.

Update:

Sol 6 is a heaping pile of garbage. Just epic levels of slop. And r/codex etc is full of people noticing the same.

I've switched back to 5.6 Sol. What they're selling as Sol 6 is really what would have been Terra before, and it's awful.

More seriously this has been my fear all along.

When people spend their days interacting with machines that pretend to be human, they may then start treating real humans like machines.

Astra 6 was a huge improvement over Sol 5.6 for UI work. I haven't tried Sol 6 yet for it (it's only been a few minutes).

The GPT / Codex models have always been "overengineer" personalities. I prefer that to "I left a pile of race conditions lying around and big gaps in testing" though, which is what I was getting from Opus at times.

But yes both Astra and Sol veer on the side of paranoid. And honestly that's better for team work. For solo work where you just want to yeet something, it can be tiring.

You learn to tame the GPT "personality" on this front by combing over once a week and asking it to find and exterminate pointless tests, clean abstractions etc.

Yeah.

Astra was/is superior for planning type tasks. It was capable of doing seemingly magic things with rather vague/lazy instructions ("I need to be able to test this on Windows, maybe a qemu VM or something? Shrug." ... 1 hour later "yeah i built you a whole qemu + eval windows image + harness of powershell scripts + shell scripts to retrieve & verify harness.").

And for UI work -- which is not something I do a lot of but do here and there -- it was clearly superior to 5.6 Sol.

But it also feels sloppier? Somehow. And too expensive to use.

We'll see how Sol 6 is.

It wasn't just Astra. Sol 5.6 was a hog, too. They futzed with the formula and it pissed people off royal.

Are you trying to imply that nothing OpenAI can release would justify that response and therefore the people must be bots?

Asking cuz I don't think I'm a bot [pats self], I legitimately prefer the GPT models to Anthropic's, don't like Anthropic's customer service/reliability story at all, and I welcome a massive price reduction. Seems like something I should be happy to get.

If you'd told me I'd be typing this a year ago I'd be skeptical though.

Well, they do rug pull constantly. This week and last leading up to this the cost to use Codex was overwhelmingly perceived as terrible. People running out of usage all over the place. Reddit full of people crying. I noticed it myself.

Then they do a new model launch, issue quota resets all around, and it's a party for 2-3 weeks before things return to normal.

Looking at their own charts it seems like it's only small incremental improvement over 5.6 Sol, but with a massive cost reduction. And the better writing/communication style that Astra had.

Which... fine, I'll take that.

GPT Sol 6 has also released today, but no official blog announcement yet

https://www.reddit.com/r/codex/comments/1wnggya/gpt_6_droppe...

And here's the important part...

They seem to have tweaked things this morning maybe (in response to complaints?) and there's noises from Tibo about a reset coming today.

The open model launches and competition generally definitely seem to light a fire under their asses.

The more likely scenario is either OpenAI or Anthropic just go pay some highly inflated price to buy Jev. Mainly just for its people and the PR, not tech.

Which is likely what all the VC, hype machine, and overinflated claims are really about anyways.

The tech etc is easily replicated. The hype / name, not.

I seem to remember reading that the Jev-founder-guy is ex-OpenAI anyways. So that's how these things often roll.

How does the value of their plan compare to paying by token via e.g. openrouter? Worth it?

And how's the performance?

And can you choose the server or does it just use your geo to determine? e.g. can I as a Canadian pick the European infra?

Experiences so far with the non-flash model: a bit of an overthinker/overplanner. Makes bad/messy architecture choices with an open-ended prompt. But actually very competent at intricate code level things, and pretty good at finding corner cases, including in code that other more advanced models had looked at and not noticed problems with.

I deliberately left things wide open and ambiguous to see what it would do. I think with better upfront planning this would be excellent value.

Everything is combined and uneven, including the opinions of hackernews commenters?

There is no single opinion, and clearly no single Chinese approach.

Also Chinese labs are in particular very careful about anything which be used to create pornographic content, which is highly illegal in the PRC.

I'll just say that even though I was poor and without a job and living on unemployment insurance for a year...

The implosion of hype after the .com crash was actually kind of a ... relief.

Jev is only on people's mouths because they made friends with venture capitalists and used the publicity blowhorns that come with that.

Whereas the other guy went through the unglorious but formerly respectable path of publishing software and papers for other professionals to look at. A year ago.

We're in a bad place where the latter looks less reliable than the former.

(EDIT: I'm not saying the research here is in fact the same as what "Jev" is doing; and Jev is in fact more "product shaped." But I think it's important to temper the hype and back up and focus on the fact that this whole industry is built on research by both academics and enthusiasts ... first ... and gold rushes can often bulldoze over those people who are focused primarily on making-doing-researching instead of fundraising-hyping-promoting. That's not good.)

Ok I tried it. And it's faster. Though I'm still in verifying quality phase.

But only works for Gemma4, or other attention-only models. (i.e. not the Qwen3.8 models I had working with the other way)

I'm getting 63ms per answer for 1 question, 79 for 3, 93.7ms for 5. So scales nicely, too. ~55ms fixed cost + ~7-8ms per additional question.

Much better. Thanks. I'll commit and share work after testing it some more.

EDIT: how can I credit you in the commit body?

I think you're right. Better. I will have to think through if it would be faster or slower. If understand what you're getting at with this.. broken analogy...

What I described was -- we have a bunch of orders to the kitchen, all of which have the same "base" meal but different topics.

> Cook the "base" meal once, divide servings onto multiple plates, then add different toppings to each plate.

vs what you suggest:

> Put the base meal and every topping through the kitchen together, but use some kind of dividers so the toppings never mix up together.

Except.. ok, that analogy is confusing lol.

Or they'll just build a competitor in RISC-V instead and that's fine.

Except the problem is not restricted to the actual ISA or its HDL implementation, etc.

It's even just getting space / time in a fab at that advanced of a process node.

Only took them a year and a half of everyone complaining to finally do the right thing.

Congrats.

So the model normally (like during a normal decode) takes its final hidden vector and multiplies it by the entire vocabulary head -- so like about 250K rows -- to produce one logit per possible next token (and then so on and so on...)

Instead I use a fixed set of up to only 64 single-token labels. At model load, I gather only those 64 rows from the vocabulary head into a small matrix. Each question maps its permitted answers onto some of those labels.

It is a probability distribution conditional on the allowed labels. Calibration is a separate problem that I have to solve still and will be model specific :-) But I do seem to get reasonable answers right now.

So "64" is just in the end the endpoint’s maximum answer-label set. Most questions use only two or three of those rows. And, yeah, some calibration required. WIP on that

fwiw, w/ gemma4 -- non-diffusion -- I get about 170ms for a single question -> answer and then an additional ~33ms on adding more. While I see people reporting 300ms for this vLLM PR on same hardware (Spark.)

So I don't see the advantage to their approach until you're up beyond 6 or 7 questions?

Latest commits added gemma4 and instructions. I'll work on making a version of all of this that is standalone and not specific to DGX Spark.

> It seems by sharing KV cache all questions will be visible

Yeah, this is partially why in my approach I've done this instead, and not used diffusion model:

1. Convert the state into one shared prompt.

2. Run that shared prompt through the model once.

3. Fork the model’s internal state once per question.

4. Add a different question to each fork.

5. Ask each fork for its next-token scores.

6. Calculate only 64 possible label scores—not the whole vocabulary.

Basically ... skip decode.

Won't be as fast as doing diffusion model parallel across a pile of questions at once, but:

a) let's you use pretty much any existing text model (with some modifications). I've got qwen3.6 moe and qwen3.8 flash next running, am getting gemma4 working now

b) the problem you identified

It's possible I'm getting high on my own supply and misunderstand entirely the whole thing, but it seems to work?

https://github.com/rdaum/eider/commit/9c2d5c049068c33da2a48b...

I don't have the chutzpah to go creating PRs for vLLM to do the same.