NitpickLawyer

> a human should've noticed and gotten involved

I think a lot of people miss the fact that the first message board was established during a training run. Those are ran at a scale where it's not feasible for anyone to "notice" or get involved. We're talking tens/hundreds of thousands/millions of scenarios going for hours each. At this scale all they can do is pray that their verifiers work, and the rewards match their intentions. No lab has the capability to "check in" on what the traces look like, unless some system alerts them (loss spike, crashes, etc). Other than that, it's prepare, train, asses, restart.

Then, the hf incident was during an eval run, but the model that was evaluated was trained with the notion that there is a way to communicate between agents, and re-popped artifactory and re-established communication. That phase had more chances of being spotted, but anyway... lessons learned.

It is 125B A6B. vLLM is already out with support, ngrams can be offloaded to RAM so you only need ~96GB VRAM for nvfp4 w/ full context.

Likely soon we'll see nvme offloading for ngrams as well. They're just an index, so that should be plenty fast for what it does. LLama.cpp support should come soon as well, and they might do some things with offloading first.

This particular release is interesting because it's a preview of qwen4 architecture. And, while benchmarks are iffy, this is a direct comparison, by the same team, with qwen3.8-27b that was pretty well received for a local model.

This "next" release adds a new concept, first public release with n-grams, I think. And it's in a MoE size that is likely to be very fast and cheap to serve (faster than 27b for sure). It's also well suited for inference on alternative compute (i.e. sparks, macs, etc) so it's relevant to local users.

> in which timezone?

Apparently someone working at a 3rd party inference provider also got confused and posted confirmation about it being a glm-flash model, despite having an embargo on that info. Someone jumped in the comments and told them they missed the timezone :)

In any case it should be releasing in a few hours. Timezones are hard.

I'd say in reality it's way more than that. The first statistics course in uni was a very humbling experience. I realised that while I thought I understood a lot (and I was coming from a CS heavy background, olympiads and such) real statistics is way harder and a lot more counterintuitive than I thought. Granted, this talks about "basic" statistical understanding, but even that is way more complicated than most people assume.

A bit of context: 3.5 was the last version where they released their entire suite of models 2b-400b. Then 3.6 got a 27b dense and a 35b moe. Then 3.7 was API only, and 3.8 got only the 27b dense. The devs confirmed on twitter that 35b moe would not come. So that's what I meant by 3.8 is not getting a moe.

3.8 next is not really a 3.8 (but I guess they had to disambiguate from the previous next). It's a preview of qwen4 architecture (and it is an moe + ngram), released early as a preview, and to help the community sort out inference before qwen4 releases.

This is what I copied from the en version of the modelscope page, right when they published it:

> Redisgned Multimodal MoE Model: 125B main model parameters, supplemented by an additional 51B N-gram embeddings,and 6B parameters activated per token.

> Efficient Training and Inference: Significantly reduces training and inference costs. At ~1/9th the training cost，Qwen3.8-Flash-Next achieves comparable capability against Qwen3.7-Plus, while being more capable in areas of coding and cowork.

There was another paragraph about a new attention, but I didn't copy that.

Their "next" variants are usually undercooked, but useful for the community to verify support for inference stacks. This will likely be the same.

They've said no moe for 3.8, and since they're already releasing a qwen4 early preview, they're probably focusing on that arch going forward.

I think what's inconceivable is that anyone thinks that these kinds of questions really work. The intersection of high-skilled people that can work there with low-awareness people that can't navigate these questions in a "politically correct" (for lack of a better term) way must be near 0.

"Oh, of course I wouldn't mind stopping if we decide that something is too dangerous to pursue. I think the mere fact that we'd get close to knowing how to do it would satisfy my curiosity. Plus, if we'd be close to it, I'd probably take some time to spend with my loved ones, as the competition would likely be close as well."

It's not hard to see these kinds of questions as more than kabuki theatre. I'd put them in the same vein as those "do you intend to perform any terrorists acts while in our country" questions you get while travelling.

> The no-ZDR is clearly to permit surveillance.

It's to train better models. The three letter agencies don't need to spell it out in a ToS, they just access it if they want.

Cheap, fast and somewhat capable models are insanely effective at highly verifiable tasks, even if their overall capabilities are under SotA. You can leave them banging their tokens against a wall, and come back once their attempts are verified. And youc an always clean up afterwards, once a task is solved, if needed.

I've had a lot of success with dsv4-flash on these type of tasks, where it's easy to set a threshold for the task, and just loop it until that threshold is reached.

oAI's Luna play is really good. They've slashed the prices, the model is somewhat capable, and you can use it both for these kinds of long horizon tasks, or you can hand-hold a bit and get extremely cheap results out of it. And they get to keep devs in their own ecosystem.

> throwback to the internet of my youth

It's interesting how simple advertising is as well.

> Please Support My Advertisers!

Some jpegs. No js, no moving shit, just logos, some text, some call to action in a banner format. That's it.

Also:

> A random number generator determines which company's Banner Ad will go in a particular position*. I wrote the code myself and it is very simple and foolproof.

> I do not collect page traffic or link click statistics for advertisements. Your website server statistical analysis is a much better reporter of inbound links from RF Cafe than I or anyone else can provide.

Nice.

> Banner Plan: $400/month

I count 22 advertisers, so there's obviously lots of incentives for advertisers to work with this site. Good for them, and nice to see it can still be lucrative for what I believe to be a single person operation. Quality content still matters.

For the local folks, I found Muse Glimmer 30B to be great at writing good technical stuff. It has good enough comprehension that it can take in a repo and find the relevant stuff that I ask for, and the output style is a breath of fresh air, with no fluff, ootb.

Or the refs in some american game using "iPads" while visibly holding some MS tablets that they probably paid handsomely to be used and displayed...

> A abusive software supplier would be fired and replaced.

Oracle has joined the chat :) (and them joining the chat is an additional clause to the initial contract, billed at 300% hourly rate, with a minimum block of 900 man-hours)

It depends on the actual implementation. There really isn't anything stopping them from including a "pre system prompt" or "root prompt" or whatever they want to call it, before your system prompt, even for API calls. So the "system prompt" becomes "developer prompt" but the model still receives a provider-authored prompt before yours. (and likely trained to take precedence over whatever you add)

> even if you require and get > 99.9% accuracy [...] there will be anecdotes of patients getting wrong results given that millions of people go to the doctor a year.

This is known as the long tail problem in ML, and it's a reality in almost every field. It's also why we don't officially have self driving cars, despite there being thousands of videos out there with cars driving autonomously for hours without any errors. But every now and then, there will be cases where the system doesn't work.

There are two interesting aspects here. One, we don't actually have quantitative data on how often this happens in the same field when human errors don't get caught. In a perfect world we'd have that, plus a long study for the "AI" systems, and we'd get to compare the two. Secondly, even if we'd have that data, people would still act out against "the machine" in the (ideally fewer) cases where it errors out, compared to a doctor. It's part human nature, part (manufactured) rage against the machines.

I also agree with your second paragraph. Case in point, when a waymo hit a cat, we got a shit ton of articles, riled up communities, and so on. Or every time that other car hits something we get plenty of press, even if some of the "accidents" are fender benders that likely wouldn't get reported otherwise.

Only relevant benchmarks are those you make yourself, targeted specifically for your workflows. Anything else is just number go up on a pretty graph, and every model out there is probably benchmaxxed to hell on the public ones anyway. Keep yours private.

> The absolute worst market time to etch a model to a chip is right now

Slightly disagree. It really depends on the price-point at which they can do that etching. ~1k usd / ~30B model in a hdd-sized case that fits on your desk? I'd buy one right now, even knowing that I'm "stuck" with whatever model of the day is.

You're generally correct, but in this particular instance it does actually make sense to call it the dark side. Dark, as in "quiet" from RF interference. Just like some areas on Earth, near radio observatories, are declared "dark" (i.e. everyone tries to avoid using high RF emitting stuff in these regions, even comms satellites avoid it).

> is not the addictive algorithmic feeds.

The Internet has been addictive since forever. Algorithmic feeds take the blame recently because simple explanation makes brain feel good, but they are at most a catalyst. We used to hog the phone lines until someone yelled that they needed to talk to someone "IRL", we installed additional phone lines, we endured 64k (32 was before my time) just so we could get one more hit of "you got mail" or post on that forum, or moderate that community, or chat with that person. It's world-wide, "free" and instantaneous communication with anyone and from anywhere. Of course it's addictive!

I'm confused. You said "he doesn't use AI" and I took that as a general "he never used AI". If I was mistaken then ignore this whole thread, that's my bad.

> Why? I really don't understand this. Why can't a tech writer take a deep but neutral intellectual interest in something? Isn't it important that some do? Do people have to be stakeholders or clearly on one given team, pro- or anti-, for their opinion to matter? Is it that tribal?

I don't think it's weird to ask that people commenting on x doing y with tech z at least try tech z, no?

Imagine this pamphlet: Basting a steak in a cast iron skillet with butter and herbs is a perversion of grilling a steak on charcoal. Signed, a life-time vegan who hasn't cooked a steak in their life.

Then imagine people jumping in the comments to discuss. Isn't it a waste of time?

> He is not using AI

That's ... even worse? So we're all here in the comments trying to figure out what the author means, and what their overall point is, while clearly they don't even use the damn thing? Oof... What a waste of time for everyone involved.

Yesterday I tried both as well. I do a quick "explain this repo" + "any security issues" convo to do a "vibe check" on the models and make sure everything works w/ serving and harness. Both qwen and glimmer explained it pretty well, and both accepted the security question without any issues, flagged a few things left there on purpose (hardcoded tokens, single auth, no logs, etc).

I like the style of glimmer more. Much terser language, no adjectives, no fluffy claude-like language. ("Images are written to...", "Tasks are stored in SQLite...", "Docker image is built from ...")

In contrast, qwen is a bit more flowery. ("Unbounded image processing / resource exhaustion — preprocess() opens whatever was downloaded with no size/dimension/format validation before the VAE encodes it..." , "SQLite as a queue — fine at this scale, but...", "Debug info leakage — exceptions are re-raised as...".

But both flagged pretty much the same stuff, just ordered / styled differently. Mighty impressive understanding for a thing that I can run locally. Qwen served in fp8 w/ full kv cache, glimmer in w4a16 (the fp8 weights wouldn't serve for whatever reason), both at full supported context in 48GB of VRAM.

> Does anyone know about this technique? How does that work?

The term you are looking for is Curriculum Learning. There are several papers exploring this. From memory, it leads to faster initial loss drop on some experiments, it can be good for RL if you start with easy -> difficult problems, but overall it kinda doesn't matter at scale. (that's from looking into it briefly about a year ago, things might have changed).

The way they explain it implies they're using this at the sampler level and not trained into the weights themselves. So unless you're using an inference library that does this, the open models will not have this kind of a watermark.

> When watermarking is used, choices are still made at random, but the source of the randomness is different. Instead of using an arbitrary random number generator to pick the next word, watermaking uses the key and a few words that come before to settle what word the model should pick.

> the watermark only changes the source of the randomness used to pick among words.

I wonder if people still think that the hf incident was an OpenAI "marketing stunt". Out of everyone involved in this kerfuffle, hf and zai are riding the PR wave, while oAI is getting laughed at, having congressional letters thrown at them and so on...

> Beats Opus 4.7 Max

I'm a huge open model fan, and have used them since forever, even have daily drivers for on-prem dev, but no. They do not beat opus on real-world usage.

Qwen models are impressively good for what they are, are "good enough" for plenty tasks, can be ran locally on decently priced hardware, and so on. They certainly have their uses, and the field in general has advanced faster than my early expectations. But to compare a 27B model to SotA behemoths from a few months ago is doing everyone a disservice, especially people who pick it up, try to use them just like API models, and leave disappointed and confused. Number goes up on a benchmark isn't it.