sroussey

No compelling results because summarization is really hard.

It is also super common on e-commerce websites that will then send you an email about a cart you started to the point of adding an email but then abandoned.

Same price as GPT-6.1-sol however.

GPT-6.1-sol costs less than half of Gemini and way less than Anthropic on the cost per task chart of the listed parent page.

You open the same convo on another device and the partially written text is there to continue. Does it not do that for you?

Classifiers (instead of LLMs) return results with confidence scores.

Name Entity Recognition (NER) is one example.

So many of them... https://huggingface.co/models?language=ner&sort=trending

Also used to block SSN and CC #s from logs, etc... as small and fast enough to do it. You don't want to call OpenAI GPT-6 and ask it to return your text with the SSN blanked out. I am sure people do though... (SSN is a bit simple, but all kinds of PPI in one model is more likely).

The nice thing about Jev is that people started taking about models that are not LLM text streams again.

I’ve had Claude opus do that too.

Um, maybe don't allow babies on planes.

That would make my flying a bit nicer.

This is a prime example of short term thinking, often espoused by politicians.

Why would you need multiple agents and not one agent with multiple repos?

There are AMD and Intel devices on similar process (not talking about A20Pro or M6 which are set to ship later this week), and they do not get the same gains.

And honestly, they have historically had different markets.

When the design is for only one customer, you don't need to generalize things, and those things you generalize to give different customers different options has costs.

AMD will soon be a larger customer for TSMC than Apple (NVIDIA is already there) so Apple's pre-booking new processes is likely to be gone in the near future.

HBM also trades bandwidth for latency, and your regular computing is much more sensitive to latency than bandwidth.

That would be like skipping land line phones for mobile...

what do you call bare metal in my office?

Curious about people’s experience here. I am working on a small model, verify by jev, and escalate to big model. Some cases, the small model is not a model but some regex.

cheap-confirm-escalate

Using jev as the confirm step.

For reference: https://huggingface.co/blog/webgpu-kernels

I think it will be the basis for a rewrite of transformers.js v5, but no need for you to wait as you would likely want direct access. It is also way better than loading WASM, and faster to boot!

Would love to see this implemented with @huggingface/kernels for shader compilation for Webgpu.

Those machines with GPUs still need RAM of their own, and they generally want large caches to avoid SSD penalties. You even see this spill out in the form of costs for KV cache in <1min, 5m, 1hr rates etc.

Someone really needed a few hundred TB to waste on inference and went looking under the rugs…

Having worked in hardware for a moment, everything we do in software is like this. Even C.

Maybe these big ai labs will uses their own devices to find and fix bugs up and down their stack and contribute that back.

Yeah, isn’t that Claude Codes sandbox? That drops and every npm install it taking over the world, lol.

Or AMD’s putting sram on a separate chip. I’m surprised they didn’t go there yet except for extra L3 instead of all of it. At some point the costs will shift the decisions.

Yes, exponential back off and jitter are the first things to work on, and good if you don’t have a better signal (like loss of network).

Also, a simple signal status server or queue system helps to keep global state such that everyone doesn’t retry all at once.

If you have a central error rate server you can skip your retry based on the error rate (100% error rate, don’t retry, etc).

So many variables, but the simple thing is to set things up like normal rate limiting (which you would want to do anyways). The one generating the errors passes back a retry time. You can add jitter here, tell low priority requests to wait longer, etc.

BTW: do keep track of priority. It’s like having a database that gets flooded with connections and won’t allow new ones in—but will for admin users (btw, it did not used to be that way in the early days of MySQL).

Yes! And maybe get a hf fused webgpu runner for that model so it’s fast!

The hybrid nature of this thing is not a detriment, nor does it make it an LLM.

I dunno, I would consider Waymo and Tesla to have frontier models.

I think AlphaFold and related are also frontier models.

Being an LLM does not seem like the qualifier for frontier.

I have the same problem when i touch a result and it changes the result line at the exact moment.

And there was the Tesla thing CNAMEing time server pools and hiring people to pen test, which sent automated attack systems on volunteers servers. Last I heard, Tesla et al didn't even care enough to respond.