cavisne

"knowing why a model refuses to answer something matters"

The companies that create these models cant answer that question! Models get jailbroken all the time to ignore alignment instructions. The robust refusal logic normally sits on top of the model, ie looking at the responses and flagging anything that they don't want to show to users.

The best tool we have for understanding if a model is refusing to answer a problem or actually doesn't know is mechanistic interp, which you only need the weights for.

This whole debate is weird, even with traditional open source code you cant tell the intent of a programmer, what sources they used to write that code etc.

Only one person in the groupchat needs to be using Telemessage, ie. a CIA agent can use a government device with Telemessage to talk to sources on Signal. Signal has a great protocol & robust clients, and getting caught with Signal on your phone is probably a bit better than being caught with CIAChat on your phone.

The actual implementation here is atrocious though.

The signal protocol is public, using their servers is frowned upon but its not a source code license violation.

From the paper it was a speedup on the XLA GPU kernel they wrote using Jax, which is probably not SOTA. I don't think Jax even has a official flash attention implementation.

"An H100 GPU has 989 TFLOPs of half-precision matrix multiply compute, and ~60 TFLOPs of “everything else”"

I always thought there was a lot of crossover between gaming GPU's & DC GPU's (and the volume is why NVIDIA is so far ahead). Are tensor cores somehow related to the pre tensorcore SM's (like an abstraction on top of SM's?)

MITRE has a trademark on the term CVE.

Hybrids are gas cars.

This is somewhat true but there is also a chasm where you earn good money but not enough to afford a mortgage. This chasm grows wider ever year in countries like Canada and Australia (and to a lesser degree America)

So now the finished products are tariff free, but the input costs when bought into America are tariffed. The exact opposite of what you would want to do to bring manufacturing back to America.

This is being sold as an air gapped product, it has to work offline by definition.

Sure you could hide some way of phoning home and deploy it into the SCIF, but would you really want to risk a firing squad to improve some advertising metrics?

Seems pretty high, this is an air gapped product so at some point the employees of whatever government they are giving it to would need to SSH into the VM's to load new weights etc. Lots of ways to make it tricky/watermark the weights though.

The raw computation is just a bunch of matrix multiplications in a row, most of the algorithmic complexity/secret stuff would be around scaling & efficiency.

For training the model the HW is much more important as you need to scale up to as many chips as possible without being bottlenecked by the network.

This would just be inference, and it doesn't need to be very efficient as its for on prem usage not selling API access. So you could strip out any efficiency secrets, and it would probably look like a bigger Gemma (their open source model).

I wonder if they would/could try and strip out stuff like whatever tricks they use for long context + video support (both of which they are a bit ahead of everyone else on).

Europe is mostly "old money" so this makes sense. They didn't earn that money they are custodians of it.

Factually despite all the noise there are no significant tariffs in either direction at this point in time, 98% of trade is tariff free under the free trade agreement. Mexico and to a lesser degree Canada are in a more favorable position than they were before the tariffs as they are the only countries in this position [1]

[1] https://semianalysis.com/2025/04/10/tariff-armageddon-gpu-lo...

In addition they use their government work to increase private sector profits, either implicitly "we work with your government" or explicitly (PWC offering briefings on tax policy they were helping with to the targets of the tax policy)

None of that onboard stuff ended up working afaik as the Tensor chips shipped right as giant LLM's became how dominant.

"In addition to the use of Signal, some DOGE staffers are bypassing other vetting processes and chains of custody for official government documents by working simultaneously out of Google Docs instead of circulating single copies of drafts, a source briefed by a government official said. “There's multiple people in one Google Doc editing things simultaneously,” the source said, referring to the online word processing software. That was partly how DOGE was working so quickly, the source added"

Outrageous!

You can do the same from the browser on any phone (including iphone) by going through AI Studio

https://aistudio.google.com/live

Was that gamble wrong? I thought all LLM training workloads do collectives that involve all nodes (all-gather, reduce-scatter).

These complex 4D chess strategies never really pan out. Maybe someone in the administration thinks that was the goal but not Trump.

During the campaign Trump said he wanted 10 percent tariffs on the world and 60 percent tariffs on China. After all the noise we are now at… 10 percent tariffs with the world and 125 percent tariffs on China (pending negotiation with China).

The JAX docs have a good explanation for how a TPU works

https://docs.jax.dev/en/latest/pallas/tpu/details.html#what-...

Its not really useful for other workloads (unless your workload looks like a bunch of matrix multiplications).

Nvidia has ~60% margins in their datacenter chips. So TPU's have quite a bit of headroom to save google money without being as good as Nvidia GPU's.

No one else has access to anything similar, Amazon is just starting to scale their Trainium chip.

"that kind of thing is going to come to America... and be automated". Pretty disingenuous cut on the quote there by the bluesky post.

The H100 theoretical flops number is just marketing, as it relies on sparsity that LLMs don’t use

Billboards being rare outside of the US seems quite incorrect. The developing world is full of billboards, and places in Europe like Milan have some wild Samsung billboards.

Yes this was the case for Venezuela, they started taking flights again when the El Salvador news came out.

Refusing to repatriate citizens was part of the argument the Trump admin used to justify use of the Alien Enemies act.

If this holds up in court the strategy going forward is pretty obvious. Any country that refuses to repatriate criminals will get hit with the AEA. This will be very politically painful for any country as they rely heavily on remittances from undocumented immigrants in the US, and will quickly fold.

Nearly 25% of El Salvador is in America, although they fled gang violence not Bukele.

In the presentation they said eventually kernels can share SIMT (PTX) and TileIR but not at launch. It seems pretty mysterious why they don't just emit PTX, I would guess they are either taking the opportunity to clean things up for ML tensorcore workloads or there is some HW specific features coming that they only want to enable through TileIR.

Scaling transformers has been basically alchemy, the breakthroughs aren’t from rigorous science they are from trying stuff and hoping you don’t waste millions of dollars in compute.

Maybe the company that just tells an AI to generate 100s of random scaling ideas, and tries them all is the one that will win. That company should probably be 100 percent committed to this approach also, no FLOPs spent on ghibli inference.

This article was prescient enough that I had to check in wayback machine. Very cool.