amitport

I've been considering something like this for a while. It only makes sense as a free, open-source, decentralized sharing protocol (where anyone can host theorems and no one can limit sharing them).

If a company were to manage to commercialize this, it would end public open research.

C# did not ship with async/await, and Java didn't have virtual threads back then. I am specifically referring to the initial choices made in C#'s foundation.

The mystery of why .NET got so many things right is simply that C# was built several years later by the exact same Microsoft engineers who had previously worked on extending Java, giving them a perfect blank slate to fix the architectural flaws they had already encountered

Second mover advantage.

Let’s look at the actual historical record, because the pattern is undeniable: the root cause of this conflict is the absolute refusal of the regional and local leadership to ever accept a sovereign Jewish state within any borders.

In 1937 and 1947, the Jewish community accepted partition, but the local leaders rejected it and chose war.

Following the Six-Day War in 1967, the Arab League shut down any path to compromise by issuing its 'Three No's': no peace, no recognition, and no negotiations.

Decades later, Israel engaged in direct talks at Madrid in 1991 and made massive concessions under the 1993 Oslo Accords to establish the Palestinian Authority, only for the process to be dismantled by waves of suicide bombings.

At Camp David in 2000 and the Taba Summit in 2001, Israel offered an independent state encompassing Gaza and over 90% of the West Bank, but their leadership walked away to launch the Second Intifada.

When the Arab League proposed a plan in 2002, they conditioned it on non-negotiable terms regarding refugees and borders that compromised Israeli security, while Palestinian factions rejected the deal entirely regardless.

In 2003, Israel accepted the international Roadmap for Peace, but the process stalled permanently because the local government failed to fulfill its basic obligation to dismantle terrorist infrastructure.

In 2005, in a one-sided prayer for peace, Israel completely dismantled every single settlement in Gaza, dragged out its own citizens by force, withdrew every soldier, and handed the entire territory over to Palestinian custody.

In 2008, Israel proposed its most sweeping offer yet, including near-total territorial withdrawal and international administration of holy sites, which Mahmoud Abbas flatly refused to sign without any counteroffer.

You cannot blame Israel for a lack of effort when it has repeatedly offered statehood and land for peace. While the West labels Hamas as an extremist group, the reality on the ground is that they are viewed by a vast number of Palestinians as a mainstream governance structure and a legitimate movement of armed resistance against occupation. The true obstacle to peace remains this foundational objective, which Hamas champions and the local political culture reinforces, to establish an Islamic state over the entirety of the land and eliminate Israel entirely. If at any point there is a Palestinian majority and a leadership that genuinely seeks peace, it will happen in a second, despite whatever imperfections Israel may have.

"parents and grandparents,"

You don't have to go into historical events. This is still happening now.

Jews are still fighting for their survival and the moment Israel stops fighting, millions of Jews will die.

To be fair, NPM sucked long before it got acquired by Github/Microsoft.

And to be fair 2: The other package repos also suck.

Hi, thanks! I appreciate your input and generally agree. The TDS article wasn't really aimed at the HN crowd, but it did help a bit with the more general audience.

I do plan to also develop an interactive guide that breaks down post-rotation quantization fundamentals in a more educational, hands-on way.

For context: https://towardsdatascience.com/how-a-2021-quantization-algor...

I recently wrote a beginner-friendly explanation of this situation in TDS:

https://towardsdatascience.com/how-a-2021-quantization-algor...

Hi, I’m the author of the post above.

tl;dr

TurboQuant is a recent paper from Google and NYU that has gained massive traction in mainstream media and the AI community. As implementations of TurboQuant are integrated into various popular projects, it is important to note its relation to EDEN quantization.

TurboQuant is essentially a partial implementation of EDEN quantization (first work published in NeurIPS 2021, extention published on ICML 2022). The few differences that do exist make EDEN significantly better.

We have also published a detailed comparative report here: https://arxiv.org/abs/2604.18555

In the vLLM documentation quoted above, TurboQuant (which is a restricted version of EDEN) is referred to as a specific case of HIGGS. Note the symmetry: EDEN acts as a special case of HIGGS; hence, HIGGS functions as a generalization of EDEN.

In any case, the quantizer is indeed an extension, regardless of whether it was explicitly framed that way in the paper. I say this not to diminish their contribution at all, but just to clarify the relationship, as it was also stated in the vLLM doc.

Thanks for the pushback, and I appreciate the reference to classical information theory.

While I probably overstated things by using the very general phrase "taking advantage," I want to be very precise about the claim, as I believe these works are foundational to quantization, beyond the scope of deep learning. The mechanism of applying a deterministic biased quantizer, such as Lloyd-Max, to the induced post-rotation distribution, alongside mathematically correcting its inherent bias, is a distinct contribution (which asymptotically improves the worst-case error).

If there is a classical paper that utilizes such a combination, I would genuinely be very eager to review it. But to my knowledge, this was not introduced prior to DRIVE and EDEN.

Thanks for that!

It is worth noting that taking advantage of the post-rotation distribution was not actually done until DRIVE (2021), which was made possible via our proper scaling. Furthermore, applying a Lloyd-Max codebook post-rotation was introduced EDEN.

We consider these to be the foundational works in this regard.

Thanks for that! Note that the residual chain is empirically and theoretically inferior to our unbiased scale; furthermore, it requires an additional bit in certain cases. Additionally, TurboQuant was not the first to apply EDEN to KV-cache (see for example https://arxiv.org/abs/2411.17525 from 2024).

Those works did cite DRIVE/EDEN :)

HIGGS is an extension of EDEN (using the well known method for blockwise Lloyd-Max).

The proper framing of this "TurboQuant" layer in vllm (which does not include JQL) is precisely EDEN 22 without the scale correction.

I believe our claim at this point is more fundamental than just lack of citation.

The quantizer in TurboQuant is EDEN quantization (2021) applied to the KV-cache. It is neither a novel quantizer nor an improvement in quantization techniques.

In DRIVE/EDEN, we already introduced the version used in "TurboQuant"'s paper and suggested an optimal scale configurations which are better in both mse-minimizing and unbiased scenarios.

Thanks for the quick response and for being willing to update the explainer. I really appreciate the clarification.

When you use TurboQuant, you are essentially using the EDEN quantizer under a different name applied to KV-cache.

Both EDEN and its 1-bit variant have been implemented in PyTorch, JAX, and TensorFlow across numerous open-source libraries and are used in various applications. I am currently writing a blog post that will document these in detail.

EDEN defines a scale parameter, S, for which we suggest specific optimal values for both biased and unbiased versions. As shown in the note I shared, these values lead to clear empirical improvements. Consequently, users who rely on the less optimal S value and the unbiasing method popularized by TurboQuant will generally see inferior results compared to those using EDEN with the optimal scale values suggested in our original papers.

The note includes extensive experiments and reproduces many of the figures from the TurboQuant paper in our Section 5. Honestly, I think our case is pretty clear-cut as is. I am not sure what the overhead for those specific benchmarks would be, but we will look into it.

(In any case, I want to emphasize that TurboQuant quantizer is a private case of EDEN)

TurboQuant is a restricted version of EDEN quantization (NeurIPS 21, ICML 22). It lacks the optimal scale derivations, which makes the TurboQuant variant considerably less accurate than those works. We show this thoroughly in a new note at https://arxiv.org/abs/2604.18555.

We were the first to introduce post-rotation distribution-aware quantization in 2021. This was later implemented in many fields, including federated learning, vector retrieval, databases, inference engines, and KV-cache.

It would be appropriate to receive credit for this. Furthermore, it is baffling to see the name "TurboQuant" repeated in this context, considering the many works published from 2021 onwards.

The blog post mentioned above essentially guides you through EDEN quantization but ultimately settles on a sub-optimal MSE-minimizing version and an unbiasing trick. This trick often costs a full bit more than DRIVE/EDEN requires to achieve the same results using the unbiasing scale shown in the original 2021 paper.

They must mean 'The Arab World Has Lost America,' don't they?

We do mention and the paper you shared. Please read our paper to see how the rotation-aware bias correction we introduced efficiently fixes the bias and provides a better worst-case error.

To be clear, I am not claiming they stole an idea. They have made significant independent research. However, a specific part regarding the treatment of rotation with bias correction relates to prior work, and it would be appropriate to have that recognized.

In this context, the rotation is for spreading energy and ensuring predictable coordinate distributions rather than diagonalization; it makes coordinate-wise quantization much more computationally efficient, though it throws away learnable structure.

Yes, the important part is that the normalized error does not increase with the dimension of the vector (which does happen when using biased quantizers)

It is expected that bigger vectors have proportionally bigger error, nothing can be done by the quantizer about that.

This is a great development for KV cache compression. I did notice a missing citation in the related works regarding the core mathematical mechanism, though. The foundational technique of applying a geometric rotation prior to extreme quantization, specifically for managing the high-dimensional geometry and enabling proper bias correction, was introduced in our NeurIPS 2021 paper, "DRIVE" (https://proceedings.neurips.cc/paper/2021/hash/0397758f8990c...). We used this exact rotational approach and a similar bias correction mechanism to achieve optimal distributed mean estimation. I also presented this work and subsequent papers in a private invited talk at Google shortly after publication. Given the strong theoretical overlap with the mechanisms in TurboQuant and PolarQuant, I hope to see this prior art acknowledged in the upcoming camera-ready versions.

r is a single value per vector. You don't have to quantize it, you can keep it and quantize the billion+ other coordinates of the vector.

His point, I believe, was that it is early in the innovation cycle and they very well be replaced quickely with different solutions/paradigms.

I bet you live in the US. What you describe is not a universal issue.

Plenty *commercial* labs frequently prioritized pure science over *immediate* consumer products, but none done so out of charity. Deepmind included.