mesuvash

IIUC, The paper's notation S^(d-1) means the unit sphere in R^d (e.g., the familiar unit circle is S^1 living in R^2). So, i think, x in the algorithm is already a unit vector.

Reference: Section 2:Preliminaries ... We use the notation S^d−1 to denote the hypersphere in R^d of radius 1.

Section 3.1 Let x ∈ S^d−1 be a (worst-case) vector on the unit sphere in dimension d.

Fair point. I've updated the animation to address this. The grid now uses the correct non-uniform centroids (optimal for the arcsine distribution in 2D), so you'll see grid lines cluster near the edges where unit-circle coordinates actually concentrate, rather than being evenly spaced. The spacing does change with bit depth.

On the second quantization step: the paper's inner-product variant uses (b-1) bits for the MSE quantizer shown here, then applies a 1-bit QJL (Quantized Johnson-Lindenstrauss) encoding of the residual to make dot-product estimates unbiased. I chose to omit QJL from the animation to keep it digestible as a visual, but I've added a note calling this out explicitly.

Yes, this is important in high dimension. But sadly, very hard to visualize. In 2d it looks like unnecessary.

That's actually correct and intentional. TurboQuant applies the same rotation matrix to every vector. The key insight is that any unit vector, when multiplied by a random orthogonal matrix, produces coordinates with a known distribution (Beta/arcsine in 2D, near-Gaussian in high-d). The randomness is in the matrix itself (generated once from a seed), not per-vector. Since the distribution is the same regardless of the input vector, a single precomputed quantization grid works for everything. I've updated the description to make this clearer.

Yes. Great catch. I simplified the grid just for visualization purpose.

I've updated the visualization. The grid is actually not uniformly spaced. Each coordinate is quantized independently using optimal centroids for the known coordinate distribution. In 2D, unit-circle coordinates follow the arcsine distribution (concentrating near ±1), so the centroids cluster at the edges, not the center.

Author here. Sorry still working on refining the post. Will share once the post is ready.

TurboQuant explained with an easy to understand (no-math) animation https://mesuvash.github.io/blog/2026/turboquant-interactive/

I am glad you liked it :) You might like this https://mesuvash.github.io/blog/2026/rl_for_llm/ as well :)

Thanks for the pointers.

From my personal experience, Auto-encoders are amazing for dense input (images, audio etc), more specifically, when the input feature space is not large. However, in many real-world problems such as recommendation, ranking etc. the feature space is generally very sparse for eg clicks, purchase of items (say 100M items). In such cases, scaling can be challenging with neural models esp Autoencoder.

>>I think the hashes could take some work. Any suggestions or thing that are not clear?

Thanks for your feedback. I shall update the post accordingly.

Nothing but #respect. It's hard to see people who give up fortune for what they consider right thing(atleast from his perspective).

Thanks for the info :)

Btw, If someone wants to drop out from the course. How can he/she do so ?

Awesome. MITx platform is superior than any other online learning platform i have ever seen. Very well done. Congrats.

videolectures.net videos are generally about technical topic. Mostly related to Machine learning these days.

If you are looking for inspirational videos or Interesting videos you can have a look at http://www.lolzwow.com

Well i think you need to adjust your routine. I think both startup and relationship are equally important and you have to manage accordingly. Relationship stands on top of understanding and i think you both should understand each other. If your girl understands you and you do the same then i dont think it wont be any problem.

Good luck.

Yeah, i am in verge of adjusting my expectations.

Watch these videos

http://bit.ly/ehfTc3 http://www.youtube.com/watch?v=UF8uR6Z6KLc http://bit.ly/eNZPda

I do it whenever i lack motivation. It helps me a lot, I hope it will help you in same way.

And you if you really think you want to take a break, Come to Nepal, Its a nice place to be. However its landlocked so you wont find beaches.

Nice work.

I will suggest you to start with the one you are most comfortable with. Both has strong community. Being a pythoneer, I can assure that python community and IRC is very has lots of people who are really very helpful. I dont know much about ruby.

Yeah sometimes i feel TechCrunch is biased. I am not comfortable with people who write news on behalf of some company. We expect them to write the real news rather than supporting some and reprimanding others.