jjssmith

14 karma · joined February 15, 2021

jjssmith··on Zen and the Art of Machine Learning Research

> One really impressive thing about OpenAI is that most of the people running the company (on the technical side, at least) are under 35. Many of the important decisionmakers behind chatGPT are under 30.

During Gold Rush most 49ers were under 25, so there's still room for improvement!

[Continuing the analogy, you may find that many AI heroes are just those who happened to be closer to pools of TPUs and GPUs in the early days...]

jjssmith··on TurboQuant: Redefining AI efficiency with extreme compression

LOL. This is a classical technique, Johnson-Linderstrauss etc. In this context, rediscovered every few years (recently months), e.g. here's 2017: https://proceedings.mlr.press/v70/suresh17a

jjssmith··on Quantized Llama models with increased speed and a reduced memory footprint

You might like an information-theoretic take on SpinQuant and the likes [1].

tl;dr: round((2*R)*x) is not a great idea for an R-bit quantization.

[1] https://arxiv.org/abs/2410.13780

jjssmith··on The Stanford Startup and the MIT Startup (2013)

AFAIK, the Markov chain page ranker existed before Google (Jon Kleinberg's papers, e.g.), what Google paper added were some smaller bells and whistles, like using the text description of the hrefs.

jjssmith··on Mark Braverman Wins the IMU Abacus Medal

A big part (majority?) of this work is rediscovery of the results from info theory (EE) community in the 90s (ctrl-f Orlitsky, Shor in the pdf).

jjssmith··on DeepMind achieves SOTA image recognition with 8.7x faster training

Rediscovering NLMS now, good for ML folks.

https://en.wikipedia.org/wiki/Least_mean_squares_filter#Norm...