HNHacker News
TopNewBestAskShowJobs

pavelstoev

288 karma · joined January 23, 2023

https://www.linkedin.com/in/pavel-stoev-p-eng-674951246/
submissionscomments
pavelstoev··on The FFT Strikes Back: An Efficient Alternative to Self-Attention
I wrote down the following in the internal Slack chat on 01.06.2025, but of course, the performance of the actual effort is much more than writing it down.

Large language models (LLMs) operate in a high-dimensional token space, where tokens (words, subwords, or characters) can be viewed as discrete signals covering the multi-dimensional knowledge space. So FFT analysis methods can be applied to reduce time domain complexity to frequency domain representation with an idea to reduce computational complexity. So we can map token signals into the frequency domain. This transformation allows us to analyze token dynamics, such as their frequency of occurrence, temporal correlations, and interactions across contexts, with computational efficiency. In this approach, embeddings are treated as signals, and their relationships in sequence are captured as patterns in the frequency domain. FFT could be used to decompose token streams into dominant frequency components, revealing periodic or recurrent patterns in language usage - these patterns are repeatable across human generated knowledge and generally follow a predefined set of rules so the signals are not just white noise, they are predictable. By analyzing these frequency components, predictions of the next token can be made by emphasizing high-energy components in the frequency spectrum, reducing noise and focusing on statistically probable outcomes. Using this method we can reduce computational overhead during training and inference by enabling lightweight spectral analysis rather than heavy attention mechanisms, especially for long-context or repetitive sequences. Also using classical signal filtering techniques (LPF, HPF, band pass) could help align model behavior with human linguistic patterns, refine token embeddings, and improve efficiency in both training and inference phases.

pavelstoev··on Introduction to CUDA programming for Python developers
It is written in Python itself and emits efficient CUDA code. This way, you can understand what is going on. The current focus is on inference, but hopefully, training workloads will be supported soon. https://github.com/hidet-org/hidet
pavelstoev··on Introduction to CUDA programming for Python developers
The hallucinated code was reusing memory buffers filled with previous results so not performing the actual computations. When this was fixed the AI generated code was like 0.3x of the baseline.
pavelstoev··on Introduction to CUDA Programming for Python Developers
or use Hidet compiler (open source)
pavelstoev··on Introduction to CUDA programming for Python developers
Is this hypothetical person someone you know? if yes, please email me to pavel at centml dotz ai
pavelstoev··on Broken legs and ankles heal better if you walk on them within weeks
20 years ago, I broke my ankle in a friendly soccer game on the Cambridge Commons field. I was shocked when the doctor who performed surgery to put it back together ordered me to walk on crutches the next day after surgery and to continue placing more load on the ankle thereafter. 20 years later I can't complain :)
pavelstoev··on Musk-led group makes $97B bid for control of OpenAI
poison pill offer ?
pavelstoev··on Open source AI: Red Hat's point-of-view
It is really just “open use” with detailed defined by the license type (MIT, etc)
pavelstoev··on The impact of competition and DeepSeek on Nvidia
Interesting take. They are now reading our minds vs looking at our kids and interiors.
pavelstoev··on The impact of competition and DeepSeek on Nvidia
English economist William Stanley Jevons vs the author of the article.

Will NVIDIA be in trouble because of DSR1 ? Interpreting Jevon’s effect, if LLMs are “steam engines” and DSR1 brings 90% efficiency improvement for the same performance, more of it will be deployed. This is not considering the increase due to <think> tokens.

More NVIDIA GPUs will be sold to support growing use cases of more efficient LLMs.

pavelstoev··on Making AMD GPUs competitive for LLM inference (2023)
The problem is that performance achievements on AMD consumer-grade GPUs (RX7900XTX) are not representative/transferrable to the Datacenter grade GPUs (MI300X). Consumer GPUs are based on RDNA architecture, while datacenter GPUs are based on the CDNA architecture, and only sometime in ~2026 AMD is expected to release unifying UDNA architecture [1]. At CentML we are currently working on integrating AMD CDNA and HIP support into our Hidet deep learning compiler [2], which will also power inference workloads for all Nvidia GPUs, AMD GPUs, Google TPU and AWS Inf2 chips on our platform [3]

[1] https://www.jonpeddie.com/news/amd-to-integrate-cdna-and-rdn.... [2] https://centml.ai/hidet/ [3] https://centml.ai/platform/

pavelstoev··on Epic Allows Internet Archive to Distribute Unreal and Unreal Tournament Forever
Love both games spending countless hours, Unreal single player story was great (for its time). Are there any servers still online for UT?
pavelstoev··on GPU utilization can be a misleading metric
I recommend hidet backend in torch.compile - implements many advanced model-specific optimizations automatically. https://github.com/hidet-org/hidet
pavelstoev··on Euclid's Proof that √2 is Irrational

   that feeling you get when you realize the person who lived ~2300 years before you is smarter than you now...
pavelstoev··on Google releases smart watch for kids
Keyword - survived.

Everything must become better.

pavelstoev··on Google releases smart watch for kids
I strongly disagree with your comment. First of all, ALL kids must survive, not most. Safety and safety of knowing are two different things. Lastly, this tech looks a lot less intrusive than the current another watch that everyone is getting for their kids — this one appears to have more activity-engaging features.
pavelstoev··on RAG, fine-tuning, API calling and gptscript for Llama 3 running locally
Very interesting project and good progress on making private LLM use cases more accessible and usable, please keep going !
pavelstoev··on Ask HN: Video streaming is expensive yet YouTube "seems" to do it for free. How?
Next iteration of this will be video generated on demand with GenAI running closer to the request, ideally at the request.
pavelstoev··on Ampere Readies 256-Core CPU Beast, Awaits the AI Inference Wave
I think what will emerge eventually are LLM architecture specific ICs.
pavelstoev··on Seismologists suspect earthquake on San Andreas Fault is imminent
Should I be worried ? I just landed at SFO
pavelstoev··on Intel Gaudi 3 AI Accelerator
Best Tflops/$ is actually 4090, then 3090. Also L4
pavelstoev··on Llm.c – LLM training in simple, pure C/CUDA
Check out Hidet [1]. Not as well funded, but delivers Python based ML acceleration with GPU support (unlike Mojo).

[1] https://github.com/hidet-org/hidet

pavelstoev··on Llm.c – LLM training in simple, pure C/CUDA
How in Mojo do you support GPU data parallelism and all the benefits it brings ?
pavelstoev··on CA train project needs another $100B to complete route from San Fran to LA
Meanwhile - Brightline, Orlando to Miami 125 MPH, $6B and 4 years. https://www.wptv.com/news/region-martin-county/stuart/bright....
pavelstoev··on Nvtop: Linux Task Monitor for Nvidia, AMD and Intel GPUs
you can also profile AI/ML performance without actually running it https://github.com/CentML/DeepView.Profile
pavelstoev··on Kaist develops next-generation ultra-low power LLM accelerator
We build software acceleration for LLM, effectively running smaller llama2 models at the same performance on several L4's as on 1xA100.
pavelstoev··on Nvidia is now more valuable than Amazon and Google
You can do this in software, effectively.
pavelstoev··on I designed a cube that balances itself on a corner
This is precisely how I passed the lab portion of my Control Systems course in undergrad engineering.
pavelstoev··on Google TPU v5p beats Nvidia H100
yes there is a much more performant competitor that actually supports Nvidia GPUs [1] https://centml.ai/make-your-ml-models-run-faster-with-hidet/
pavelstoev··on Juniper Networks to Combine with HPE
I have an SRX1400 cluster with nearly 11 years of uptime. But this is the bare metal BSD-based firewall. When later, Juniper started adding a hypervisor layer on SRX platforms, and I started having problems.
← PreviousPage 2 of 4Next →