Hacker News Reader

I'd love to see how this compares against Llama.cpp on speed. Do anyone any benchmarks that compare PyTorch.compile vs Llama.cpp?

This person claims to have compared llama.cpp against gpt-fast on a 4090, and found gpt-fast about 20% faster.

Awesome, thanks!