HN
Hacker News
Top
New
Best
Ask
Show
Jobs
Comment by zzzoom | Hacker News Reader
Parent
Full thread
zzzoom
·
Prefill (GEMM) is compute bound, decode (GEMV) is memory bound.
View on HN
Const-me
·
> decode (GEMV) is memory bound
Decode with batch size 1 is GEMV. Batching makes the decode GEMM too.
Reply on news.ycombinator.com