Hacker News Reader

Isn't equating active parameters with cost a little unfair since you still need full memory for all the inactive parameters?

Well, since it affects inference speed it means you can handle more in less time, needing less concurrency.

Fewer parameters at inference time makes a massive difference in cost for batch jobs, assuming vram usage is the same