HN
Hacker News
Top
New
Best
Ask
Show
Jobs
Comment by d3m0t3p | Hacker News Reader
Parent
Full thread
d3m0t3p
·
You can batch only if you have distinct chat in parallel,
View on HN
mlyle
·
> > if I want to run
20 concurrent processes
, assuming I need 1k tokens/second throughput
(on each)
Reply on news.ycombinator.com