Hacker News Reader

Hmm I get 44,7 tokens/s on the smallest 1.7b one in the browser on my m1 max.

I get 9-10 tps on my MacBook Pro. Though the small model seem really limited

Interestingly, I get 44 tps on Safari, MacBook Pro base model.

60tps in Chrome here - 14" MBP M4. Safari loads 3% of the model and then stops - repeatedly.

Basically zero using a 7700S in Chrome on Ubuntu though. Must be not working at all, as the performance difference can't be that large.

18.8 tps on an 11th gen mobile i7 here.