Hacker News Reader

Sortof a letdown that after 24 32Gb RAM sticks you only get 6-8 tokens per second.

But a token is not just a character.

"hello how are you today?" - 7 tokens.

And this is so much better than I could have imagined in a very short span of time.

And only get to use 20k context length before it OOMs