Hacker News Reader

nah, Lora quantized LLM’s are going to be at the OS level in 2 years and consumer architecture refreshes are just going to extend more RAM to already existing dedicated chips like Neural Engine

client side tokens per second will be through the roof and the models will be smaller

View on HN

LoRa is not a quantization method, it's a fine tuning method.

It's two adjectives on the noun.

you read that whole paragraph and assumed this prediction didn't involve consumers using fine tuned models despite lora being explicitly mentioned?

-EQ moment

edit: its about the combination of those methods making models accessible on consumer hardware

Reply on news.ycombinator.com