10x slower is a deal breaker lol.
I think max I could tolerate personally is 2x slower for same quality.
I think max I could tolerate personally is 2x slower for same quality.
Well, other people are working on it, I'm just trying to get what they do running on my system.
MTP looks good at the moment.
https://www.hardware-corner.net/multi-token-prediction-llm-s...