This is basically a GPT-4 level model that runs (quantized) on a 32gb ram laptop.
Yes it doesn't recall facts from training material as well but with tool use (e.g. wikipedia lookup) that's not a problem and even preferable to a larger model.
Yes it doesn't recall facts from training material as well but with tool use (e.g. wikipedia lookup) that's not a problem and even preferable to a larger model.
Can you share more insights on this? Going by @simonw's testing, the quantized model doesn't seem close to GPT-4 level.