There is extremely little quality loss from dropping to 4-bit for LLMs, and that “extremely little” becomes “virtually unmeasurable” loss when going to 8-bit. No one should be running these models on local devices at fp16 outside of research, since fp16 makes them half as fast as q8_0 and requires twice as much RAM for no benefit.
If a model is inadequate for a task at 4-bit, then there's virtually no chance it's going to be adequate at fp16.
Microsoft has also been doing a lot of research into smaller models with the Phi series, and I would be surprised if Phi3 (or a hypothetical Phi4) doesn’t show up at some point under the hood.