Well the thing is you can do some interesting tricks they haven't thought of, one trick I am doing with these tiny language models is to train them on a handful of authors in one genre rather than training on every story ever written, a distinct style emerges. Nowadays you have these mini pc's with Rocm support (under linux) so its becoming cheap to build your own models.