User owned and controlled inference in their compute context is what beats enshittification, it is equalizing Big Tech power asymmetry against users, or at least keeps it in check. And so, I wish this team much luck, and await their results from their experiment. Many thanks to YC for funding them.
I'm honestly not certain myself how we'll monetize this, but I have had a lot of fun building it and using it myself, and seeing how others use it. As you said, if we continue down this path without success, then worst case, what we built will still exist.
Re: local models, I am a big proponent, but they aren't there yet. This task is non-trivial. Try taking raw HTML from a webpage (minified, bundled, abstracted variable names, no comments, etc.) and using it as a basis to make useful edits. It's tough, and very impressive that any model can actually do it reasonably well. It tentatively looks like we're starting to reach a plateau for general models and open-weight is catching up, but I know the big labs/companies are aggressively capturing massive data and squeezing everything they can out of RL for more task-specific tuning. I hope open-weights can continue to compete!