this exactly the future we are working towards as well.
You should checkout AppLess
http://github.com/thesysdev/appless
To take this a step further: https://chatjimmy.ai/
Imagine AppLess running a model as good as qwen at 20,000 tok/sec. It would be generated in a shorter amount of time as downloading a webpage right now. If this works out the consequences are kind of scary. The end of SaaS, the end of software being the moat or the property of companies, the embolstering of data protection (since that's fundamentally what code operates)...