The AI models we currently have still maintain their working state with a finite and laughably-small context window, but that is already starting to change. My .claude directory contains over 300 .md files that I didn't put there myself. Their contents are eye-opening. The question of who owns, stores, maintains, and can access that data is going to become insanely important over the next couple of years.
If you thought LLMs themselves were disruptive and contentious, just wait until the fight over object permanence gets under way. That's when owning your own box full of graphics cards is going to become important. My own bet is that I won't care too much about the electric bill or my opportunity cost when we all find out what the AI labs really have in mind, and what they're going to have to do in order to justify the valuations they're seeking.
TL,DR: it's not about the tokens, IMHO.