I suppose, combine this with pressure from public or private investment, and the way to get ahead is to package anything into a prospect of revenue generation. I'm sure that's part of it too. Everything has to monetize because some business school graduate hasn't "made it" until they have a yacht like their ivy league friends.
Eh, probably comes across as curmudgeonly or "who moved my cheese". But if there is an area that can improve this longstanding problem in tech, my guess is teaching the right skills and concepts at the collegiate level. And that's not a simple thing either.
Edit > reading a bit more, this focuses on chat applications and seems to be a decent caching implementation tailored to that domain, of which, I'm guessing will allow AT&T and Verizon to save money on their gobsmackingly horrible AI chat bot in their mobile app. As an individual, it's unclear how this benefits me though. I don't think it does. ME: asks chat bot question about insurance coverage, CHATBOT: immediately serves canned response in no time about how that's covered in my individual insurance plan which I read more about on their website (pro-tip: no, I can't, those details are actually never on the website)
It seems to me like you’re easily hand waving away a hard problem in a different part of the stack you’re less familiar with.
Again, the novelty is in getting cross attention to work correctly despite the fact that you’re stitching together arbitrary caches together. It’s akin to taking snippets of compressed portions of random compressed files and reconstructing a new correct plain text. That’s obviously not possible but clearly this has been accomplished with the KV cache for arbitrary models (ie not trained for it) despite the KV cache working like decompression where all the preceding bytes have to be computed correctly for the subsequent token to be correct.
Meanwhile the AI engineers are doing the exact opposite. Bragging about the hard parts and rolling their eyes at the easy parts.