I couldn't parse all of it, but largely I honestly think software is a sum of it's details you write better software the more details you add to the picture while maintaining simplicity. The art of software is balancing the two.

You can also find this in science, I feel like software is about finding the Gauss’s law for the dozens of electrostatic principles the masses keep coming up with.

AI strips away the need for simplicity by making brute forcing details via verbosity that ruins the need for "good" software, and makes all software by definition unmaintainable since maintainable product as a unit has to be simple enough for a LLM or a human mind to understand.

Currently a human context window is in 100s of billions of tokens, while LLMs seem to max out at 1M.

For anyone thinking but humans can't produce all of the text verbatim, that's the difference b/w humans and machines we can parse and take the most essential parts from 100B token context it's just are speed of prefil and recall is slower. And we memoize learning so if we know X then all variants of X are learnt on the same foundation.

Human learning is so very different from LLMs is so very fascinating as a trad "AI engineer", well some person who has worked and built/shipped ai agents and models at scale.