Things like tree-sitter took forever to come along, but the research projects that built the algorithms they rely on are a quarter century old.(https://www2.eecs.berkeley.edu/Pubs/TechRpts/1997/5885.html)
Meanwhile, incremental lexing and incremental recursive descent parsing and LL are actually not that hard (incremental LR is trickier).
It's just that few people really understand how any of it works to implement it.
This is not a "it was not needed" issue. Lexing times of files for syntax highlighting is still "seconds" for larger files, for example.
Meanwhile, i can explain near-optimal incremental lexing to someone who knows how lexers work in a few bullet points, and it's trivial to implement:
1. As your lexer looks at characters, keep track of the maximum lookahead and lookbehind used during a lex.
2. When some range of the string to lex is invalidated, adjust the invalidation bounds to include the maximum lookhead/lookbehind as well.
3. Relex the invalidated portion of the string.
That's it. For optimality, compute the maximum lookhead/lookbehind on a per-token basis, instead on a per-lex basis (this is trivial if your editor is already storing the tokens under the text somewhere)
If you store position info in your tokens, you can easily adjust as well.
For incremental recursive descent parsing, it's similar - only rules who ever looked tokens that changed could be affected by them. You can easily keep the interval of tokens that a given rule looked at. (If you want truly optimal, you can store the set of intervals looked at, but this is almost always overkill)
If all you ever have is a generator, you will never implement any of the above until your generator does.
I have watched tons of projects spend lots of time optimizing grammars, doing weird caching, etc, when a few simple/better algorithms would have saved them from all of it.
(which is why in that sense, tree-sitter has been a revolution)