Perhaps we expected that the lessons we collectively learned about good design would be self-evident to the next generation or that there would be a knowledge transfer, but that doesn't seem to be the case.
The web (and most software really) seems to go through such extreme cycles of destruction and rebirth for the sake of marketing—and resume building, if we are being honest—that it's sometimes hard to find a throughline from the designs of yesterday to today.
I think about it all the time when there is an OS update (mac) that moves the state of design off into a direction that is in conflict with the design ethos of the past. I realize then that the designers and engineers likely never even used a classic OS let alone read the work of Jef Raskin or the Macintosh HIG.
It's really weird that the industry casually sloughs off lessons of the past. I don't think we should be restricted to only designing as we did in the past, but I do think that it's beneficial to understand where we came from.