I recently had a document break because I used umlauts together with the subfigs package. Apparently both use " characters internally and clash badly. This is not a particularly exotic use case.
The core/official LaTeX code is really quite stable, but it's also very limited, so it's pretty reasonable to conflate it with the LaTeX ecosystem as a whole. But yes, the LaTeX Team is definitely aware of the problems caused by Babel shorthands (which is what " is in German), and they're trying to figure out some way to fix it without breaking other documents.