How often website redesigns are only about html? Usually everything changes. New flows, new dialogs, new toolbars, new frameworks.
Often the web UI gets and Android UI and iOS UI siblings. Gets replaced altogether by a completely different UI, developed by a different team.
APIs change much less often, and can be shared between many UIs. You can also version them. How are you going to version a web UI? Like reddit with "turn back on classic look"? How long can you keep the old UI around just because you have 1000s tests around that use it.
> but the idea that the tests are inherently brittle is wrong
I don't think so. Tests are great calcifyiers (check my blogpost about it if you want). If you test though your UI, you're calcifying your UI. The more you test though it, the more you calcifying it. No way around it.