I've never actually seen anyone in 40 years actually swap out that database.
Probably a bigger benefit though, is being able to use a different DB in production (probably big scale) vs on your personal machine while developing.
So we've just switched underlying connection and fixed a dozen of (mostly reports) places with plain SQL queries with MySQL-specific syntax. ORM library handled rest just perfectly.
So I agree with you with this one important exception.
But I think the point is still valid. You probably don't want your early design to focus on an edge case.
That said, a clear API between your DB and app is worthwhile for a lot of reasons (in general).