For example, if you're rate limited (429), a robust API will still give you back data on how many requests you've made and how many requests you're allowed to make, so you're still going to have to check the payload regardless.
The combination of specific error status codes along with error messages has been redundant in my experience.
Furthermore, in Axios, checking for `error.response.data` isn't terribly far from `error.response.status`, so I'm unsure of this "worst client experience" you're talking about.
It's pretty intuitive for me.