If your service is returning 5xx, that is the the definition of a server error, of course that is degraded service. Instead we have pointless dashboards that are green an hour after everything is broken.
Returning 4xx on a client error isn't hard and is usually handled largely by your framework of choice.
Your argument is a strawman