You don't even allow that. I'll use REST as an example because that's what I use, but any form of RPC would work.
App server has no database access. No DB credentials, and no network access to the DB. It does, however, have access to a service that sits in front of the DB. It can ask the service:
GET /users/10
... and that's it. It gets one record. The service fronting the DB has a single query it can execute:
SELECT * FROM users WHERE ID = ?
Sure, if someone can compromise the user-data service, then they have more access. But that's what defense-in-depth is: if an attacker compromises one system of yours, they don't get the crown jewels: they need to keep going, and each new attack they need to mount costs them more, and sometimes even makes them fail.