Are you sure that's not just for Bluetooth access (which shows the same message, since it can indeed be used to derive somebody's location)? What business reason do they have to send users' location to their servers?
> graphene os with location disabled and always on VPN etc.
If you don't trust Ledger – how much do you trust your VPN provider? They know both your original IP and what you're doing on the Internet (or at least to which services you're connecting).