> This fundamentally can't be fixed because otherwise you can't have something like a wallet that stores a balance which is assigned to you and only you.

This statement is false. Zcash's shielded transactions hide both the wallet ID and the transaction via zero knowledge proof.