1) Custody: with fiat, the concept of self-custody is "store your cash under your bed," and as we continue to move away from cash into digital money, this option is being eroded. In crypto, the concept of self-custody may be an EOA (account based on a private key) smart contract (such as a multi-signatory account), or some combination. This is not to say that all users should be choosing non-custodial wallets, but the ability to choose and diversify your assets across a range of these technologies is, in my opinion, better than having no choice at all.
2) Mitigating the need for intermediaries in certain situations. Currently to handle escrow of digital assets in fiat systems, you typically need to hire or use a third-party agency. In some cases escrow could instead be managed by a (cryptographically-verified) program execution, e.g. "transfer X domain name record for Y amount of stablecoin assets".
3) Privacy-preserving attestations via zero-knowledge proofs, which could use a public (decentralized) ledger so that all parties can verify the validity of the proof.
4) The immutable and public nature of smart contracts gives them certain properties we do not have with our traditional financial systems. For example Libor manipulation went unnoticed for several years; partly because the systems were closed and difficult to analyze. The public ledger is comparatively easy to analyze and difficult to manipulate. (The most common manipulations rely on social engineering; e.g. convincing somebody to sign a transaction, or buy ShitCoin2000.)
5) General security and operational improvements. Much of our traditional system is upheld by FORTRAN and questionable security methods (PINs, passwords, cell phone 2FA). Crypto currencies are extremely adversarial environments that are forcing new security models and cryptographic primitives: zk-proofs and zk-VMs, multi-party computation, new methods of private key storage and management, post-quantum cryptography. Hopefully some of these advancements (being funded by crypto currencies) will eventually spill over to other sectors.