I think the scariest thing that people think of and I originally thought when I heard Ethereum will scale with "Layer 2's" is you are sending your money to a blockchain you do not have the keys to do not actually control the funds. The layer 2 scaling solutions will seamlessly interoperate with the current dapps that will be ported over looking seemless for users. You are essentially holding your capital in a contract you can call on at anytime and retrieve to the settlement layer (Ethereum layer 1).
These are the trade off imo and do not make either system better. I just prefer decentralization if I'm holding capital on a blockchain for an extended period of time.