You hit the nail on the head. Unless it’s totally virtual - on chain finance, gambling, etc. the contracts are only as good as the oracles. I guess that’s why DeFi works so well.
Fortunately decentralized oracles are an area of focus right now, but it’ll be a slow process since most expansion will require a special integration and potentially expert oracles.