Your understanding of smart contracts is correct - whenever smart contracts deal with stuff in the physical world you need to have an oracle.
In my experience over past half decade, there's been a lot of misleading information spread by people about smart contracts with schemes like claim to be a good use for smart contracts that suffer from oracle problems. A lot of this has come from naive people that just didn't other stand, but some has been spread by people looking to make others part with their money.
The way I think about smart contracts, is that if you need to trust someone in the real-world, by default you probably don't want or need a smart contract.
That said, there are a subset of such instances where smart contracts that depend on trust in real world entities have proven to be very useful. Wrapped tokens and stablecoins are an example of this - Tether being the most famous.