In the scenario you gave, the trusted oracle never needs access to the funds. This is important for legal/regulatory reasons. This would not be possible without smart contracts.
Yes, its not 100% trustless, but its far more trustless than the alternative.