There's very little you need exceptions for with STL. Data structures, algorithms, etc. all work just fine without exceptions.

C++ without exceptions is great. Google doesn't use exceptions and they still use STL. Gamedevs usually don't use exceptions and they're fine. Just use some other mechanism for errors, like error codes, absl::Status, std::expected, etc.