Are there any drawbacks, like maybe it’s slower for single-threaded code?
In my experience, the main drawback is cognitive complexity: there are not one but four different implementations of map and set provided, each with slightly different memory and compatibility tradeoffs, and using the wrong one may break existing code that depends on (for example) stability of pointers to elements or iterators during set mutation.