The more code I need to READ to understand what I'm looking at, the less optimized for maintenance it is. If it takes 5 lines to understand that the map I'm looking at is effectively used as a set, it's worse than if I had a Set abstraction.
Of course now I do, 20 of them, because when there's no built in, there will be packages made by the community.