Are they, though? Consider I have a linked list (or a map/dictionary, or any data structure that allows removal of arbitrary elements). I add elements with indices 1, 2, 3, 4, 5. I remove elements with indices 2 and 4. I'd like to make memory consumed by these elements available again. At the same time I don't want to change indexing scheme of the whole data structure. So I have to keep "holes" with indices 2 and 4 at the very least. After few million operations that may become very inefficient.
In a typical language, that would be resolved by having a global allocator that allows indices 2 and 4 to be reused in neighboring data structures. If I don't implement a global allocator that shares indices between objects (as that kills the whole point), I consume more memory than needed and rely on my own clunky implementation of garbage collection/defragmentation, don't I?