Ironically, we could say that we're losing the true ownership relationships between the objects, and we're making medium-term memory leaks more likely; drop() can free a Box pointing to a child, but can't free an index. In other words, we lose the guarantee that an element is released for reuse.
AFAICT, a language would need something like higher RAII [0] or linear types [1] for that. I'd love to see Rust adopt these features too one day, though it may be difficult to do backwards compatibly.