You don't need Box here because Vec<T> is already a fixed size handle to heap data.
Hell, even if you’re not moving things around explicitly, don’t forget that vectors resize themselves. If you use Box, then these resizes will be less painful with large objects.