In all the instances I have personal experience with the buildings are very small. No more than a dozen units and so there isn't usually a big risk of running into anyone. Also, I don't think you can actually ring multiple floors in those elevators. The elevator would need to take you to your unit then go back to the lobby to get the next person.
The building this attack happened in had a total of 7 units.