Taller buildings will require you to transfer to a different elevator once you've reached a certain floor; one because else you get cables that are hundreds of meters long (with the weight to match), and two because else you'll need more elevators and -shafts to service all hundred+ floors in a timely fashion, which takes up real estate / floor space.
Floor space is generally too precious for that. In a typical multi-bank elevator setup, lower floors get their toilets or other utility spaces placed between two banks of elevators that don't stop at that floor; the corresponding point in the upper floors is the elevator lobby for an upper-floor elevator bank.
There's a widely reported story that a group of people stuck in an elevator in the WTC on 9/11/2001 used a window washer's squeegee to cut their way through multiple layers of drywall to escape into a 50th floor bathroom.