It's within the same process and realm (window)
It has a cost, but it's nothing compared to putting every dependency of a large app in a separate iframe/process and figure out a way for them to communicate.
Plenty of objects in the browser API contain references to things that could be used to defeat the compartmentalization.
If one were to enumerate all properties on window and document, how many would be objects with a reference back to window, document or some API not on the allowed list?