Baloney. chroot and execvpe work with mere temporary directories. This is roughly how homebrew does "isolated" builds. You don't need virtualization, but you may want it.
EDIT: User and process namespacing would help a lot as well, to improve build time isolation.
On a tangent: I’m a big fan of Homebrew, but Nix’s deterministic builds are more isolated than Homebrew’s. Last I heard, Homebrew also tries to avoid keeping around any but the latest version of a formula, which would force you to upgrade. With Nix, your installed libraries are isolated, so I can (for example) open a shell with GCC 5 and Python 2 and another with GCC 12 and Python 3.12 with absolute confidence there’s no “contamination” so to speak. With Nix you come a lot closer (even on macOS) to achieving the dream of having your entire system declaratively managed, so you can get a brand new machine, pull in your config from GitHub, and have an identical setup with minimal hassle. I think even with `brew bundle` it’s not that easy to achieve this.