I’m confused by the wording “without wasting disks for the base OS” - I wouldn’t normally consider this a “waste”, would anyone else? There are big downsides to running off of a USB key all the time unless I’m missing something
SmartOS was developed by Joyent for their cloud computing product, it's primary use case isn't desktop computing. I think the advantages mentioned above were probably a bigger factor than the disk space. I would also guess that PXE would be the standard way to boot in a datacenter, not USB.
How exactly does it make any sense to use ECC memory and ZFS RAID for error correction and redundancy, but then rely on the modern floppy disk for the OS itself?