What is your actual security model?
Here is a simple solution: have a single machine running a web app ("paste your code here") and one or more sandboxes (physical machines running off CD-ROM/NFS/anything nonwritable). On submission, the web server sends the code to one of the sandboxes, which compiles and executes it and returns any output (SSH seems convenient for this). Any other network access to/from the sandboxes is blocked. If one of the sandboxes stops accepting connections or otherwise misbehaves, reboot it.
This doesn't isolate the untrusted programs from each other. Sufficiently nasty code may be able to persist across reboots by nestling in firmware, but that does not pose an additional security problem. This setup is trivial to DoS, but it's simple and doesn't rely on, say, the Linux kernel being secure from local attacks - you just need a properly-configured firewall, e.g. on the web host ("drop all packets to/from sandboxes except locally-initiated SSH connections").