I did a few things to get this running:
- The application server and build server are separate, with access control to the build server controlled by an API key that I can revoke should anyone gain access to it.
- The process that compiles and runs your code is an unprivileged user.
- All import statements are disabled in user submitted code
- Foundation is imported selectively with only a few classes allowed to be used by submitted code (namely, the NSString, NSArray, NSDictionary, etc APIs)
- Code is checked for a list of blacklisted APIs before hitting the compiler
- The execution of your code has a timeout of a few seconds, and the process will be killed should it take any longer.
I looked into more advanced techniques for sandboxing but couldn't find anything on OS X that was feasible. Happy to be schooled on security though, if you know better than I do :)