Yes, this is what I'm currently doing with a cloud-based website automated-testing system.

In my case, code supplied by the end user is compiled into a different language such that I think I can prevent intentionally-malicious activity.

Nonetheless, spinning up a VM to create an environment in which potentially untrustworthy code is executed before then destroying the VM seems the safest option.