We've made something similar at Repl.it -- we call it "Repl from Repo". It's far from running everything but it can run a lot: https://repl.it/blog/github
We tried to make it as seamless and as low configuration as possible. Our "run on repl.it" badge is currently growing exponentially on GitHub.