« By running real Java code directly from the web browser, students are able to try out Java without installing it. » their "directly from web browser" solution is using http://learnpythonjail.appspot.com/ so I wonder if it's using jython to interpret the java code.
Now, as it happens, I have some experience trying to get Java to compile and run on Google App Engine. Actually, at the outset I thought it was impossible because of the way the sandboxing works on App Engine (writes to the file system are disallowed and the javax.tools. package is not part of their white list). So straightforward approaches such as invoking the Java compiler from code will not work.
Here are various approaches that I have tried and the results:
1. Use Beanshell and approximate Java:
Beanshell is a scripting language for the JVM with Java like syntax. It also has a "strict" mode where it will behave like a strongly typed language.
Pros: you can run this on App Engine and can trust Google Engineers to worry about security of their sandbox. You also get the scalability of App Engine.
Cons: It is not really Java and language features like Generics are not supported. The error messages are also not as useful as what the regular Java compiler outputs.
2. Get users to compile code on their own machines and upload the .class file. Then write a class loader that will load this class and then execute whatever method you need to.
Pros: Works perfectly and no language issues.
Cons: The interactivity is lost to an extent. Also, beginners need to start worrying about installing Java and configuring CLASSPATH etc. from get go.
3. Use an in-memory compiler like Janino. Pros: Works like the Java compiler and error messages are useful.
Cons: it does not support generics and it also does not support other features like the for(String value : values) {} syntax.
4. Use Google App Engine as the front-end and then run a compiler-as-a-service somewhere else (Linode etc.).
Pros: Full language and you are in control.
Cons: You better know how to sandbox the code (chroot / SecurityManager etc).
5. Use Google App Engine as a front-end and outsource compilation and execution to some other service like Ideone.
Pros: Full language support and someone else worries about security issues.
Cons: Expensive (the Ideone API only allows 1000 free submissions per month).
If you know some other method, please let me know. I love trying to see how to make things work on App Engine (I learn quite a bit this way).
(minor edits)
My Python website (http://www.learnpython.org) uses Google App Engine's eval (which is pretty safe, since Google secures the environment).