It doesn't have to be an interpreted language. As long as you can read environment variables and write to stdout then it can be any language.
And, well, simplicity can be relative: if you already know language X, using it will likely be simpler for you than some other better language that you would have to learn first. The beauty of CGI is that its simplicity makes it a language-agnostic API, letting you pick the simplest language for yourself.