I think you're suggesting a false dichotomy.
Of course a complex language can be used to write simple code and vice versa. Ultimately every program is going to increase in complexity-- that quote about failure vs legacy nightmare applies here.
IMHO it's better to reduce the complexity of the language to a minimum so that language/runtime complexity doesn't have a multiplicative effect on application code complexity.
For instance, C++ is a complex language. That complexity inevitably accrues to applications written in C++. This is a conscious trade-off of the language -- performance + high level programming, or what have you.
In the end, though, I am not sure I'd argue there's much correlation between simplicity of the language and simplicity of the code. It's too hard to get anyone to agree on what simple means anyway. Me, I prefer Hickey's "Simple Made Easy" for that, but not everyone agrees.
ETA: I guess what I would say is: would you prefer to start with something simpler and build complexity appropriate to the problem domain out of simple, but sound ideas? Or would you prefer to start with a set of inherently complex primitives and build something else complex on top of that? It's subjective, but I've probably leaked my bias in the phrasing.