There are different ways in which a language can be difficult to learn. Haskell has a steep learning curve because it has a lot of unusual concepts (for those who come from an imperative background) and requires a shift in the way we think about code. J is also difficult to learn because the syntax is crazy, and again, the language is very different from what we're used to.
I found R difficult to learn because it is seems inconsistent to me. Now one reason might be that I picked up bits of R here and there without taking the time to learn the syntax from a book (unlike what I did for other languages), but it took me quite some time to be able to make sense of the different structures (vector, list, matrix, dataframe), their differences, and in particular how functions operate differently on these structures. I also have a deep hatred for the system of attributes (why would anyone want to give attributes to a vector...), and find the indexing system (especially for lists) to be nonsensical. In fact, I think that lists themselves are terrible to work with.
The general impression that I've had learning R is the language is not coherent and systematic in its design in the way other languages are (I know python, c, common lisp, ...), and I find myself spending a lot of time in the interpreter simply trying out things because I'm not sure that it will return what I want, in the format that I want (which never happens to me in any other language).
Now don't get me wrong, I use R daily and it's a useful tool. It has lots of great libraries, including the fantastic ggplot2, and the remarkable Rcpp (best C++ interface I've ever seen.. ok I haven't seen any other, but this one is really great). But learning it was no fun, and if the statistics community decided to move to a cleaner language, I'd definitely be running ahead..