With the exception of Finnegans Wake I don't think that's true at all. Dubliners and A Portrait of the Artist as a Young Man are both fairly conventional books, nothing wildly different from the books that had come before. Ulysses, while very dense and difficult, is not incomprehensible. You don't have to understand every literary reference to appreciate the sheer mastery of language in that book. Finnegans Wake... well that's a bit different, I think probably one in a hundred people who finished Ulysses got to the end of that one. It's better suited to reading a random page or two at a time than reading from start to finish.