I agree with Wells about the frustrations with the endless riddles and the incessantly opaque style.
However an unexpected pleasure was how much fun it was to read in a group. For example, a set of allusions sprinkled on the page might catch the eye of someone for whom the subject was a pet interest or hobby, otherwise we all would have missed it. I learned a lot about world history, art, philosophy, etc., and I almost always ended the evening astounded at the oddest bits of information my friends had tucked away in their heads.
Finnegans Wake works really well as a scaffold for learning and conversation; as a story, I'm not so sure, but I think it is a remarkable literary experiment. There are also several truly beautiful passages hidden away in the book, which hit you by complete surprise when you stumble on them.