As I understand it, that's kind of the point.
This is why HN dislikes humorous fluff-posts: they both easily rise to the top, and encourage humorous fluff-replies, which means the first few screenfuls of comments will be guaranteed to induce the kind of "scroll-pain" that makes people close the tab.
I've yet to see any discussion forum solve the problem of long threads with lots of useless fluff floating to the top.
While I'm usually a kind of hard core html-first, ajax/js/webapps later kind of guy -- I'd love for the comments to be served up in a json-blob, with a couple of user-settable preferences ("Show only comments rated higher than N, hide threads with lower (median/mean) rating than N, show all direct replies to my comments -- and similar).
They use a very simple system, collapsing posts after a certain depth, and hiding more comments after a certain number (10, I believe) at the level below the top comment (sorted by score obviously).
It works extremely well, and thanks to the fast JS collapsing, it's not at all a hassle to read a subthread that happened to get collapsed if it piques your interest.
It's so simple it may easily be overlooked in its obviousness, but really you don't need a very complex system that is strictly a lot better than no collapsing at all.
If uninteresting comment threads could start collapsed, that'd be great. But they won't, because humor-fluff and other such things are superstimuli for upvotes, so you can't use anything about the vote tally to determine collapsed-ness. (If there was a secondary voting system--like, say, if enough people collapsing a thread would make it start collapsed for others--this might work. But then you'd have to take into account the people who collapse everything as they read it, to mark their place...)
The problem is, your definitions of what is uninteresting, useless and fluff are subjective, and no more relevant than anyone else's, which leads to a conflict of interests within the community about the bounds of what Hacker News content should and shouldn't be.
I think the models which would satisfy the most people are opt-in, in this case, choosing to collapse threads and ignore users rather than expecting the hivemind to do it for you.
Perhaps if users had a custom set of filters which automatically collapsed threads for them based on their own criteria, that would solve part of the problem. But I don't think it's too much to ask of people to actually take the minor effort to form an opinion on what they read, or curate their own account, if they expect content they don't like to be hidden from themselves and potentially from others.
But for some reason or other pg won't do that.
I know there's some extensions/bookmarklets and I use them, but they can't do the auto-collapsing based on score+amount, the very simple yet elegant algorithm Reddit employs, pretty much the one thing that is absolutely necessary to have a well-usable threaded commenting system.
Instead, pg "just wrote the simplest thing first", or something.
If the worry is that comment threads are too long, HN could implement something like reddit, where you click to read additional comments in a long thread.
eg:
Two commenters X,Y with sub-threshold karma (<1,000) could never have a dialogue (two-way, real-time) as a third party Z would need to endorse their each and every comment.
Two commenters X,Y (Karma 1,000+ each) in substantially different timezones with a-synchronous dialogue (eg, overnight replies) would need a third party z to endorse each and every comment (at least until the other wakes up).
Two commenters X,Y (Karma 1,000+ each) with opposing views, could never have a real-time dialogue without a third party z to endorse each and every comment (unless #)
Hopefully these are at least helpful to dilineate.
# "Thank you sir, can I have another".
When everyone knows that what's being discussed is public, it tends to keep the tone more conversational and clear -- I think. The "feel free to contact me, email in profile"-response seems to work well enough for those that do want a private (albeit not anonymous) conversation?