I do think off-topic comments can still contribute valuable information and insight provided they're not unreasonably off-topic. Tangential conversations, as krapp mentioned... everyone is going to approach a topic from their own unique perspective.
That said, I think dang has mentioned penalizing off-topic comments before so it does seem like something they can do.
This may be solved by folding the threads by default. Presumably, the root comments are addressing the original post directly, so they will be the most on topic. But requiring users to take the (minimal) effort of opening a subthread to view it means only people who are interested in reading further will ever read it. Currently, having to scroll past and ignore threads you're not interested in appears to be causing a lot of friction.
Since the staff has mentioned wanting to add thread folding anyway, mentioning this is kind of moot i suppose.
Also, due to /comments, it's entirely possible to start a thread from somewhere in the middle rather than the top.
Sorry, that was off topic. I think one of the reasons is the points system. You can still get upvotes/karma for comments off topic. And it is easier to write something when you're not restricted to a topic.
If that were the case, I'd still make the contribution. I don't blame those that do.