Duplicates will always happen, and as you have pointed out, sometimes there is positive value to that. The idea isn't to avoid duplicates, per se, but to try to ensure that the same discussion doesn't get hashed out again and again in different different, parallel and unconnected threads. There have even been cases[0] where the mods explicitly dipped in and moved comments from one submission to another to avoid the "split discussion" problem.

So the idea is to ensure that anyone interested in an item can, should they choose, easily find and read the comments made by other HN contributors. Feel free to disagree, although the 17 upvotes (at the time of writing - votes may go down as well as up) indicate that there are people who do find it valuable/useful.

[0] https://news.ycombinator.com/item?id=8499286

Edit: Just checking your profile I see that you've been here less than a year, so you may not be aware that because it's me to whom you're replying, I can't down-vote you. In fact I've up-voted you, because while I disagree with you (obviously) I don't think your comment is especially negative.