I would say that an even larger part of the HN community is lurkers. Those that read the articles, comments and upvote, but don't actively participate in discussions.

I've been lurking on HN for 4+ years, and I rarely ever contributed comments, mostly just read articles and read the comments to see others opinions.

Don't think this is true? Take a look at how many comments are left on articles on the front page, then if you were lucky enough to reach the front page of HN, take a look at your analytics software. You'll see something like 20k visits in a short period of time, there's no where near that number of comments. Most people passively participate in this community.