That's also how reddit works, and in 6 years it's gone from technology, programming and world news to memes and cat pictures.
When anyone can create an account and start voting, both moderation and reminders from existing users about appropriate content are needed to prevent or slow the move from specialized articles to general interest timewasters.
It happened to digg and reddit, I'm perfectly happy with people voicing concerns where appropriate to prevent HN from just becoming another reddit/digg/fark/somethingawful/thechive