You did the right thing by adding noreferrer to their links. Knowing where people got to your site from without them explicitly telling you is a privilege, and it's one that they chose to abuse.
Besides, by your argument, connecting to websites at all is privilege. HN wants the privilege of reading their content (otherwise why not just block their site) without taking any responsibility for it.
Similar to how freedom of religion also guarantees the right to not have any declared religion.