To me, the reason is simple - it got big.
When a site/group is small there's little motivation for bad actors, advertising spam, fake accounts, etc. Why bother making spam bots on site X when site Y has 10x the userbase? Why try to make people angry with poor headlines when it won't reach a large audience? Why try to do propaganda when you could go elsewhere?
Smaller communities are generally also tighter knit. In a small group there's more notable reputation that sticks. People can't get away with being jerks.
As well, there's the corporate side. Reddit has to think and act very differently than HN does. They need to think about their profitability. What do advertisers think of their site? What kind of opinion does the public have of them? They can't afford to let things be loose and wild, they need to carefully control the content or the hammer could come down on them hard. That means a lot of accusations of violating "free speech".
If HN suddenly got as popular as Reddit did, the exact same things would happen to HN as Reddit. 0 doubt.
So no, it's not directly the format of reddit that is the problem. It's just the size.