HN's design currently delivers a solid performance and its content is great. Looking a bit strange to passersby is probably a good thing, I don't think HN's focus is attracting new people, especially new people who are only attracted by the visual design.
I have no idea why the visual design of Stack Overflow is called horrible. It's ok, not awesome but certainly ok.