More importantly it's about where you spend your time. Pg's a busy guy but HN is good advertising for his business. He does spend time optimising the whole thing though, technology and community. Just look at the experiment, now finished, to see whether displaying average comment scores would alter community behaviour in a desirable way. These and the other tweaks - downvoted comments fading toward background colour, the front page algorithm, the effect of flagging and associated moderation, when people get voting rights - are the things which improve the website. There are plenty of places and forums with which are standards compliant and have a lot of seemingly whiz-bang features, but their commentary sucks and their community has an aggressive tone and is ridden with trolls. Too many people here think it's all about the technology when there are other factors.
Even if pg held a "redesign HN" competition, to which many would respond for free, reviewing the submissions and implementing the changes into the server would be a waste of time compared to the opportunity cost of trying out something else to improve the community and defend it against degradation.
EDIT pg made the same comment about tabular data in the linked thread.
gmail lists emails using tables. What's wrong with that?
gmail is a table - columns are email properties, rows are documents.
HN is a list.
gmail is a table - columns are email properties, rows are documents.
HN is a list.
I believe PG argues a list is a smaller, single column version of table, and that nobody should care about the output of a compiler anyway.
I'd argue that the HN header is a cell, so are the links at the bottom of the page, and this is clearly a case of tables for layout. Also there may be things that parse the page for different audiences, who expect lists to be used for lists and tables to be used for multi columned data. Also, it's not hard to fix.
Reading the previous discussion, things got quite nasty last time, so I won't fan the flames and say anything more.
Now whether or not HN's front page is tabular data (my vote is not) is left to debate.
That HTML has specialist elements for lists is nice, but I don't see how that implies that a table with one column isn't also appropriate for a list.
Using css might make it easier through @media queries to adapt the site to different devices (I actually only care that it looks good, so a table based solution is actually fine with me), but nevertheless the site does need a revision.
edit: Another example, on reddit I have to use m.reddit.com to make it light weight enough to be tolerable whereas HN works faster and more cleanly without losing the capacity to vote and comment.
I couldn't care less about divs vs tables or spacer images. However, I really feel tree structured data should be encoded as such, especially when the document itself is tree structured. It seems more difficult to do it otherwise.
Not that it really matters that much. I have bookmarklets that add nesting and trails, but it was just annoying to make them. I realize I'm an edge case, and overall HN is hardly as impacted by it's markup as it is by the community.
Of course, you have all of the information we'll require to make this decision, so you'll have to answer all of our questions.
Oh, sure, we could just trust that understand the options and your own specific situation and have already weighed everything and taken the path that makes the most sense. But how would that be any fun?