I post on HN or Reddit when I feel like adding a point of data to a debate. I post on MeFi when I want to talk to my peers.
(Admittedly, even sites like MeFi can't compare to talking with people in real life. There's something to be said for taking control of someone's locus of attention while you're talking, which you simply can't do when you have to talk in discrete, comment-sized chunks.)
None of italicize everything quoted, "put quotes in quotes (especially for long passages)", nor > prefix quotes with a greater-than sign work particularly well IMO. Though I suppose they generally suffice.
Chowhound has a nice compromise where it's threaded but all the comments you've already seen are folded by default. I'm sure there must be other communities that do the same thing.
I've seen flat discussion style particularly on G+. While I strongly feel it doesn't scale well, what you do get for small discussions is a conversation which feels a bit more like a handful of people talking in a room. Assuming people are clueful enough to read the preceding comments and take them into account (observed more in the breech), there's more of a chance for side conversations to work their way back into the main stream.
The problem is that, as with a discussion in a room, the discussion tends not to scale well. Past a half-dozen to a dozen primary participants, continuity tends to be lost. Threading allows for break-outs of specific interest. Sites such as reddit (with RES) that allow expanding or collapsing of specific sub-threads (RES's "hide all child comments" is great for busy posts) are particularly useful. I find that navigating reddit threads on busy posts to be somewhat challenging: too many open/close options and long threads are cut off too soon, particularly when you're viewing just that thread.
HN wants badly for a specific "response to comment / post" feature. You can view your threads but have to find and respond individually, and responses to anything but recent comments get lost.
HN: hn_unread by Jens Wille http://userscripts.org:8080/scripts/show/124778 (also a Chrome extension, if you search for hn_unread)
Mefi: MetaFilter Scroll Tag by Plutor http://userscripts.org:8080/scripts/show/23270
Greatly improves my user experience on both sites. I suppose this kind of thing isn't built into the sites themselves because of concern about the extra server or client storage needed. Super useful though.