I was a fan of XHTML for a while, largely because it made it easier to use XML tools to verify my documents. But that became a non-issue as HTML parsing tools (specifically what I could use in Ruby tools while developing) got better.
I wasn't serving my pages as application/xml, and on consideration of what Ian Hickson wrote (http://hixie.ch/advocacy/xhtml) I went back to using HTML 4.
I'm puzzled by suggestions to use HTML 5. it seems a peculiar reversal from the "design with Web 'standards'" mantra. It's not a W3C recommendation, it's still a draft.
If XHTML is not working out as planned, why not use HTML 4.0.1, a current W3C rec that works?