I think XHTML failed due to a lack of tooling. It was an attempt to force HTML into a formal "compiled" form which machines could understand. Thus what was needed to authors was to use tools to write "source code" (like Markdown) and then "compilers" to transform that into XHTML. Then you needed to be able to verify that it was correct (otherwise you'd get an error message).

As far as I know those tools were either immature, nonexistent or proprietary/expensive.