In pouring over the relevant RFCs fairly recently in order to implement my own client, I learnt (though didn't implement - raised a not implement exception, and I've yet to receive exceptional mail) that you don't actually need HTML for inline images.
Tl;dr a multipart message can include an image with an inline 'content disposition'. It's intended that clients show it inline, but I suspect many don't. Like I said I haven't received any yet anyway, so it's probably not that consequential.
Another quirk I'm aware of, by implementing my client quite unforgivingly, is that many emails are structured with the attachments only 'tied' to one alternative, often the text/plain media type, so if you interpret the parsed tree structure of alternatives/attachments literally, the HTML-formatted message might refer to an attachment (I mean a PDF say, not an embedded image) that's actually only available on the plaintext version. Having never encountered this before, I'm pretty sure other clients (like Gmail, Fastmail) present all found attachments alongside whichever alternative is viewed, not taking the actual email too literally (as I presently do).