There are definitely not enough pins to do straight pin for pin HDMI through lightning. Of course, if you are going to go to the trouble to do what they did, you may as well bitstream it through the adapter, ...
However, MHL-through-Lightning might have prevented the AV adaptor having a daisy chained lightning socket of it's own. Maybe that's why Apple jumped through these hoops.
The cynic in me says this is an end-run around any DRM issues giving access to that 1080p bitstream could introduce: instead let's just output a degraded, upscaled stream and call it HD.
This allows them to support different formats in the future, for instance, say when HDMI isn't popular.
The 30 pin connector didn't have HDMI on it at the time it was created and they have repurposed pins over the years regularly.
Here they can have a much less complicated solution-- a standardised serial bus over which they can stream any future protocol they want to support.
It's a lot more future proof.
It would have been just as easy to use micro-usb 3.0, which had plenty of bandwidth, and do the exact same pin or format conversions.
Standardized serial buses that are less complicated already existed. What exactly do you think USB 3.0 is?
If the only reason for lighting was "a general purpose next generation serial format", then it is definitely a horrible idea.
Of course, none of this (except DRM concerns) answers why they'd not bitstream the format and convert it to HDMI signaling instead of doing the weird crap they did here.