To add to the downsides you have mentioned, we practically don't really own them. I mean, almost.
I buy a fair number of ebooks, mostly technical, and mostly from O'Reilly. I love that they're DRM-free; in my case I take the PDF version, but they have other formats.
Pakt has a very low-intrusiveness form of "DRM." They print your name on every page of the pdf that you buy, but it's otherwise unrestricted.
I don't think Pakt or O'Reilly is going to come after you for sharing an ebook with a family member, and I've given a few copies of O'Reilly books to colleagues, to evangelize both O'Reilly and the tech in question.
It's a pretty reasonable compromise.
"Buying" an ebook currently is effectively renting it. If/when that changes, if/when I can actually buy an ebook, I will start buying ebooks.