I am with you but we are a small minority. I literally don't know anyone IRL who cares about DRM. I doubt people like us factor into their decision making at all.
Yes, not many people know what DRM is but many of the effects are noticed when it ruins their experience.
For me, I bought a single DRM book against my better judgement because it was the only available way to get it quickly at the time. Predictably, I eventually lost access to it and never bought another one since.