The fact is, we shouldn't need AMP to refocus our efforts in favour of the user... we should be making lean and performant pages as a matter of course.
I do agree that if Google really want to do anything to help the cause, they would more heavily lower the ranking (or completely remove from their index) any sites that don't meet basic performance and readability thresholds.
AMP is clearly a play into their own walled garden.