If Google just flat-out said they would penalize sites in rankings for poor performance, then management would very much prioritize it.
Same thing happened with responsive design a year or two ago and now https this year -- my clients never cared about doing either of these when presented with the choice (since it would cost more money), but the second they received and email from Google about it affecting their search rankings (or showing the site as "insecure" in chrome, as per the recent push for https), they were asking us to do this for them.