How precisely rubberband works is something I've not looked into deeply, but the end result is increased CPU load and less choppy audio at high speeds. If you throw together a little script to switch back and forth between rubberband and scaletempo in realtime, I think the difference is pretty dramatic.
It's just so much faster, more visceral, and more random access to read that I've basically just completely abandoned the linear presentation modes (audio and video).
Podcasts also tend to have a lot of side chitchat, breaks etc which don't add anything for me. I know some people really like that; I've hear it makes the conversation seem a lot more personal and connecting. Great, just not for me.
Video/audio are such timewasters if you can read at a decent rate.
(2x is generally my max, unless speed-seeking.)