I suspect why the majority of people don't skip is because the ads themselves aren't terrible. The worst aspect of ads is repetitiveness, which is why Hulu with ads is unwatchable. Of the podcasts I listen to which are typically rated high on popularity, the ads typically aren't repetitive: they feel ad-hoc since they are done differently per episode, where it's almost like it's part of the entertainment. The only repetive ads I'm aware of are from Malcolm Gladwell's Revisionist History, which start to get tiring because they are exactly the same episode to episode.