It has been a few years since I regularly shopped at Whole Foods, but when I did my impression was that at least some of their reputation for high prices came from the fact that they had a wider range of minimum to maximum price in a given product category than a typical supermarket.
I found their produce and "365" store brand items pretty comparable with other shops, but they also stocked high end versions of many things that you couldn't buy elsewhere, sometimes with eye-popping prices, so you'd have to be more selective about what you picked up off the shelf.
Taking spices as a random example: a regular store might have own-brand cumin for like $0.75/oz and McCormick brand for $2.00/oz. Whole Foods store brand is likely somewhere in that range, but they'll also stock Morton & Bassett for like $4.00/oz, so if you're not careful you could pay 2-5x what you might have elsewhere.