I never had a problem with the depots, though they definitely weren't classy places. If you knew the towns along the route, it was usually easy to walk to a better restaurant, store, etc.
I continued to ride the bus long after my economic situation improved, because I actually enjoyed it. I met a lot of interesting characters taking the bus. I've heard a lot of "Greyhound stories," smoked a bit of weed with strangers, and found myself in some unexpectedly cool situations. Always an adventure. Air travel is so sterile in comparison.