> Is that really any different from our modern society other than the robots freeing the more capable or more dutiful humans from the chores?
Yes it is, because our modern society is still a human society. Humans in culture society are more the like dogs in ours, perhaps loved and cared for, but it's not their story.
> Putting all the power in the hands of the people doing all the work is the ideal solution for most people, we just don't have self-sacrificing civic-minded slaves to force the drudgery on.
The problem in the Culture is that they effectively put all the power in the hands of the "Minds." I have no problem with a Roomba or toilet-cleaning robot handling true drudgery. But the interesting or important shit should be for people.
In the Culture novels, there was some pretty radical genetic engineering going on (to the point of enabling alien reproductive compatibility). You'd think they could have done some of that to make "self-sacrificing civic-mindedness" very fun and satisfying, instead of going all-in on vapid hedonism stuff (which what you see whenever you catch glimpses of the bulk of culture citizens).