Asking about work seems to define the direction of the conversation, and some people don't work, which makes it a little awkward. Everyone does _something_ so this gives whoever you're talking to an opportunity to pick what they want to talk about - "I work in marketing" or "I'm trying to get my black belt in karate". Then you just starting picking around that topic and ever so often drop in a short annecdote or story of your own (followed by another question - "has that ever happened to you?!") to keep the conversation going.
And I know people for whom this question is really anxiety inducing.
I’ve known people who have tended to respond to that question with sarcasm or hostility. They also tend to be the most socially shut out. And then they wonder why people don’t like them. It’s one of those things where you expect society to be better but it isn’t. The choice then is either to become bitter or to redirect it into something more positive and gracious.
If one is unemployed or one doesn’t have a career a good neutral response is “I’m in transition”. Any number of creative responses will work, and will keep the conversation going. But one responds with hostility because one hates that question, one will invariably pay the social costs and may be killing potential budding friendships. I can tell you, most of the time it is not worth it. The downside is greater than the upside.
On the initiator’s part, a good alternative to it is “what keeps you busy?”
I would encourage others to avoid asking "what do you do?" As well. Ask what the person's favorite book is, their favorite trip/place, their favorite memory from childhood, etc. I promise the resulting conversation will be much more interesting and also give you a better view of the person.
"Tell me about yourself" is something else entirely. I look forward to your description of how it goes with strangers.
Edit: by badly, I mean badly for me. I'm getting a bunch of downvotes above - obviously this is just my opinion, based on my interactions with people over the last few years.