Artie Shaw was another guy who walked away from fame, and he was a lot bigger than Tom Lehrer:

https://en.wikipedia.org/wiki/Artie_Shaw

Gerry Rafferty (probably more commercially successful than Lehrer, at least) also didn't like the business or performing aspects of music.

Bill Murray said (paraphrasing), "I always say to someone who wants to be rich and famous, 'try being rich first, and see if that doesn't get you 90% of what you want.' Being famous is a 24-hour-a-day job."