That's the question. And the answer is that different people prefer different things. I for example vastly prefer to listen to a full releases, that would be "shuffle by album", others want it to be as non-repetitive as possible while staying in a genre, yet others would want to stay in a certain time period of music, but shuffle between genres occasionally, etc.

Shuffling music on a per song basis is an interesting problem, with the ideal solution being something a top class DJ would do, e.g. matching one characteristic of the two tracks (e.g. Tempo), but updating another (e.g. Timbre).

But even with that implemented, not everybody likes the same thing. E.g. I love to hear new music that I don't know yet, but as a DJ my experience has been that many people like to hear music they know. Meaning app developers have to make the right choices available and those choices are under the hood far more complex than how you can sort a list of tracks by artists.