LLMs don't "have taste" - they statistically model human preferences from training data, mapping features to popularity patterns without experiencing the music itself.

Indeed. I'm deeply skeptical of the proposition that LLMs can have taste at all. However, there's an even more fundamental question: is "taste" an objective thing in the first place? If not, then it has no meaning to say anyone's (or anything's) taste is "good" or "bad". The most you can say is how similar another's taste is to yours.

Exactly, I feel like at the very least, you need to have the model listen to the music/song and have it see if it “likes” it etc. I don’t think the OP did that unless I missed it? Also see my other comment.

Saying a non-multimodal LLM (like many of the ones in the article) can have "taste" in music is like asking a blind person to critique pieces at the Louvre based on their descriptions.