If someone hires me as a secretary, and they ask me what is the lyrics of a song, there is no law that prohibits me from telling them if I know and I don't have to license the lyrics in order to do so.
If they hire me primarily to recite lyrics, then sure, that would probably be some manner of infringement if I don't license them. But I feel like the case with a language model is much more the former than the latter.