I consider all the manual curation effectively a form of RLHF that can be imposed automatically later on. We saw how much this can improve a raw LLM by looking at the output of ChatGPT. Otherwise, the criticism of LLMs being just glorified autocomplete machines isn't that far from reality. In other words, it is just an expected requirement for LLMs to be effective.

You are probably right that lysozyme is an easy target and may have large sequence variety between homologs so saying "very different" for 30-40% is not correct. But that is only in the context of biology and protein structure and function. This is an LLM trained on primary sequences only. It doesn't know anything about the folds or domains or functional sites (unless I am wrong and those are part of the metadata fed to it during training). Yet it did learn enough to generalize to the point that even with only 30-40% identity, it still produces soluble proteins with the same function. I am sure you know that at 40% differences, one protein can be in an entirely different superfamily from another. So it is still an impressively low identity score.

Also, I think it is more appropriate to compare the amino acids to things like the alphabets than vocabs. Domains would probably be an equivalent to LLaMa vocab.