That's not to say that LLM's can't mash together reasonable new recipes for certain audiences, especially for variations on popular modern standards, but the idea that their working space of recipes is exhaustive across all cuisine and tastes is absurd.
Predictors don't just grok what is explicitly stated in a dataset but also what is implied by its structure.
In a language model trained on protein sequences alone and nothing else, you will find biological structure and function emerge in the inner layers.
https://www.pnas.org/doi/full/10.1073/pnas.2016239118
Language Models are predictors. If the text you give them is the shadow in Plato's cave, they don't try to draw shadows, they try to build walls. They are trying to reverse engineer the computation that must have led to that output. They will not stop at "surface level similarity" or "plausible" by choice. They will train until they have completely succeeded or until the architecture or data fail them.
With a capable enough architecture and sufficient data, there is no recipe a predictor couldn't divine with enough training.
For the perfect predictor of recipes, Is the architecture capable enough ? Is the data sufficient (both variance and quantity) ?
I'm not sure but this is not a question a cook can answer.
The OP claimed
>Actually generating a recipe is obviously much harder, and should be effectively impossible unless you include a humanoid robot that can cook and taste.
This is just false. And you don't need the hypothetical perfect predictor (just a capable one) to see it.
This is the fantasy of GenAI boosters stated concisely - that these statistical methods present a new objectivity that avoids the shortcomings of human subjectivity. Of course, these web-scraped models are distilled subjectivity in aggregate, subjectivity at scale. ChatGPT is made of people!