Your premise is that these are not editable models, but you restrict that to a situation that is not true in practice. These models are constantly changing based on human feedback, updated information, config changes, and weight manipulation.
If you follow the conversation around LLMs, one of the top issues people have is inconsistent responses and the models changing under their prompts and breaking them.
> We find that meta-in-context learning adaptively modifies priors over latent variables, ultimately leading to priors that closely resemble the true statistics of the environment. Furthermore, our analysis reveals that meta-in-context learning can not only be used to change prior expectations but is also capable of reshaping an LLM’s learning strategies
This was done using OpenAI's public facing APIs
Even if it is not a permanent "edit" it still influences the model at the lowest levels
> the practical constraint of a finite context window coupled with meta-in-context learning’s rapid prompt length increase
The model is not influenced by the added context. The answers are. The abstractions, or, what the LLM "knows", are in the model itself, whereas the answers are just byproducts.
There is an ongoing related discussion on LLMs lacking world model. One could say LLMs do implement a computable model, albeit a dead, static, non-editable one.