The github [0] hasn't been fully updated, but it links to a paper [1] that describes how the smaller code llama models were trained. It would be a good guess that this model is similar.
Without knowing any details, I'm almost sure that they did not train it on internal code, as it might be possible to reveal that code otherwise (given the right prompt, or just looking at the weights).
Would be a really bad idea to train on internal code I would think. Besides, there is no shortage of open source code (even open source created by Meta) out there.
Correct that it’s a bad idea to train on internal code. However surprisingly there is a shortage of open source code. These models are trained on substantially all the available open source code that these companies can get their hands on.
people are able to extract training data from llms with different methods, so there's no way this was trained on internal code