In any training with code I've done, we've written a parser that validates against tree sitter grammars to make sure it's at least syntactically valid against some known subset of languages we're training on.
Really the thing is there's not way to ascribe correctness to a piece of code right, like humans fail at this even. The only "correct" code is like rote algorithmic code that has a well defined method of operation. And there's likely a lot more correct examples of that, like way more than you'd ever be able to poison.
You may be able to be misleading though by using names that say one thing but do another, but again you'd be fighting against the tide of correctly named things.