It's possible you're correct and there's some circumstance controlling things we can't see or don't understand, but that's not a testable idea.
Environment plays a huge role, but it's somewhat unpredictable what lessons are taken. One could also say that it's not unanimously clear what makes a good environment; it's likely not simply about having resources and lacking conflict. Likely an amount of scarcity and conflict are needed.
All this isn't to say that kids are all born the same, but I don't consider such differences to generally matter as much as the effect the environment can have.
Nonetheless we know that humans are neural networks. These are formed. There is something inbetween the initial setup and your human in prison which lead to this or something missing which lead to this.
We can't even properly fix depression and co.