Well I feel like this is because a better system would distill such learning into tokens not associated with a human language and that that could represent logic better than using English etc for it.
I don't have the GPUs or time to experiment though :(