But just because it's in the training set doesn't mean the model retains the knowledge. It acquires information that is frequently mentioned and random tidbits of everything else. The rest can be compensated with the 20 web searches models more get. That's great when you want a react dropdown, but for that detail that's mentioned in one Russian-speaking forum and can otherwise only be reduced by analysing the leaked Windows XP source code AI will continue to struggle for a bit.

Of course AI is incredibly useful both for reading foreign language forums and for analysing complex code bases for original research. AI is great for supercharging traditional research