Is it possible that GPT-4 already had this new optimization somewhere in its training set? The optimizing patch DeepMind published on has been floating around for several months at this point…

Edit: I found the merges for sorting (Jan 2022) [1] and hashing (Jan 2023) [2]. Both of these are very plausibly in the training set for GPT-4, which was frozen sometime in March 2023.

[1] https://reviews.llvm.org/D118029

[2] https://github.com/abseil/abseil-cpp/commit/74eee2aff683cc7d...