It does not generate secrets. The Twitter conversation does not mention that word.
Most certainly, it regurgitates secrets it has seen on crawled repos. Can the title be adjusted, please?
They are being generated. It's a Generative Pre-trained Transformer. It's generative. It generates things. That's what this technical term means. It's correct.
The generated output has building blocks (characters, words) which themselves are not "generated" in the sense that they are novel.
A random number generator generates numbers. It doesn't generate the digits used to represent the random numbers. Those are taken from a set (such as 0..9) and just "used".
But it's not generating the secrets, it's generating code that contain those secrets.
For a comparison point, afl fuzzer actually generates never-seen, valid JPEG files "out of thin air", see http://lcamtuf.blogspot.com/2014/11/pulling-jpegs-out-of-thi... and matching conversation https://news.ycombinator.com/item?id=8571879
It would be interesting to combine both approaches and (interrupted by the sudden singularity ;-).