Is AI also superior to humans at black box challenges and attacking actual targets on the internet? That seems like a really important question.
The reason LLMs can do CTFs so well is partially because the challenges are usually designed to avoid wasting time and to introduce a single concept without noise.