As a developer becomes better, they become better than an LLM, being able to deal with more complex things than what an LLM can handle. Some people will not be able to pass it, but others will.

When there will ever be AGI (I don't think this can be achieved with the current architecture, it needs another AI breakthrough), then we might not be able to surpass it, much like chess currently.