They make that claim because what CEOs write down could easily be written by a Markov Model, nevermind an LLM. If that was the primary value a CEO brought to the company they'd be right.

The most important thing a CEO brings is relationships. LLMs can't do that (yet).

Post script: there's still a chance that LLMs replace CEOs due to LLMs being easier for the board to influence/control.

because 95% of the time it's "follow the herd" BS + always defer to the lowest cost. the rest is schmoozing, making people feel comfortable, sounding confident, and working the soft skills.

chatGPT always sounds confident, and it's not hard for it to calculate the lowest possible option and take it.

because ceo bad/dumb and eng good/smart