That's the entire essay's point and I largely agree with it.
Its why I think current LLMs are bad writers, not because their prose or ideas are off, but AI generated writing does not have the same quality of robustness from ideas that are thoroughly vetted through the author's editing of it.
At the end of the day, good writing takes a lot of time thinking through what you are really saying and standing behind it. LLMs cannot do that for you (yet).
LLMs can certainly be a helpful tool, mostly by unblocking authors via creating prose, honing in on accurate expressions, researching edge cases and suggesting arguments or counter arguments.
But the craft of sharpening an idea to a very fine, meaningful, well written point is something that is still far off.
Of course, until the next research paper completely proves me wrong.
Step 1. RLHF on real time edited documents Step 2. Profit??