Believe it or not, an incident in 1917 involving Arthur Conan Doyle, inventor of Sherlock Holmes, is instructive.
The "Cottingley Fairies" were imagined when a couple of teenage girls took photographs of themselves with pictures of fairies. [1]
The thing that is important is that, to my eyes and I think to a typical person of this era, these photos of girls with cutouts of fairies look like ... exactly that. When I first saw these pictures, I couldn't believe anyone could be fooled by them. But this period, circa 1917, was a period when photos only recently had appeared and so only recently had photo fakes. So the skill to spot the difference had only recently appeared.
Which is to say, I'm pretty sure the author is correct that deal of the OpenAI text generated isn't intelligent text generation but stuff with enough of the markers of "real" text that people might not notice it if they weren't paying attention.
Moreover, I strongly suspect that if this sort of sham were to become more common, the average person's ability to notice it would increase significantly.