Do we? I do a lot of mental arithmetic as a software developer. When I was an exchange student my host-dad was an engineer and he was incredibly good at it. As he was thinking through a problem he'd do all sorts of ballpark math to get a feel for things. Eventually, yes, the numbers would all get crunched properly. But even then he'd also be doing the math mentally to make sure that something hadn't gotten screwed up. A person who uses a calculator for everything starting from a young age just doesn't develop that skill.
And I think there will be a similar problem with ChatGPT. Can an excellent writer get a performance boost starting with ChatGPT? The jury's out, but it's not impossible. But can one become an excellent writer by tarting up LLM-generated prose? I doubt it. Learning to write is a process of sweating the details. Especially so when the output is an essay. Real higher-level creativity requires foundational work. I think at best you'll get the sort of content that comes when an uninspiring politician hires a mediocre ghost writer.
That said, a lot of writing is bullshit. And I mean that both in the Frankfurt sense ("speech intended to persuade without regard for truth") [1] and in the Graeber sense ("completely pointless, unnecessary, or pernicious") [2]. And that's where I think ChatGPT will shine. Marketing copy to fill in the "lorem ispum" space on the product page? ChatGPT to the rescue. Long sections of TPS reports that nobody will read? ChatGPT again. And once they expand ChatGPT to being able to make PowerPoint decks, a zillion ineffective middle managers will be able to return their focus to Candy Crush, harassing the interns, and creating pointless meetings.
I think that's also bad, of course. Lowering the cost of bullshit production looks like a net societal negative to me. But it would be successful in the traditional tech industry metrics of "gains users" and "produces revenue".