I am fighting against people doing this every day, and we've made it extremely clear that your computer and processes running on it are an extension of you, a human, who is responsible for the actions on your computer.
Does it matter if the developers understand the system they maintain?
I guess that depends whether the genAI maximalists turn out to be right.
Do you have people frantically trying to reverse engineer a basically unfamiliar-to-them code base at 2am while bleeding cash?
I've certainly seen people on this site defend that idea vociferously.
I won't tell you it's impossible - Claude does do legitimately amazing things.
It does not, however, seem to me to actually have deep understanding of what it's doing. "Amazing" does not mean "admirable" or "trustworthy."
Are they maintaining it if all they do is type prompts into Claude?
They may be doing a poor job of it - time will tell.
But, at present, they're still the ones merging PRs and deploying changes to production.
If a person in a Chinese Room doesn't actually understand Chinese, though the system as a whole behaves as if it does - can we say that the engineer merging PRs and deploying to production understands the system or maintains it?