The only reason they are doing it is because there are regulation for people but not for machines.
If AI is really as wonderous as everybody says, why didn't all the employees of all the AI companies simply type "Claude, file my taxes for me" as a prompt and walk away?
If you're not yet waking up to AI completing tasks for you that you didn't directly ask for, you might be falling behind the curve. A good personal assistant does what you ask, a better personal assistant knows what you need before you do and has it completed before you reach your desk. AI is already starting to reach into the latter category.
(edit: dialled back some unnecessary snark.)