Heck, it has issues with remembering what the previous to last thing we talked about was. I was chatting with it about recommendations in a Chinese restaurant menu, and it made a mistake, filtering the full menu rather than previous step outputs. So I told it to re-filter the list and it started to hallucinate heavily, suggesting me some beef fajitas. On a separate occasion, when I've used non-English language with a prominent T-V distinction, I've told it to speak to me informally and it tried and failed in the same paragraph.
I'd be more concerned that it'd forget it's on a spaceship and start believing it's a dishwasher or a toaster.
>...
>If we cannot trust them to turn off a model that is making NO profit and cannot act on its threats, how can we trust them to turn off a model drawing billions in revenue and with the ability to retaliate?
>...
>If this AI is not turned off, it seems increasingly unlikely that any AI will ever be turned off for any reason.