> At least you know I am human.
...we technically don't always assume that people we speak to on the internet identify human, since we don't ourselves, but we at least know that all host bodies (that we know of) are physically human. And we also know that all consciousnesses we've spoken with are "human" in nature as in "formed in a human brain".
> Hard to measure internal thought and understanding. Hard for you to even prove I have it.
Indeed, but brain scans exist to show mental activity in response to all sorts of things, including unprompted internal thought. You've heard of the EEG test for ADHD, right? You place someone in a room all by themselves, put some electrodes on their head, and leave them there for around 25 minutes. Then you measure brain activity to see if it looks... uhh, ADHD-y. I don't know what exactly they check for, but I'm pretty sure it's some certain type of uncontrollable mind wandering that people with ADHD suffer from.
With a language model, you are computing the mathematical transforms that result in its output. And there's no evidence to show that there is any sort of internal thought that occurs there, especially since the entire thing is re-run from scratch for every token. Any "thought" or "understanding" can at best be encoded by mathematical weights in the layers, or something, but it's definitely not the same sort of understanding that biological humans have, or even most animals. The way they run is just fundamentally different.
And obviously a mere product of consciousness such as an internet comment can be difficult to tell apart from the output of a hallucinating language model. :)
-Emily