levitatorius

I know when we will reach the next level of AI. It will be when a user asks it to make paperclips gets a response: "WHY?"

If the book speaks to you (let's say it's an audio book), and answers your questions as they arise while reading a book, is it conscious?

The post resonates deeply with me. I am a health professional in diagnostics and through the years I have observed different extremes in approaches to solving diagnostic challenges - the one extreme is to rely on "knowing", the other on "thinking/reasoning". The former is usually very fast, but not easily explainable - just like pattern recognition. The latter was slow, but could give a solution from "first principles" and possibly not described before. Of course it's a spectrum and the thinking part requires and includes the deep enough "knowing" part. One usually uses both approaches on daily work, but I have seen some people who relied much more on knowing than thinking/reasoning, sometimes to the extreme (as in refusing to diagnose a condition on their own because they "have not seen this before").

Yes! If algorithm is conscious (without being alive) then the eaten magic mushroom is also very conscious, judged by it's effect on the subject.

This! One simple argument is that language is NOT a magical reasoning substance in itself, but a communication medium. It is medium for passing (a) meaning. So first there is a meaningful thought (worth of sharing), then an agent puts a SIGNIFIER on that meaningful thought, then communicates it to the recipient. Communication medium can be a sentence, it can also be an eyewink or a tail wiggle. Or a whistle. The "language" can be created on the spot, if two subjects get a meaning of signifier by intuition (e.g. I look at the object, you follow my gaze).

So the fallacy of the whole LLM field is the belief that language has some intrinsic meaning. Or if you mix the artifacts of language in some very smart way, the meaning will emerge. But it doesn't work if meaning occurs before the word. The text in books has no reasoning, it was authors. The machine shuffling the text fragments does not have a meaningful thought. The engineer which devised a shuffling machine had some meaningful thought, the users of the machine have same thoughts, but not the machine itself. To put it another way, if there was an artificial system capable of producing meaningful thoughts, it is not a presence of language which produces a proof, it's communication. Communication requires an agent (as in "agency") and an intent. We have neither in LLM. As to the argument that we ourselves are mere stochastic parrots - of course we can produce word salads, or fake mimics of coherent text, it is not a proof that LLM IS the way our minds work. It is just a witness to the fact language is a flexible medium for the meanings behind - it can just as well be used for cheating, pretending, etc.

I think that there is a possible misconception, that evolution allows (or results in) only in necessary features and thus all phenomenons must be a consequence of evolutionary advantage. Yet, if genetic changes are random (at least some of them), some features could exist just because there was no evolutionary pressure to lose them.

Yes, you mean why beauty has to be utilitarian :)

What do you think is missing then? Unsupervised learning == general intelligence?

It's fascinating that we have simple primitives or notions of analysis, deduction, causation, yet no artificial system where those features of intelligence emerge on their own.

Aaaand the ultimate question: if electric fields are controlling the neurons, what (or who?) contols the electric fields?

I am a practicing pathologist and I have seen many attempts and publications to use ML in pathology, which all lack in these aspects: 1. ML is trained on simplified sets (preselected ROIs, limited choice of diagnoses), 2. ML is biased by the experts who labeled learning sets, 3. there is no obvious process of learning from failures after initial training, 4. who is responsible in case of ML error with substantial consequences for patient? The first point is especially for the lack of better word.. wishful. In the daily practice we are used to account for "things unexpected" - non-representative biopsies, parasites in tissue where tumor was suspected, foreign body reaction from previous operations, laboratory accidents (such as swapped paraffin blocks of two patients), and so on (the list is much longer). We deal with it. That ML can discern between 5 most common diagnoses is fine, but it is rather narrow problem to solve.