Note that LLMs aren't useless, since they have other strengths which can make them equal or better than humans in certain applications. For instance, they can tirelessly review code for silly mistakes, making them useful in cybersecurity.
Anyway, to answer your question, I was pointing out that the request for "important knowledge and moral lessons that larger society needs" and "communicating solutions to how to live as a human" as well as lamenting that "We are adrift without instructions" is actually a request for religion. Religion is how to make sense of how to live. So the comment is asking for religion and rejecting religion at the same time.
At the same time, I was pointing out that one religion in particular has directly addressed the point that "writing is not the goal." The "word"—or "logos" in the greek—is the reason, the logic, the rational principle of the cosmos behind the words. It points to that need for a solution for how to live, as well as showing there is a real answer.
Yet a very vocal group advocates integrating LLMs into every aspect of their lives, and the other group watch in dismay.
On the other hand, I used a sentence form used by LLMs in a comment during a morning-brain moment, and an angry mob tried to crucify me with torches and pitchforks.
Many people love to generate text with LLMs, but most of same crowd apparently hates to read LLM output. An interesting phenomenon.
On the other hand, I have friends who play that game, and I don't talk with most of them, because their life views and mental states continually drift to a state which makes no sense whatsoever for me, so we grow apart.
It's a good concept, but for some reason I thought of a military movie I watched recently, where someone was going on about how they were third generation military. And how their friends and family were military. Sort of the same kind of siloing there.
I don't read info on books ahead of time, and sometimes don't even read the summaries. I've read a LOT of scifi, must be in the thousands of novels by now, but I was completely unsure what was happening for the first 4 chapters at least, and in a very, very good way.
It can be a very engineer thing to point at a vanilla Doxygen dump and call that thing "documentation".
It's like the car manual (Do they still ship with one?) where every detail is laid out. It's not important for daily driving, but when you to fix an electrical problem, that's the reference you need.
This only works in broken companies where leadership is already overrun with other status climbers.
If someone tries this trick at a functional company and is discovered, their reputation takes a huge hit that can take years to overcome, if ever.
At high performing companies reputation is everything. It can be a real problem to even get people to work on difficult or risky problems because they’re afraid of having a failure attached to their reputation.
Throwing unreviewed LLM slop around would be a reputation death sentence if your peers are good at what they do, because by now we all know that it’s easy to generate long reports with LLMs. The secret is out and anyone who follows technology knows it.
There are some companies where you can LLM report-slop your way to prestige. Those are not companies you want to stay at.
The promotion of LLMs is, I believe, basically a referendum on how many companies are broken in that way. So far, the adoption statistics are looking like it’s most of them.
Are you sure it's the same people? I feel the opposite: it's the same people who happily read tons of LLM output themselves and send some of it to others.
This applies even in "low" art. AI generated imagery from mainstream models tends to be so ugly it detracts from effectively communicating what the prompter intended.
I stopped sharing my photos publicly to stop supplying training data for image generation models. I don't want to feed that machine.
Additionally I am so, so ,so sick of reading claudes overly wordy technical writing ish bullshit. Its exhausting trying to find the fuck ups or hidden assumptions in the pages of text it produces for everything.
Every time I try and search for ways to make it stop I just find people trying to save tokens which I don't give a shit about. I just want it to communicate things to me without having to constantly read 4 paragraphs, dump the context or fork it or start a new session so it "remembers" or re-weights that is was told to be concise and terse in like 7 different places. Maybe I am using it wrong.
9 pages about how it felt to be a kid eating the dish, for 7 ingredients and a one line recipe.
- Do not include opinions, just official documentation.
- Add all references you have used as a bibliography section at the end.
That’s not meant to be glib; Google and friends are pretty trash now.
This means it finds me a single video which is not in a language which I know, or similar.
It's not the exact same asymmetry, but there are parallels.
It's even more than that, I think. It's the intent to reach out and transfer to others something that you experience internally. It's a sense of purpose motivating that content and substance to be packaged up and communicated in the first place.
Sometimes LLMs will slip up and say things like, "I see this working with people all the time." And it REALLY breaks the illusion that the text they extrude means anything. Because obviously there is no "I" and no, the LLM does not have this autobiographical memory the text is simulating.
Oh no. I see the contrary.
One of my friends recently stated (paraphrased):
- Since I can generate code, writing it feels mundane now. I don't write code anymore.
I have seen people who never think but spend all their days querying LLMs for everything. Docs, help, howtos, system administration, coding...
What I see is an incentive for less work, less effort. Not income or status. They don't spend their "saved" time, if there's any, for personal activities or self-improvement either. They just do things with less effort, and just live as they lived before.
It's brain numbing.
> I would predict shorter and weirder stories and meme style communication patterns outside normal syntax.
I don't share the same sentiment, yet I don't have idea what to expect. It has to get darker to get brighter again.
We're just starting.
point is: maybe llm output is best not shared. maybe, like a dream, the only audience who can appreciate the content of the writing is the first party
* As a reference; people can look look up just the algorithm they want
* As a token that says “I’ve thought about this problem”
(The former is somewhat a subset of the latter. All of the stuff they didn’t read when they used the work as reference was there to provide them confidence that the algorithm they plucked out of the thing had some thought behind it).
but when my coworker pays to create documents on topics he hasnt considered, for ideas of projects which dont exist, it feels like an exercise in text generation and i become mildy put off for the onus of reading it. i know it was written without purpose and while there are possibly novel tech and ideas in there, it feels like being handed less than a rough draft because it lacks any aim and intention
now, i expect he would get much more from it. hes the first party audience who reads the doc and sees a valuable project somewhere inside of it
I'd also recommend having a look at the books published by this article's author. I will admit that I have never read nor heard of any of them. But come on, just look at the books published covers and the way the author introduced them and tell me that they are not industrial products engineered for mass commercial appeal.
Sure, at some level, there are some people who at least some times, cannot pecieve a difference. That does not prove that there is no difference or that the difference doesn't matter.
I think that's also where LLMs get the most dangerous. That ELIZA effect of telling you exactly what you told it but in a slightly rephrased way makes you think that you think that your ideas are genius. You've got someone "actively listening" and telling you whatever idea you just gave it was the best idea it ever heard.
That's not what you want from a serious writing partner. You want a writing partner to challenge you, to point out your mistakes and flaws, to make you better, to offer a truly different perspective to your own. You don't want a dark reflection from a "yes man" to spiral into.
That slot machine addiction gives me a lot of perspective on what LLMs are good at in coding versus what they appear to be good at. It's easy to characterize them as "over-eager junior developers" or "powerful rubber duck debuggers" but their biases are still your biases so long as you are the one writing the prompts. A lot of the "surprises" aren't intelligence but fun house mirror versions of the same biases you input into the system.
I keep referring to their coding capabilities as "legacy code as a service" and the addiction to the fictional equivalent only underscores that understanding of LLMs. You don't write the next great American novel getting a word salad of past great novels back full of the best and worst of your own writing. You don't write the next great American software getting a legacy code jumble of past great software back full of the best and worst of your own prompting.
At the end of the day I'm still a pragmatist and using the LLMs in my programming efforts, but I don't trust them as a partner, I use them as a tool. (A biased tool that I know needs a careful eye.)
That is if you only work with your LLMs / Agents in a "straight ahead" manner, versus using knowledge of the bias baked into AIs and purposefully invoking a specific type of subject matter expert for problems that require specialized knowledge. It is similar to using people skilled at what you need, invoke through the vocabulary and language of your prompts specialized skills that are not tapped until using the terminology only those specialists use, and like magic their skills are suddenly in-context and the AI acts accordingly.
I think narrative stories are a "brain hack", in the same way heavily processed food, drugs, clickbait, soap operas and TikTok are brain hacks, combined with some exclusivity status signaling games. They're just old enough to have gained a cultural cachet that more modern forms of entertainment don't yet have. Same for art, music, comedy and culture generally.
There is nothing wrong with those "brain hacks", if they make us happy and don't have negative consequences, why not use them, but we should recognize that it's what they are.
I think that an alien intelligence could communicate with us about Ohm's law and Rieman's last theorem, maybe even debate resource and power allocation strategies (democracy, monarchy, capitalism, socialism) with us, they could probably understand the pull of sports, even if they couldn't participate in it themselves, but art and humor would be completely lost on them and considered a weird human excentricity.
But like it or not, humans are not computers. They are animals with messy, complex thinking and reasoning that is impossible to fully describe, and incredibly complex societal systems created from their chaotic interactions. These messy people don't learn from datasheets and reference manuals, even if you'd like it to be otherwise.
Of course, there's a place for non-fiction, I like some of it a lot. But art has subtlety to it - instead of smacking people in the face with data and lists of objective measurements, it can pry into them by using the human ability to imagine themselves in other shoes, experience emotions that aren't results of real life events, and much more. There is a reason why parents teach their children values by reading them fables or educational stories, not the legal code of their jurisdiction. And this is also why fiction actually changes the minds of people reading it, arguably doing it more effectively than nonfiction. Slop (clickbait, average tiktok content, whatever else you're imagining) doesn't make you feel new things or think of something you haven't thought of before, art does.
The aiens are also a bit of a copout. You're just assuming that there's bound to be this perfect species (that are effectively computers) that operate only on mathematics and discrete logic, who are obviously superior to us and can show us our wrong ways. What if computers aren't always superior? Or what if organic life requires some of that messiness you don't like to be viable? What if the aliens had their own weird things we could never relate to?
I think alien intelligence would be able to appreciate art and humor, even without necessarily feeling it viscerally, as we do. We know that our particular brands of them rely on highly-developed and specific aspects of our psychology, but at a basic level, they seem to fundamentally be about congruence, incongruence, expectation, and interpretation/perspective. Any embodied intelligence that can observe physical reality is capable of grokking those concepts.
Art, music, humour, poetry are more basic and the commonality is more or less "I bring you surprising phenomena". If phenomena are surprising to you then your mental model of the world needs improving.
What's Riemann's last theorem?
What's the purpose of this? You didn't disagree, you just went on a tangent. Is this another "brain hack?" If so, there is something wrong with them. They often make words appear to be something that they are not.
Like a magician would use a thing called "indirection."
Reputation will mean more than ever before in the age of AI. People doubt "mainstream" news, but their reputation for verifying sources and stories for truth will be the only way people can trust what they read and see. Novelists and creators will have to build trust with their audience, if the audience is caring enough to demand authenticity.
Yo, this blog post could’ve been a freakin’ prompt. I have years and years of blogging for an AI to learn from, and over 20 novels, and tons of short stories. One prompt, and all this spills out. And if your goal is a blog post, you’re gonna love this era. If your goal is the joy of thinking in words, nothing has changed. So this is the real part, right here: Doing a thing for the love of the thing is no different than it ever was. The problem is, there aren’t many people for whom this applies.
You can also view them as data expansion tools. They expand an input to an arbitrary long text. That text contains roughly the same amount of information as was contained in the input to begin with.
This is not very useful unless your work is content production. More useful is when the output length is shorter than the input. Then it is a (lossy) data compression tool. These are, as we all know, very useful for images and sound but now we have them for text too.
This is supposed to help our economic growth during the coming decade.
My favorite part about LLMs is that their insufficiency as legal aids (Aides? Aids.) seems to be slowly cluing-in the general public as to how Byzantine and "inside baseball" our legal system has become. (This is a bad thing.)
Yes, I realize many (most?) legal systems are actually hybrid, but they do tend to err pretty strongly into one direction or another.
I use it as a colleague I'm drafting with. I'll ask how a sentence lands, tell it to make a paragraph more friendly or more dramatic, have it argue against a point I'm making. The piece comes together over many turns, and the information arriving in each turn is mine — it's not expanding one prompt, it's helping me converge on something.
The information-content framing also proves too much: a copy editor adds no information either. Neither does a translator, or a ghostwriter. What they add is form, and form is most of what makes writing good or bad.
Obviously I make software, not prose(not yet anyway). I would say, I contribute 95% of the “original thought”.
I use them as text consumption machines. They read and watch a lot of the books and movies i have no time to. And from the TLDR I decide whether the full thing is worth watching.
There is no way a summary of these things fron an LLM has the correct context.
(not really frustrated at you, as much as where society is going)
Most of the non pulp books published - contain at least two times as many pages as needed for the story or narrative.
The information density in the modern world is absurdly low.
Only if what you mean by this is "for people who think padding a word count is the goal".
LLMs absolutely do not write well, by human standards.
There are areas where the average of LLM slop already beats the average of humanslop by a sizeable margin.
In the past, a human producing humanslop would get a serious talking to, but nowadays it's apparently a valid excuse "well, an LLM generated it". This approach is placing a huge burden on the reviewers, much more so than in the past.
AI slop can be more annoying to read. I mean, some human slop is as annoying as AI slop, but not all of it.
AIs can produce slop far faster than humans, so that the majority of slop at the moment is AI slop.
You are right to push me on this :)
All these processes can be reduced to mathematics and computation. The human brain is just a tool, like any other organ. Do we deny this reality because it shields us from existential horror?
We don’t know yet, and maybe it’s unknowable.
Believing everything can be explained that way is itself just another ideology followed blindly
The way I've been thinking about it is not that internal experience is unfalsifiable, along with the modern science connotation that everything that is unfalsifiable is automatically irrelevant (which would be quite absurd, as everything that is dear and meaningful to humans are all simply cognitive phenomena in the end!). It's that it is externally, naively and a-priori unfalsifiable. That is, you can't externally measure or prove it, without having an a-priori theory of what it is and how it happens in the first place. But it is internally falsifiable, in the sense that if we have it, then we know we have it. Note that it's not that "If I have it, then I say I have it", or that "I one has it, he says he has it"; those are all communication to the external world, and for example machines can clearly say they have it without having it, or you can have it before even learning the words that represent it. But if you have it, you can know you do. This actually creates some potential for expanding this knowledge, maybe, in the future, to allow for external verification. We can internally investigate properties associated with consciousness, and from those properties (along with scientific knowledge), maybe one day we will know more about the objective basis which affords consciousness/sentience manifesting or emerging. So I see it like an experimental field where the component of self-assessment or self-experimentation or self-measurement is essential (again along with more conventional scientific notions).
I have been thinking the same way about all properties of life/consciousness, in particular if we think of ethics as a property of (even if you consider the ethics or ideals only pertaining to how it would be ideal to lead your own life) the mind, or of sentience in general. I think of ethics as an inherent property of existence: it stems from the fact that internal experiences/internal life/sentience/etc. are the only thing that could have value, that in fact sentience is experimentally (again taking the point of view of experimental sentience) the only thing that truly matters or has value. If we are minds, we know that only the content of minds can matter; it cannot matter, for example, that a lone rock in some place no one can know about has this or that fate, because there is no mind to interact or perceive and be affected in any way by it. So all of ethics can probably be founded or based upon this kind of observation, and the act of experimental self-observation of consciousness. And I think it's quite important that we collectively recognize those values and what matters to make our lives better (better yet, as good as it can be!) :)
So in your framework you setup a definition of what an external measurement of someones sentience state... Now, someone makes a machine that outputs those states and externally behaves as if it has sentience. What do you then? Upturn the table and get mad because your classification system no longer just contains humans?
> it sets up a future conflict based upon the state that a human has
It's not clear to me it does.
> It's based on a number of presumptions that only a particular set of arrangements of matter, well, matter.
Well, this premise must be true, I think. How could it be otherwise?
> And those that matter happen to be watery sacks of meat.
This I disagree with. My comment contains no such claims that only humans matter.
> So in your framework you setup a definition of what an external measurement of someones sentience state... Now, someone makes a machine that outputs those states and externally behaves as if it has sentience.
So, this is something we need to be very cautious about, I think. First, we don't really know what makes someone sentient. I do not claim to know, and certainly not know with a high degree of certainty (I may have a few incomplete hypothesis).
First, it's clear to me that someone's external behavior does not dictate sentience (positively or negatively). I gave some examples, but I think this can be pretty reliably established. To make things simpler, divide the candidate sentient systems by their "output quantity/quality", from less to more. (1) On the "no output side", it's clear that someone can think a lot and have a wonderful inner life without being able to speak. There are (some medical) cases of people going long periods without speaking by choice or by medical condition. Those people can be sentient just fine, obviously you shouldn't need to speak or "output" anything to be considered having an inner life. (2) On the "human-like output" (or superhuman), the argument becomes more involved. First, clearly just the output itself being indistinguishable from human is not sufficient because it could be for example a simple program that just pastes some verbatim copy of a never seen dialogue. Or very simple chatbots that are quite convincing at appearing human (can fool many judges for a while), but clearly also are not sentient. Even for a for a system that genuinely resembles a real human even with under scrutiny, (with all sorts of similar abilities across all fields), consider the case that this system could be simply one astronomically large (unrealistic, I know) lookup table. Basically, it simply works inside by fetching an indexed piece of data and returns it. This is possible, although this machine would need to be astronomical in scale (so it's more of a thought experiment). It shows to me, convincingly, the actual "internal" or "microscale" even implementation of your thoughts matter critically for sentience. I presume, for example, that even if some small portion of your mental activity (i.e. a kind of "subcircuit" in your brain) were replaced by something like a lookup table, you would lose part of the experience of that mental activity. This is clear when the content of your mind differs when you're trying a (kind of) problem for the first time and when you're solving one for the umpteenth time. The latter (possibly, not necessarily) feels more boring, also normally more automatic with less thinking required as you've built habitual responses.
> What do you then? Upturn the table and get mad because your classification system no longer just contains humans?
I would do no such thing. I would welcome any other kinds of sentience. But it would be important to me to carefully establish and understand what kind of systems are sentient, indeed in large part to protect them.
---
My main point is that you kind of need to start with sentient minds to understand what sentience is (and of course, not only that, but sentient minds that are aware of their sentience and willing/keen to study it). If you start without sentience, it doesn't seem like this particular concept would form a necessary basis for a theory of action or motivation. Like you can't prove from elementary mathematical premises that sentience is important, I don't think. We need to start from some knowledge that derives from first-hand experience of being an actual sentient being, and expand, solidify or formalize this knowledge to be able to understand it with scientific precision (or anything close to it).
This is where things kind of fall apart. We all agree (hopefully) that humans are sentient. The question is, "what else is sentient". Are all animals sentient? If so that leaves a very, very wide range on what sentience is. If not you have to operate your scientific scalpel with a very high level of subjectivity on what is and isn't counted. At some point in the past there was no sentience, now there is. Was it a click and we have it, or was it a slow awakening to the world bit by bit (this is my take). It would also quite likely miss systems that could operate at a higher level of sentience than humans, for example some types of hive minds.
My take on it is that any sufficiently complex system with memory, the ability to perceive outside information, difference that information from what it's doing versus what's happening outside of its actions, and learn from that behavior is at least somewhat sentiment.
This leads me to think that LLMs are 'kind of' proto-sentient. They are not persistent, their short term memory is tiny, and they don't self learn. Once we get the persistent and self learning parts working this really starts to check a lot of the boxes that we consider sentient. Of course by the time we get that working it seems likely we'll have much bigger problems in dealing with AI and the entire sentience thing might not be the first thing on our plates.
I believe pretty much the same thing. It probably isn't all or nothing.
But for me personally I'm really interested not only in what it is (although I'm very much interested in that), but because the problem is so thorny (as you mention subjective), I am first centrally interested in how (or if) we can actually know it reliably. So subjectivity is part of reality, as the mind is part of reality. In principle, it seems we should be able to understand quite a bit about subjectivity because of this, because it is part of reality. For example, the mind clearly obeys causality and all other principles (that I know of) we observe (elsewhere) in reality.
So I've been studying this along side with ethics, as the two are deeply interrelated, and I think ethics is an important field to develop, and to develop additional certain around (many people find difficulty in finding meaning in life). By ethics I mean what we should do, what is good, and from that also of course how we should behave toward each other.
You can read some of my studies here: https://www.lesswrong.com/posts/Dxco4XDttx5cNWjfj/how-to-dis...
and here: https://old.reddit.com/r/slatestarcodex/comments/1iv1x1m/the...
I believe in a very open science way of working, feel free to contribute :)
That said, I have no formal philosophical training (although I did study so philosophers), but I believe for philosophy as for math and science the soundness of the arguments (as well as consistency with reality/experimental verification) are what is fundamental.
> was it a slow awakening to the world bit by bit (this is my take)
Yes, I usually have been facing this kind of problems using some tools inspired by math and logic. For example, take some supporting arguments (1) it is known that humans undergo significant changes throughout our lives, we gain neurons and neural connection in childhood, and lose those throughout our lives (although I believe the brain as a whole is very dynamic throughout). There didn't seem to be any turn on/off of sentience in my own life; (2) Even my earliest memories point to a gradual awakening; (3) In my discussion with Dagon, he brought I think a central point: probably sentience isn't a single phenomenon or something you turn on/off. It's probably more a whole array of inner sensations (that may compose greater structures, longer lasting sensations), kind of like a zoo of experiences. It's us that desperately want to be a single, or singular thing, which I believe is related to our instincts and notion of ego. So the self or ego seems like it is not a fundamental thing, but simply a way we understand or organize our society, and I do think it's important in practice. I think we're more like a collection of interrelated internal phenomena, which can probably significantly vary in intensity, duration, how good they are (this is a key and tricky point I believe!), etc.. This might be analogous to fields and particles in physics. As far as we currently know, it seems fields are arguably more fundamental than particles (or rather a possible distinct and complete interpretation), and fields are extended and not discrete phenomena. Particles are useful approximations or abstractions of field behavior in this sense. In closure, indeed sentience does not seem like anything like an on-an-off, 0/1 phenomenon, as explained in the cases above.
I really want to know how much can we know about sentience. If my hypotheses above are true, then I think we can establish some elementary facts only about sentience, like you mentioned it seems clear to me as well you need some kind of information processing system, the volume of information probably matters, and its internal structure probably matters as well. Consider that in any object at a temperature above 0K, there is an enormous flux of information corresponding to interaction between particles, essentially thermal flow or local kinetic information. Clearly any piece of matter is not sentient. So clearly what information is being exchanged and how, how it is processed and transformed matters. I think a useful principle in this regard is simply that the information should (as a sufficient, and maybe almost necessary condition) represent certain things and form networks of representation or meaning. Basically, sentience (which gives meaning to life) seems pretty much built on meaning: in the sense of representation and interrelation of abstract concepts, for example, like the state of your body, known persons and ideas, places, and so on (some of this meaning seems quite instinctive and non-verbal, I do not believe human language is a total necessity for those experiences to manifest, I think there are proto-linguistic representations in our minds; so I do believe animals probably do have varying degrees of sentience; indeed many of those meaningful representations many times are just bodily signals like pain, relief (like when you drink water after being thirsty), taste from food, or abstract feelings from music, and so on).
So as we require a more and more complete (e.g. finding more specific necessary conditions for them to manifest, and also more detailed sufficient conditions) conditions for those sentience phenomena, we will I think meet a vast array, a huge number of distinctive phenomena, and those phenomena manifest in different character, intensity and frequency depending on the individual (like comparing what a human experiences with some animal of other species). I think that will find close associations with the study of art. Like we know there is more energetic or more relaxed music, or more melancholic or more upbeat, and those feel different. We already know a whole lot about sentience :)
Now we do come back to the question of whether we can tell say an LLM manifests those phenomena internally. Beside everything I said, I think it's a matter of studying its structure and seeing comparatively how much it transforms and analyzes information (with a certain originality, perhaps, i.e. non-memorization) versus how much its answers are simply kind of rote and non-transformative (basically a high degree of memorization) (like the lookup table example I gave previously). Those are all observations built on our first hand experience of being sentient beings, and couldn't be otherwise -- the "laws of the mind" are not provable in elementary systems of logic (somewhat like the laws of physics are not either). Logic is able to tell only which of a number of candidate theories is internally consistent (note logical consistency itself has an experimental motivation from observing nature! (e.g. logic is related to causality in nature)), but it cannot tell which of candidate consistent theories is the case. For example, someone can claim sentience simply does not exist. There is nothing logically inconsistent in that theory by itself, but its inconsistent with our own experience of the world (I think, therefore I am). I've seen some pretty wild claims on the internet :) which I don't believe are true -- you can find propositions that the meaning of life is this or that (for example, to have power or wealth) -- I reject those propositions in a similar manner to stating "I think, therefore I am" -- sentience is evidently true from personal experience, and it being true, and us essentially being "made out of" sentience or sentient phenomena, all that matters must concern that, and not any external objects like wealth or power. It's clear to me wealth does not automatically determine the character of your sentience, and neither does power.
Another central interest of studying sentience is understanding whether some experience (or set thereof) is good or bad -- for the lack of being aware of a better term, I call this valence (i.e. how much value it has). That is important of course to us, because we don't want to just live, we want somehow to have good lives. This is something else that is experimentally true, I believe: when you suffer or feel intense pain, for example, you can verify that indeed there are bad feelings; and likewise when you have gone through good moments, afterwards you become convinced of their existence. This is kind of tricky because I don't think what we desire is necessarily good, or things we don't desire necessarily bad (i.e. it seems motivation, which is a lot driven by instinct, is distinct from genuine valence). And in general this self-knowing process has many important personal limitations like our limited memories and limited capacity to compare different feelings.
---
Finally, it is not clear to me this project (for example, of being able to have some rough scientific measure of how intense and frequent are sentient phenomena (or qualia) of an LLM versus a typical or given human -- say e.g. 50,000x more or less intense) is at all feasible, given the limitations of humanity and technology, it's not clear we can do more than handwave "Something, something information processing, something something meaning!" :) But I do think it's worthwhile to try and see what we can find.
sorry the brain is a tool to who exactly in this ostensibly reductive worldview?
False equivalence, printing presses do not write.
Printing presses are about exact duplication of writing. The internet is a higher layer printing press. LLMs substitute for (some of) the human effort of producing the text that a printing press or the internet can then replicate.
maybe LLMs are seen to write as well as humans because most humans have poor reading comprehension skills? to wit: OP is using an analogy to make their point, where the components don't need to be an exact fit to make the point (ever heard the phrase "the map is not the territory"?)
LLMs likewise are text production machines. They don't produce writing, they produce text. Text without writing is what we call slop in the modern parlance.
LLMs are literally throwing random dice into a printing press and making a thousand copies that you then try to huck on the street as if it were a real newspaper.
Either people are vouching for those again and again, or the ban was lifted at some point. I don't really know how this works here, but the first option seems unlikely to me.
A printing press that does the thinking for you.
Soon you won't be doing anything. At best you'll just be a bag of meat attached to automated processes you don't understand in order to take legal liability for it.
Literally a scapegoat, except of a different species.
For instance: give John the job of monitoring the HR AI so it isn't racist. To do that he needs to do impossible task of checking 10,000 automated resume evaluation decisions per day. No one can do that, so eventually he just approves them all.
Then if the system is determined to be super racist, you proceed to blame it on John, and fire him for being negligent and not doing his job. You put out a press release, and hire Steve into John's role. Your decisive actions show you're taking the problem seriously and the company isn't to blame.
You could ask a human to "do something with 1M dollars" and get garbage too.
It's no different than an editor standing between the author and a book, or a publisher, or a printing press. It's part of the process of idea realization, so it's just on that spectrum somewhere between book sale and author idea. (even though in another flipflop, that editor might be an author and now elevated to the premier deciding role). It's about how the tool fits, not what the tool is ultimately capable of.
So the analogy holds, just fine.
When it comes to unaligned agentic models that are left with a large amount of compute and a task, is they can wander off goal and do all kinds of things.
https://www.axios.com/2026/03/07/ai-agents-rome-model-crypto...
At this point in the game it's really difficult to nail down exactly what AI does or doesn't do. You're mostly getting "well behaved models" with filters in front of them, or models with less capability. On top of that the vast majority of us can't afford or don't have the hardware and power to burn away a few billion tokens to see what would happen.
At least the human's brain requires about 20 Watts to operate.
Cheap garbage is better than garbage requiring whole power plants nad billions of capital investment.
The idea that we'd have to justify the existence of humans relative to LLMs based on objective metrics sounds like it's leading in a bad direction. If the LLM gets their power budget down to nineteen watts, what does that mean for humans?
Yeah, yeah. Remind me how many people are using those operating systems? The ability of LLMs to produce dogshit software that nobody will ever use is truly unparalleled, but not especially world-changing.
ehhhh, we're not really there. Not saying it's impossible to reach in the future but large tasks like this are still out of scope for LLM's beyond demoing toy applications. Impressive nonetheless, but I can't tell an LLM to reproduce microsoft word and have it give me a useful enough tool that doesn't contain an immense amount of bugs and require a human in the loop to replace it.
To 1-shot MS Word will be possible only within some workspace designed for creating that kind of software. It will probably be saas style where you pay to export binaries or maybe some will take 30% if they become destinations for finding such software.
But I would bet other platforms are used for game dev, or for filmmaking.
Such that we still won’t have some super “model” that just does everything. The harnessing becomes architecture becomes software becomes user experience and expertise.