Are we offloading too much of our thinking to AI? | Hacker News Reader

Are we offloading too much of our thinking to AI?

artfish.ai

526 pointsyenniejun111480 comments

I don't know if this is a good framing. "Too much" is subjective, and every heavy AI user will assert that they're just unlocking their potential, that calculators didn't make us dumber, etc.

But to latch onto the calculator argument: if you outsource adding numbers to a calculator, you're still you. On the flip side, if you use an LLM do most of your thinking, what's left? We have people here who use LLMs to raise their children, to manage relationships, to design products. So what's your unique contribution to this world - is it the prompt you once wrote? You're standing in front of a token-generating machine, pulling a lever, sometimes receiving gifts. Is that your edge, your unique experience, your purpose in life?

Many LLM maximalists say they use the tech to learn new things, but to what effect? Are you going to apply that knowledge of physics or computer science yourself, or will you just prompt the LLM again?

In my mind, it's pretty simple: I'm a human, LLMs are not. If a human writes a novel, it's inherently worth more because it's hard-earned and anchored to experiences we share. I want to support that. And I want to be a human who can write novels, the old-fashioned way. I'm not good at lifting weights or running, so my thinking is the only thing I have.

Have you read the "Whispering earring" essay? I love it for the LLM era.[1]

You can treat AI as a whispering earring - "What should we do now? How do we fix this? What do you think?" Or you can treat it like an exoskelton - "Implement kd-tree with metric space xyz for this problem, mapping this to that blah blah".

That's pre-thought execution automation that makes review much simpler - you already know the shape of the desired output. The whispering earring is atrophy.

1. https://croissanthology.com/earring

Thank you for the surprising read

Something I never considered much is what happens when everyone else is using the Whispering Earring.

You may be more free and independent, but you may also be unable to compete as everyone else easily gains wealth and success. Natural selection doesn't particularly care about freedom of consciousness.

Bleak.

Eventually, that becomes interesting. Before we get to that stage, I think we pass through (almost) everyone following almost the exact same advice as each other.

Such people are extremely predictable.

You may have already noticed, before LLMs, cliché? Talking points that make a group identifiable? Words and phrases that act as applause lights or cognitive stop signs? (That last sentence itself being a pair of clichés that you can use to identify where I hang out online).

Anyway, point is, LLMs will give us a memetic monoculture before they turn us all into a world of correctly personalised Whispering Earring wearers. That makes them predictable, that makes them exploitable. It'll be like playing chess against someone you know is using a specific version of Stockfish: even though it would beat you if you tried to fight the system unaided, you can win by asking your own copy of the AI to go one step further ahead, and it will be accurate precisely because it's playing against itself and reacting to its own moves.

(Of course, the fact I've said this in writing means this is in the training data; in the general case this means the LLMs will know that and account for that, but I suspect comments like this won't shift the needle all that much compared to the aggregate output of 3 billion people reacting to short-form emotional manipulation A/B slop)

> It'll be like playing chess against someone you know is using a specific version of Stockfish: even though it would beat you if you tried to fight the system unaided, you can win by asking your own copy of the AI to go one step further ahead, and it will be accurate precisely because it's playing against itself and reacting to its own moves.

I don't believe this is how chess works, and I don't believe this is how Stockfish works, and I don't believe this is how AI works.

Stockfish isn't winning because it's playing a better sequence of programmed steps, and having access to "the next version of Stockfish" doesn't mean it can "guess the next move" and play against that.

I didn't say "the next version of Stockfish".

You have Stockfish version n, see board state s. I have Stockfish version n, see board state s. I want to know what you're about to do, so I put Stockfish into state s, ask it what the best move is, and I know you'll make that move because I know you'll ask Stockfish version n the same question of the same state. I now know board state s+1.

The steps are not pre-programmed, but the program itself is (modulo hardware imprecision) deterministic. If there's a RNG in there then sure, this doesn't work as easily as I wrote it; and there may be randomness in the thing that this is a metaphor for, regardless of if there's one in Stockfish or not, but that's not hard to work with when you want to win against an aggregate: we invented the field of statistics to deal with random numbers because they come up so often.

There is deliberate randomness in stockfish. The easiest way to see this is from the fact that, when playing the white pieces, it won't play the same opening move every time. Often it's e5, but it also goes for e4 or Nf3 or something else entirely.

This is by design, and very much necessary for a competitive chess engine. Otherwise, people could do basically what you say: Run an offline (as in, ahead of time, with ample compute resources) search against stockfish that finds a line where it loses, then make an engine that plays that every time.

As a consequence, even if you know that your opponent is running stockfish, you can't really use that against them. Your best bet is also just running stockfish.

Your whole strategy basically involves having more compute than the other guy so you can look one step further. And yeah in an AI world having more compute or some exclusive data seem like obvious ways to get a leg up.

Yes.

Have you met most people?

Most people won't bother, find this kind of thought process to be somewhere between nerdy and paranoid.

Stockfish already does that. You run it as long as you want. When it's calculating evaluations for move k+1, it already knows what move k was, because it has that evaluation. It can explore that tree first, before exploring other trees to try to improve on move k.

Stockfish never settles down to an exact move, unless it sees a forced mate sequence. You could leave it running indefinitely, but the exponential blowup will soon grind you to a halt.

What new versions of Stockfish buy you are optimizations and better heuristics. An older version of Stockfish can still beat a new one, if given enough of a compute advantage.

Jacek Dukaj in his book "Perfect Imperfection" (Polish hard sci-fi about alternative physics, mind uploading and enclosed universes) wrote about entities vastly superior to human-level intelligence having an ability to simulate humans to such a degree of precision that they were able to determine what a given human would say or do based on his own internal experience and knowledge in real time. The superintelligent beings didn't scan his brain patterns, they just had multiple simulations of him and chose the one that would best match his actions based only on external cues like position of his hands, body language, speech patterns, etc.

A solution to avoid being read to such a degree that your free will effectively is destroyed was to introduce randomness as a program in your own mind processed in an equivalent of a "cloud VM" - random changes in your body posture, hand movements, weird phrases introduced while you spoke could confuse the internal models of you enough to avoid hyper-predictability.

That's really good, I'll have to read some of his work (Jacek Dukaj). It makes me think about the role of indeterminacy and unpredictability in physics. And reminds me of..who was it, Gurdjieff maybe, who taught that humans are like sleeping machines, and that constant conscious effort is necessary to "stay awake". Many of his exercises for students had the purpose of recognizing one's own habits, routine patterns of thinking and behaving, and breaking out of them.

The sci-fi story also reminds me of, I think it was related to Karl Popper's book about "objective knowledge". There was a thought experiment about Mozart, if a scientist could quantify everything external about the composer, like the brain, the body, the room he's in, his life experiences - is it possible to deterministically compose new works exactly as the real Mozart would. And if not, what is the difference between the perfectly simulated Mozart and the real one?

This "randomness" is related to the ancient Greek concept of clinamen.

> Clinamen, derived from clinare, to incline: the unpredictable swerve of atoms in the atomistic doctrine of Epicurus. This swerving, according to Lucretius, provides the "free will which living things throughout the world have".

This is already underway.

My dog often gets misidentified as a restricted breed. This used to make apartment hunting difficult because, occasionally, the property manager would visually ID the dog breed as banned, I’d have to go to the vet and get paperwork, potentially gene testing, arguing she wasn't, it was a whole thing.

But, recently, the apartment I moved into had an online portal where I had to upload a photo and it would identify the breed to determine if it was approved.

I correctly assumed the portal was using an LLM for this purpose. I wrote a script which submitted different photos of my dog to the major LLM providers until it found a photo which all the LLMs would identify as the correct breed.

I simply submitted that photo and, as expected, passed with flying colors.

I was anticipating that it would have reversed course and flagged you. Blessings on you, and your four footed friend.

Or you could just submit a photo of someone else's poodle.

It is literally impossible for everyone to obtain wealth. Wealth isn't a number or physical circumstances. Wealth is having economic power over the people around you. Success is a little more nebulous, but when thought of as a relative of wealth it's similarly contextual.

The danger isn't everyone but you getting wealthy. The danger is that wealth tends towards concentration. And it tends to concentrate around people who are already wealthy. The danger is, bluntly, that things will get worse for all but a few and most people will be so caught up in a red queen's race that they can't see how to stop.

No. The world has absolutely gotten wealthier over the last few centuries.

Wealth is obviously not zero sum. Humanity is far wealthier today than before the industrial revolution, and the trend is still towards increasing wealth.

You are talking about access to resources and technologies, which is basically constantly rising, we couldn't stop that if we wanted to, so that's a given and more importantly orthogonal to the wealth that buys you goods and services.

The latter is absolutely zero sum, because of everybody has a billion, a billion isn't that much any more. E.g. human trafficking exists because some people are rich and others are poor, and the fact that even those poor people might have access to things an emperor 1000 years ago would have dreamed of doesn't change that, because the people who treat them like slaves have even more. It's all about the relative difference between people who are alive today, the absolute increase compared to people who lived N years ago is a red herring.

>You are talking about access to resources and technologies, which is basically constantly rising, we couldn't stop that if we wanted to

We could very obviously stop our progress in resources and technology. It has happened throughout history many times and we see plenty of modern examples.

You seem to be treating the accumulated products of civilization as though they are geological facts rather than fragile outputs of maintained institutions.

Maybe you should ask yourself why you believe that 1) access to resources and technology are "basically constantly rising" and 2) why the relative difference in wealth today is more important than the (magical?) existence of a system were everyone has continually gotten more access to resources and technology.

> Maybe you should ask yourself why you believe that 1) access to resources and technology are "basically constantly rising"

Because people have and share ideas, and I was using that as a shorthand. Even if they aren't, if you completely strike that part -- my point remains completely untouched.

> why the relative difference in wealth today is more important than the (magical?) existence of a system were everyone has continually gotten more access to resources and technology

Because in the context of "Wealth is having economic power over the people around you.", that is the definition of wealth that is used. To quote a randomly picked one, emphasis mine:

"Large possessions; a comparative abundance of things"

> You are aware that the default state of humanity hasn't been one of continually increasing access to resources and technology?

There was no default "state", at least by the time you're talking about humans, we probably already learned from each other by observation. Sure, politics and wars and religion and feudal dynasties and whatever make bumps in the line, but generally it goes up. Once things get out, they're out, roughly speaking (that's the keyword here), and even in the animal kingdom.

And as I said, even if that's completely false, doesn't matter for the wider point about wealth, and that it can only buy you power over others if they have less of it.

>There was no default "state", at least by the time you're talking about humans, we probably already learned from each other by observation. Sure, politics and wars and religion and feudal dynasties and whatever make bumps in the line, but generally it goes up.

No, that's not the shape of the line. You can view it here:

https://ourworldindata.org/grapher/global-average-gdp-per-ca...

The hockey stick upturn in recent centuries needs explanation.

I'm pushing back so hard on this comparative definition of wealth (which I notice is the secondary definition you found behind large possessions) because I think it's silly to do that when we have an unbelievably dramatic increase in wealth in recent centuries which needs understanding first.

Hand-waving away human progress and forcing zero sum definitions of wealth feels unserious. You have terms for things like inequality, why not use those instead of distorting what wealth means?

That is an entirely unserious graph.

In a sense I agree, since GDP is a bit of a silly metric.

But it's a decent enough proxy for the parts of human progress that I think are meaningful that it illustrates the point well.

That website is an excellent resource and you can find other more compelling charts there.

> The hockey stick upturn in recent centuries needs explanation.

Why? It goes up. As I said.

> I think it's silly to do that when we have an unbelievably dramatic increase in wealth in recent centuries which needs understanding first.

That's not pushing back, that's wanting to talk about something else first. That's just not something I'm interested in.

> distorting what wealth means?

It simply means both things, from context it was obvious how it was used in this instance, you ignoring that and going on tells me explaining it yet another time won't help.

> ... human trafficking exists because some people are rich and others are poor ...

No, it doesn't. This is an absurd claim, and nobody intelligent should take it seriously. Human trafficking exists because some people choose to do evil things. There are plenty of relatively-rich people who are actively trying to stop the evil of human trafficking.

Furthermore, just being wealthy has not, historically, made you immune to being made a slave. For just one example, look at the story of Joseph in the Bible. He was his father's favorite son, the heir presumptive to fantastic wealth for his time (that "coat of many colors" his father gave him cost the equivalent of what a sports car would today), and yet his brothers who hated him sold him into slavery while his father wasn't around to protect him. Throughout history, cross-border slave raiding has been a thing, too: it didn't matter how much gold you had tucked under your bed if the Vikings (or whoever) caught you alone, or outnumbered and overpowered your hired guards: once they hauled you away and slapped chains on you, you no longer had access to your wealth, and your life depended on making your captors happy with your work.

To think that slavery, ancient or modern, is caused by relative wealth differentials reveals an abysmal ignorance of both history and human nature.

> To think that slavery, ancient or modern, is caused by relative wealth differentials reveals an abysmal ignorance of both history and human nature.

I claimed no such thing, you read way too much into your missing the point.

> if the Vikings (or whoever) caught you alone, or outnumbered and overpowered your hired guards

Wait, I thought it was just because "some people do evil things", nothing to do with relative power?

I'm deliberately giving an illustration that shows that relative power throughout most of history had to do with physical strength and weapons, rather than money.

And if you didn't mean that slavery is caused by relative wealth differentials, then saying "... the people who treat them like slaves have even more. It's all about the relative difference between people who are alive today ..." was a very poor way of making your point. So what was your point that I missed, then?

That in this context, it's about the wealth that leads to "having economic power over the people around you". That is zero sum, and the general rise in absolute living standards is as relevant to that as a comment about apples is in a discussion about Apple.

Ah. When that one commenter said "Wealth is having economic power over the people around you", I ignored him, because that's a ridiculous definition, not to be taken seriously. Any argument that arises out of that definition is inherently flawed, since one of its premises is false; it would be like saying "Since the moon is made of green cheese, therefore X". The definition of wealth that nearly everyone agrees with is, well, I'll just cite the opening line of https://en.wikipedia.org/wiki/Wealth because it's a pretty good summary: "Wealth is the abundance of valuable financial assets or physical possessions which can be converted into a form that can be used for transactions." Or, to put it in simpler terms, wealth is the stuff you have that you can use to satisfy your desires. Anyone trying to make it about economic power over people is trying to redefine the standard definition for his own purposes... and that's usually a big red flag warning you of a bad argument.

it is simply one meaning of the word, and you talking about something else instead a waste of time.

As Led Zeppelin once said, "you know sometimes words have two meanings". That definition of wealth is all well and good for books about how markets are magical fix-everything pixie dust. But it's not the definition of wealth that applies when used to describe a person. A wealthy person is not someone with a lot of things. That's a hoarder. A wealthy person is someone who can use their economic power to shape the world around them.

That definition of wealth is all well and good for books about how markets are magical fix-everything pixie dust

No, it actually means something. We have access to refrigeration, climate control, near-limitless computation, entertainment, transportation, knowledge, communication, and medicine. All of this stuff would BLOW THE MINDS of medieval kings. Even if you're living in a 1BR apartment, working at Walmart and struggling to pay rent, you enjoy many luxuries Charlemagne could scarcely dream of.

Go to the store, grab a pineapple off the shelf, take it home, and eat it. Then read about the great lengths [1] the wealthy people of the past went to in order to try to grow pineapples in Europe, and how obsessed the culture became with this fruit.

[1] https://en.wikipedia.org/wiki/Pineapple_mania

If you're in that one bedroom, struggling at Walmart, you probably don't have a dishwasher or a washing machine and dryer, and possibly not air conditioning or heating, or at least, your electricity bill is enough that you have to think twice before setting climate control to a comfortable level. Your social life probably isn't anywhere near what Charlemagne had. You have a smartphone and that has brought you news from afar. But that news is terrible! Local politics, things are fucked, climate change is out of control, geopolitics is a whole other shit show. Charlemagne didn't have to deal with Israel and Donald Trump. His world was very different. Sure, I have a fridge and a microwave and can use that to sate my hunger in mere minutes, a luxury inconceivable to Charlemagne, but all the trappings of modern society, with TikTok and Reels hasn't made us globally happier. Seeing the emotional damage from whichever online platform surfaces drone videos out of Ukraine today for me as a form of wealth is one perspective. We've lost the plot, or at least I have. We're solving the problems in front of our faces without asking if that's the right problem to be solved.

>A wealthy person is not someone with a lot of things.

No one made that claim.

By nearly any sane metric the world is wealthier. You can verify this on sites like Our World in Data.

We are healthier, have more leisure time, are less likely to die in natural disasters, etc.

It's willful ignorance to define wealth in a way that wouldn't show humanity as having gotten more wealthy. You're doing harm by overlooking massive gains we've made and understanding why we made them and instead redefining the common sense understanding of basic terms.

>> By nearly any sane metric the world is wealthier.

I would also point out people living in "poverty" in the US would be considered quite wealthy in other countries like Cuba or Haiti.

Everything is relative to how you define wealth and what you're comparing it to.

My father's generation had much more leisure time than my generation. When he was out of the office at 5pm work stopped. There were no laptops, slack pings, etc, etc. Vacations were longer, pensions were a thing. You could get a job supporting a family right out of high school. Things have absolutely gotten worse in the west no matter how much you want to twist the stats.

asimov's "profession" is a very relevant SF short: https://www.inf.ufpr.br/renato/profession.html

Wow what a read! Thanks for sharing. Really helped me unpack why I’m so bothered by people who c/p AI answers verbatim.

I did a code review for a really, let's just say "challenging", co-worker a few years ago. There was one block of code that was really obtuse and I couldn't figure out why it was there and what it did. So I asked him.

He sighed like he was taking responsibility for a grave sin and I should admire him for it, and said, "I don't know. I copied that from StackOverflow."

I've felt that AI is just an amplification of what we've all done and been through with SO answers.

It certainly makes it all the easier for people predisposed to this quality of work.

I have learned the hard way that "corpo code" is ephemeral. I spent at least 2 man-years on Ruby/Rails code for Desk.com, once spending an entire month to solve a really bad and intermittent session-logout issue that stumped 8 developers before me (I could tell because they left a bunch of commented-out debug code and I had the git-blame lines), and after I left, that whole stack got retired and its clients moved to some garbage Salesforce offering as part of their crufty Java monolith. (Sorry, SFDC.)

So if we move from "bespoke handcrafted code" to "roughly-engineered, unstylish but still quite working LLM-generated code" in that space, there is LITERALLY no love lost.

Spend your handcrafted love-energy on open-source or library code that will stick around. Leave the corpo code to the LLMs, I say, especially if you don't own it.

Way too many people like to reach for the easiest path. They will often assume or have blind faith that the AI or online answer is correct. Coming up with the solution themselves or taking time to understand why something might be the correct answer, can be just too much work to be bothered with.

It cracks me up that you bring up the exoskeleton metaphor, because I'm pretty sure it originated from a 100% AI-generated essay that made it to the top of HN a while back. So I guess, AI is whispering things into our ears whether we notice or not.

That's a pretty mid-tier metaphor on my part, along with Iron Man + Jarvis, that I don't think I'd even want to claim credit for :)

I will claim the CNC analogy though - I sometimes feel like a modern machinist just walking between machines and listening for screaming metal.

Exoskeleton is such an obvious metaphor for AI (at least, one potential ways of using it) that surely no one, human or AI, could possibly claim credit for it.

I had been explaining it to others as 'My Iron Man suit' before reading the exoskeleton essay, months later. So, you are definitely correct.

It feels more like a BMX bike to me. The quality of what I get out seems to be a direct function of what I put in. The threshold of what I can achieve is raised slightly, the speed with which I can achieve it is raised quite a lot - but it's not a magic rocket ship that whisks me to far off hostile environments and protects me while I'm there. I'm still "naked". The code is still mine, and I can get off the bike at any point and walk.

Thats... not how most of the world is using llms. Students are getting measurably dumber (as in lacking actual topic understanding, since actually learning sonething properly is much harder than shallow knowledge), every other day I read about somebody losing their wealth due to listening to llms with finances.

Other recent articles - psychologists start seeing people losing their uniqueness and many people slowly veering towards same middle bland opinions.

I think top parent is correct - we are in the process of visibly losing a lot from our humanity, much more than we are gaining. We shouldnt be listening too much to loudest mouths here that praise efficiency gains, many of those are 'semi-autistic' folks with tunnel vision focused on 1 aspect and ignoring larger picture.

I dont see any change of course possible, the sweet nectar of efficiency and offloading decisions and work is too strong a drug to ignore. But everything has its set of benefits and drawbacks, and drawbacks here seem pretty brutal so far.

I think the first usage of this metaphor in its modern meaning belongs to Norbert Wiener

> I'm pretty sure it originated from a 100% AI-generated essay

There's no such thing. You're probably thinking of "AI"-regurgitated essay.

the exoskeleton metaphor is way older than the current wave of LLMs

>So I guess, AI is whispering things into our ears whether we notice or not.

Humanity is a weak hive mind species. You can, with effort, keep it from whispering it into your ears, if you really want that. But it's impossible to keep it from whispering in another ear attached to your hivemind. And the only difference in those ears is the propagation delay before it's rattling around in your own skull.

If you think the AI generated that metaphor, you haven't been paying attention.

> I'm pretty sure it originated from a 100% AI-generated essay

There's no such thing. You're probably thinking of a 100% "AI"-/regurgitated/ essay.

This has a nice, Borgesian, charm to it but I feel like it's premise is confused. If it's just about the here near-empty signifier of "happiness," what does the magic tech of the earring actually contribute to the parable that some hypothetically perfect narcotic wouldn't?

The earring doesn't make you feel better, it actually produces a better result. It's never wrong. You might regret addiction, for example? Also, it's a parable. Take it too literally and it loses its charm.

> It always tells you what will make you happiest. If it would make you happiest to succeed at your work, it will tell you how best to complete it. If it would make you happiest to do a half-assed job at your work and then go home and spend the rest of the day in bed having vague sexual fantasies, the earring will tell you to do that. The earring is never wrong.

It just has to be this in the story because it doesn't want to engage with why what the earring indicates is so good except that it provides unfailable "happiness". It implies a static world where "the right decision" always exists, but doesn't want to engage with the details. It also implies a kind of akrasia[1] that I don't think resonates with lived experience: "happiness" is a state we pass in and out of, not the accumulation of correct decisions. We undergo unhappiness for the sake of future happiness. "Happiness" is a questionable end-all goal first of all, but either way not even the kind of thing ontologically compatible with the author's framing.

It just feels half-baked, it doesn't actually say anything even when it has all the pretenses of that. Maybe that's too harsh though.

1. https://en.wikipedia.org/wiki/Akrasia

You're getting too hung up on the word "happiness". The earring gives you good advice. Good according to whom? According to you. It helps you succeed at your goals and live a fulfilling life.

This doesn't make sense with the story. If it's good advice according to you, why do you need it at all? Doesn't it provide you something that you otherwise don't have?

Maybe I am just not smart enough for this.. What is the parable supposed to be about? What are we supposed to learn from it?

This is such a good point. It matters HOW you use the LLM.

Come to it with knowledge and understanding of a subject matter, asking for an implementation? That's different from going to it with no knowledge, asking for guidance on everything.

Reminds me of Daemon by Daniel Suarez. I could plausibly see us heading there for sure https://www.goodreads.com/en/book/show/6665847-daemon

Not particularly related, but “whispering earring” made me think of the hemalurgic earring in Brandon Sanderson's later Mistborn books.

Except it's not really a mechsuit; inasmuch as it is, it's more like an Evangelion: it's got bits of people's souls floating around in there; it only works well with a high synch rate with the user, which is difficult to achieve consistently; and sometimes it pops off and does its own thing anyway.

And the way you achieve and maintain deep knowledge of the shape of the output you want is... well, to write and test it yourself. LLMs are a cognitohazard. If you aren't accustomed to programming without them (or even if you are) it's hard to see the line between using them to remove drudgery and relying on them to think for you.

The Whispering Earring story reminds me of that episode of Rick and Morty in which Morty gets a piece of crystal embedded in his forehead which allows him to perceive the moment of his death; he then begins steering his actions towards the future in which an elderly Jessica (his crush) tells him she loves him as he dies, down to individual articulated phonemes and muscle actions. In the end, it's revealed this future is not what Morty had hoped for.

That Evangelion analogy is a good one. I don't necessarily agree with all of it (for example, I think the high synch rate is the weakest part of the analogy, and LLMs can be used as tools with some decent success without needing to plunge into them too deeply), but every piece of it can be coherently argued for (for example, perhaps I'm missing an even higher threshold of success that I could achieve with a higher "synch rate" equivalent, and I just don't know about it).

But kudos for coming up with such an evocative analogy.

"Synch rate" is roughly "you know how to manipulate the model to get it to do precisely what you want". This is nonobvious even for open-weights models and can change as new models are published, often under the same name as the old models.

> In my mind, it's pretty simple: I'm a human, LLMs are not. If a human writes a novel, it's inherently worth more.

While I appreciate you laying it out so plainly, I disagree. A novel is a bunch of words and I don't care if they were written by one person, five, an AI, or infinite monkeys on typewriters. What's valuable in a novel (or a poem) is in the words.

I think there is both value that comes from the work being authored by a human, and value which does not?

I care because it’s not personal, it’s not informed by lived experience, etc. It has no meaning. AI “novels” are just facsimiles of other people’s writings assembled in a way that appears new, but ultimately it’s just an approximation of a person writing a book. Their’s no author intent or context of any kind to consider. It removes core pillars of experiencing literature.

I don’t want a mathematical approximation of writing informed by feelings, knowledge, experiences, etc. anymore than I would want to see an “AI band” perform just because the music is supposedly great. There’s no personality, there’s nothing personal period.

while I understand this argument - I think it consistently falls down when compared to the types of content that are generally purchased and consumed.

I'd argue that most folks get plenty of value from the content itself, entirely separate from the intent, context, or even existence of the author.

I don't care about the motivations of the director to enjoy an action movie. I don't care about the life history of the author to enjoy a good fiction.

I think there is (and should be) space for content where people care, but I'd suggest it's the fringes (ex - majority of music is production grade pop, not meaningful songs, majority of books sales are erotic fiction, etc).

> types of content that are generally purchased and consumed.

This is the entire issue though. We’ve boiled creation down to “consumable content” like we are all in a boardroom talking market strategy. I am reading a book to enrich my life. Yes I enjoy popcorn entertainment and “low brow“ stuff, not everything needs to be Citizen Kane, but what does it say about us if we’re willing to just sit around “consuming” LLM content which is just facsimiles of actual creation by real people? What is the point when we have more “content to consume” than ever before? It’s just saturating us with impersonal stuff lazily achieved by scraping the real thing and prompting until it outputs something acceptable. Why is the person even making it? The answer unfortunately is almost always “a quick buck,” so I’m not sold.

What is the point of reading a middling fantasy book that a person didn’t even create when there are already likely countless fantasy books in existence/being written right now?

>What is the point of

Anything.

While I subscribe to an absurdist viewpoint rather than the nihilists, attempting to take an objective view on what one should consume is a difficult and most likely completely incorrect task.

The entire talk of should we mass product LLM crap generally side tracks the discussion of 'how can we create a world where everyone can consume mindless crap instead of endless toil'.

> I don't care about the motivations of the director to enjoy an action movie. I don't care about the life history of the author to enjoy a good fiction.

...and you don't need to know those things, that's insane. But you very likely are, even (not so) subconsciously, questioning some of these things.

> I don't care about the life history of the author to enjoy a good fiction.

But the fiction is a product of the author's lived experiences. If the author had lived a different life, they would have written a different book. Or none at all. Without life experience, where would stories even come from? Why would they matter at all?

You do care though. You may not think you care but you absolutely do. If somebody tells you that in the Hood canal west of Seattle, there is a population of endangered tree octopuses, who are unique among octopuses in that they climb up to land and lay their eggs under fallen tree trunks in old growth forests, and that they are now endangered because road construction is cutting their access to the old growth forests, so they cannot lay more eggs.

You would care about that story, until you found out that this story is a lie, an old April’s fools joke that escaped confinement. The words are the same, but your reactions to the exact words have changed with new information about the source.

When we read personal stories it affects our emotions as we empathize with the author, or otherwise share the feelings that the author is trying to convey. When we find out there is no such author, our empathy and our notion of shared feelings vanishes with the new information even though the words stay the same.

> You do care though. You may not think you care but you absolutely do.

This is something I have a lot of trouble explaining and generally don't try to because I've never actually studied this or anything. So I can only go from my 45 years experiencing of experiencing art along with others.

Of course if you are just putting on music to work to, this isn't going to matter much if at all, but...

Generally people do really seem to care about the person behind something they are experiencing. The simplest example I can give is one of those extremely well shot photos that very few people have taken from a sitting position of their feet dangling off a massive building. I would have a very hard time believing anyone claiming that such a photo wouldn't give them very different feelings if they knew it was a real person v not. Again, this is the simplest example I can think of but I think it goes much deeper with all sorts of art, ie, most people to some degree are attempting experiencing art through the person who created it whether they "know" it or not. This is evident when presented with something they don't like and say something like, "Who would make this?"

Your example hinges on whether a bunch of words is true, not on how they came to be written.

> When we read personal stories it affects our emotions as we empathize with the author, or otherwise share the feelings that the author is trying to convey. When we find out there is no such author, our empathy and our notion of shared feelings vanishes with the new information even though the words stay the same.

This isn't true for me. If I read an incredibly moving poem and later learned that it was written by someone casting the I Ching and picking words out of a hat, it would not affect how I felt about the poem.

> You would care about that story

No, I wouldn't. I don't live in Seattle, and I don't care about octopi. Why project what you think a reader cares about?

Generally speaking - I agree with you. Source bias is a real logical fallacy, and failing to evaluate the content itself, regardless of the source, is a problematic view of the world.

In the same way that I don't need the lumber in my house "hand sawed" for it to achieve my goal of creating a habitable space.

---

But more broadly, I do think there's space to at least question the use and role of AI.

Because while content can (and should) be addressed directly, there's a valid meta-conversation about the intent of producing content, and the results producing that content might have.

What goal does producing this content achieve?

What is the role of this content in society?

Is this content, on this scale, an appropriate thing to be making?

These are MUCH harder questions - often because we've shifted from concrete (content) to abstract (value judgements).

To go back to my housing analogy: We're no longer evaluating the benefits of hand-sawed vs power sawed timber. We're discussing whether our housing is built in the right spots, if we're building enough of it, and are we allocating it in the right ways.

> Source bias is a real logical fallacy

This is irrelevant because we’re not using the source to judge the strength of an argument. Logical fallacies have only to do with the strength of a logical argument, nothing else.

To illustrate this, if I sell you some energy and you ask whether the energy comes from burning children or from a solar farm, I can’t say that it’s a logical fallacy for you to care because all energy powers your home just the same. You don’t owe any consideration to the content (energy) at all, because energy is not an argument that is subject to the source fallacy, to treat it as such is a fundamental category error.

Even to take your tack and say “We should consider the energy generated from burning children apart from its source, because we don’t want to fall prey to the source fallacy. However, the societal effects should also be taken into consideration…” shouldn’t be countenanced.

There's a road in California that is the last hand-built road made in the state. Nothing special about it, in particular (aside from the terrifying, exposed drop), otherwise.

But since I know the human history of the road, it _feels_ different when I'm on it than other machine-built roads.

It's the same in a hand-made house. Knowing the human labor gives the house a different vibe.

If we look at 10 paintings and one was painted by a human master artist, it becomes, to me, more impressive than the AI works, even if it isn't the award-winner.

These sentiments are incredibly subjective, of course. Some people simply feel no difference between a hand-made brick and a machine-made brick other than the latter is likely cheaper and of higher material quality.

But for those of us looking for the indescribable _soul_ of the work, we fail to find it in those produced by machine.

I just visited Lowell National Park and watched the mechanical looms in action. The cloth they produced was blandly soulless, like all the cloth we use and wear and discard. The loom itself, on the other hand, was a hand-built mechanical work of art, and felt amazingly _human_ by compassion.

This isn't something we tend to value in the US, though. The closest we get is people hanging their kids' childhood art on the walls and buying custom art at great expense to increase social standing. And a few support-your-indy-artist types.

>it _feels_ different when I'm on it than other machine-built roads.

Well, because roads are hard to build and take a lot of energy, it's going to be way narrower, curvier, and hillier. The guy shoveling out rocks all day would have gladly switched the shovel for a bulldozer.

But all of these value statements about value statements are kind of ignoring the Moloch problem in the corner of the room. What becomes the purpose of humans in a world of automation? At our current rate, it's being exterminated under the boots of corporations that swallow up all the value in the universe. The conversation of "How can things be made better for all humankind" seems to have disappeared after the greed is good conversation won.

Wow, I could not disagree with this more. What’s valuable about a novel is the social relationships it fosters, both in relation to the author, and also all the others that have read it. I read a book to better understand what other people are thinking, how they see the world — both directly from the author, and indirectly, in discovering what other readers may have found valuable.

I can maybe understand finding value in a machine-written novel if others also read it and enjoyed it, but having an LLM spit out a novel and reading it in isolation, that would be a complete waste of time to me.

What if the original idea/concept was from a human, who used an LLM to extend and write the book?

For me in that case, I'd value the story but not the writing. I want to know that a human has had the experience of writing what I am having the experience reading.

If I find out after the fact that an AI wrote it, the writing becomes bland, like a magic trick exposed.

It seems to me that you don't like reading (which is fine). Some people enjoy reading words strung together in a certain way. The value comes from the simple relationship between the text and the reader, not on some kind of social connection.

I love reading, and I was explaining precisely why I love it.

Ironic, as you didn’t read their comment…

> The value comes from the simple relationship between the text and the reader

This is not some universal truth, yet you state it as such.

People can get different things from a text.

It seems to be that you don't appreciate the relationship between the author and the reader and the author and the text, which is fine, but is a bit tragic.

Not that I doubt that one day people will simply gather around the AI infinite story creation bot. They just won't know what they're missing. :(

I once read Ben Franklin's autobiography. I have yet to experience any great blossoming of our relationship.

More seriously, the novel is a unidirectional medium of communication. Not a relationship. That said, it is meant to convey a perspective. I don't care if it's a human, a robot, or a little green man's perspective. It just has to be an interesting, useful, or enlightening perspective that I didn't already share.

Right now LLM's don't really have unique perspectives, that may change. So I withhold judgement. As of today, I wouldn't read an LLM's novel. In 10 years, who knows?

Ah I always felt like novels were a partnership in a way. He is putting down words and I am using that to create a world in my mind. 2 people reading the same novel are creating 2 different worlds.

I've always been fond of calling it telepathy. That's really what it is. Further, it can even let us see into the minds of the long dead. Humanities superpower is that it is the only race which can preserve knowledge beyond a single lifespan.

With LLM's we can also now preserve actual thought patterns and skills beyond a single lifespan. I can't wait to see what the future brings.

> I once read Ben Franklin's autobiography. I have yet to experience any great blossoming of our relationship.

I've also read it, and got to know the man that much better. It especially added a great deal of texture to the experience of visiting his home site, his church, and his grave.

I'd argue that humans rarely have unique perspectives, being highly influenced by their environment and experience. And have we not seen AI coming up with novel math?

I don't think unique perspectives are necessarily the compelling part. I think the words are just a medium for thoughts. AI only produces the former, not the latter.

That's my relationship with Ben Franklin. It's not that I read what he wrote; it's that I know what he thought.

When you read a book, you hypothesize a writer and have some kind of inner relationship with the hypothetical writer. But it's not a real relationship. It's in the words, not outside them.

I think the transmission through the medium makes the relationship with the other side.

One could certainly have a pen pal relationship where the only communication was via text. The relationship begins with the very first message.

Or a relationship with an autobiographer who spills their soul onto the page.

But if I look through the text of an AI spilling its soul, there's nothing on the other side.

> I think the words are just a medium for thoughts. AI only produces the former, not the latter.

I think this is exactly what I was trying to say above when I talked about perspective Reading is a form of telepathy, and there's just nothing INSIDE of most AI writing today. When that changes, I'll take it more seriously. I do suspect that it will very soon though.

As you say, AI's are already coming up with novel math and I've seen some glimmers of purely mechanical brilliance in the writing already.

Reading is beautiful because of language's inability to fully capture emotion and reality. Every author tries to tell a story in the way they think captures their perspective best. Every author and book you read is a different perspective on the world that shapes your own.

Reading writing from an LLM loses all that. It's just words, there is no expression. It's literally the most boring parts of reading.

novel is a bunch of words

or even a bunch of characters, bunch of pixels and so on.

To me this is the wrong level of abstraction that is not sufficient to encode the meaning of literature.

LLMs are inherently backward looking; their training corpus consists of existing works and their ability to extrapolate is ultimately constrained by that corpus. However powerful their proponents claim them to be, they aren't powerful enough to predict the evolution of human society and culture as a whole.

Changes to that corpus that reflect the real world are going to come from incorporating future works created by humans. (LLM generated training data can reinforce fidelity to the current corpus but don't alter it.) The future still belongs to human creativity.

Completely untrue. Stephen King has a loyal following. These same people aren't going to pick up a generic scary book written by a machine. This goes for music as well.

> A novel is a bunch of words

> What's valuable in a novel (or a poem) is in the words.

Even if the words are a lie? Misleading? False?

I'm not even talking about LLMs. What if it's propaganda designed to influence your thinking, possibly against your own interests; are those still valuable words that you'd cherish reading?

My point is that the source matters, intent matters, and authenticity matters. To me, anyway.

Most (all?) good novels are propaganda designed to influence your thinking.

If a machine could truly provoke thoughts as well as a human author, then yeah it'd be worth reading its work, too.

Reading a book is a way to peek into author's mind. There is nothing to peek into when the book is generated by llm.

Good news! I have so many words for sale. Humans spend so much time trying to put them together in 'meaningful' ways. I take all the hassle out of it! Dirt cheap! Billions of words! How many would you like?

"What's your unique contribution to the world" is a good question, and I've been thinking about it a lot. One framing I very much like is thinking about the priceable versus unpriceable contributions of a person. As an example, your software engineering ability is priceable (it's your labor). Your value to your parents isn't priceable. The problem is, right now the default assumed value of the latter is 0. Our society only tries to put a value on priceable facets of a person. There's reasons for that, wisdom of the crowds and whatnot, but of course it also means that if an AI can do that thing, economics drives society to adopt AI over employing people.

It's possible to set that latter value to be nonzero. You can't use free markets to set it because they necessarily cannot see what price to set, so you kinda have to guess, but IMO, it isn't zero, and I'd hazard to say it's positive.

Silly nitpick but the free market absolutely can, and has in the past, put a monetary (positive)value on your person.

We have of course abolished slavery since then.

Which is a weird way of saying I don't think we necessarily should put a monetary value to people. Instead we need to realize that money isn't everything in life. So much of life is unpriceable, and that's as it should be. Attempting to price the unpriceable necessarily makes a mockery of the unpriceable thing.

You guys have to look up where the concept of Dignity came from.

> In my mind, it's pretty simple: I'm a human, LLMs are not. If a human writes a novel, it's inherently worth more because it's hard-earned and anchored to experiences we share.

.... What?

You open up with the calculator argument. Great. Then state "on the flip side". Ok.

And then declare it very simple in your mind when there is _only_ the flip side.

The whole issue is the not-simple gray area and where each of us believes the line between empowered human and brainless idiot is drawn.

"Many LLM maximalists say they use the tech to learn new things, but to what effect? Are you going to apply that knowledge of physics or computer science yourself, or will you just prompt the LLM again?"

Many of the LLM maximalists I know don't have the skills or knowledge to excel in technology and need to use LLMs to do their job. It's seen as a cheat code to get work done.

As an example, A person I went to high school with that could barely figure out how to setup a Drupal site a few years ago, is now a frontier engineer at an AI startup. His Linkedin posts are filled with AI buzz words on a daily basis.

"It's inherently worth more because it's hard-earned and anchored to experiences we share. "

At some point, it will be impossible to tell the difference. Many people already can't tell if something was generated by AI.

> At some point, it will be impossible to tell the difference. Many people already can't tell if something was generated by AI.

So what if its indistinguishable - its not a product of human intellect or effort. I feel there is a large disconnect where people look at artful output such as music or writing as a thing no different than a box of paper clips. To them, "It's just there." They don't care how it got there, they just like the feeling they get from the consumption.

That's fast food thinking; It's engineered to be "tasty" in the sense that they put the right amount of chemicals into the food to tickle the right nerve endings. It's junk food that exists to turn a profit. Whereas even the local diner puts effort into its food and has a damn fine Greek menu and the best mozzarella sticks.

If you knew how the modern music industry and local diners work, you might not be so impressed.

It's not all artisanal, made with love goodness just because human hands did it.

But the AI would not be able to perform live on a stage, producing emotions. I can't imagine staying 3 hours in a crowded room looking at a robot or a computer screen, even if it produces great music. Or if we get to that point, what gives the right and permission for humans to exist anymore?

> I can't imagine staying 3 hours in a crowded room looking at a robot or a computer screen, even if it produces great music.

May I introduce you to the runtime of The Lord of the Rings films?

Also the American Federation of Musicians' campaign against "robot" musicians replacing live musicians in movie theatres? https://www.smithsonianmag.com/history/musicians-wage-war-ag...

One of my college buddies, when getting set up for a marathon viewing of the trilogy (extended editions, naturally), referred to it as "twelve hours of butt-numbing Tolkien goodness". Exact words: it was more than 20 years ago but that phrase was so evocative it stuck in my memory.

>But the AI would not be able to perform live on a stage

The vast majority of music consumption is by people that have never seen the artist live. Hell, I have no idea myself if there is a human behind it.

>what gives the right and permission for humans to exist anymore?

Nothing. We are just a mote in incomprehensibly small fraction of eternity. Eventually the process of equalizing entropy will erase all traces of our existence. Enjoy it while you have it.

May I introduce you to vocaloid concerts?

> If you knew how the modern music industry and ...

This just assumes I'm listening to mainstream music or whatever when I mostly buy grungy metal and whatever else I enjoy off Bandcamp. Many times, directly supporting the artist.

> ... local diners work, you might not be so impressed.

I already stated the homemade gyro platter and they're mozzarella sticks w/sauce are slammin and they are proud of it. Sure it's business but they crafted some damn fine mozzarella sticks and they always come out crispy and oozing with hot but not roof melting cheese. That's giving a damn even if its slammin a few in the fryer then tossin em on a plate and sendin em into the human ether. I hope they stay around as they're a great Sunday afternoon lunch spot with friends.

I love gyros; you're making me want to go visit that diner. Do you mind revealing its name and the city it's in? (If you don't want to because that would reveal too much about your own location, I understand perfectly well and have no objections). Odds are I'm nowhere close to it, but sometimes my job takes me to unexpected locations, so I'm always happy to have a list like "If you're ever in Topeka, Kansas, check out (name of hole-in-the-wall burrito joint), they make the best darn burritos I've ever had". (For clarity: that's a made up example, I don't know anything about restaurants in Topeka).

Esquire Diner in Ozone Park NYC. They're struggling so I hope they stay open and the quality remains. So far, so good.

>That's fast food thinking

You're simply drawing the line where it suits you.

I don't consider it pure human coding if you use anything but Notepad.

What about punch cards?

Real programmers write the code and throw it away after compilation. All the fixes happen in the binary. You are not a real programmer unless you debug hex dumps and add changes directly to the compiled program.

Text editor? No. You punch the program on cards and then wait 1 week to get your turn and get a compiler error.

Nope, I draw my line at Notepad!

EDLIN.EXE or bust!

https://xkcd.com/378/

https://www.quiche-eater.com/the_real_text.html

> You're simply drawing the line where it suits you.

I'm drawing the line where the human effort is no longer there.

There was a similar argument against electronic music. No effort because no instrument is played. But the fact is the instrument just changed shape, drastically.

Like there's an artist I am sure a few here would be familiar with: Master Boot Record, a video game heavy metal electronic blend fully composed on a DAW. MBR live streams his composition sessions while chillin in his own irc channel he's hosting himself so you can chat with him. It's impressive to hear how he lays down notes, loops it, tweaks and expands it, then little by little builds a song passage by passage. Often with the irc channel reacting to the process itself and perhaps influencing the output. You know when it finally clicks and you're left feeling "Oh that was cool as fuck!" That's uniquely human, a life form that evolved over billions of years on a mud ball floating in a vast vacuum - at random. Talking, thinking, playing, laughing meat. Amazing huh?

An AI shitting out a similar pattern because it was trained on the same and similar patterns is not interesting or original. It's just a really good illusion from a well tuned stochastic parrot. It's interesting in the technical sense, but not in the artistic one. There was no creative process involved. Prompting is just a well formed set of instructions and questions which a human could understand - a spec. Nothing special there. It's like driving to McDonalds and ordering a quarter pounder with cheese meal. As a search engine, cool. As an artist, no thanks.

> It's just a really good illusion from a well tuned stochastic parrot.

The stochastic parrot claim is long gone.

Someone with actual skill in their respective field can create something new and original from current AI systems.

The real danger is that creative/intelligent people are giving up those abilities and letting the AI do it for them.

The stochastic parrot claim is as apt as ever. It’s just that it turned out that humans are just stochastic parrots, too.

> The stochastic parrot claim is long gone.

That's just marketing.

Can you elaborate?

Excuse me, I use Neovim but with no automation - no completions, no language server. Pure human coding. It’s not Notepad, but I’m not a masochist.

I'm not sure what you're arguing here?

If it's indistinguishable, that could be because the user doesn't care to look closely, or it could be because it's just that well made now. For simple profile pictures, I genuinely stopped being able to tell if I'm looking at a real photo or not last year.

> That's fast food thinking; It's engineered to be "tasty" in the sense that they put the right amount of chemicals into the food to tickle the right nerve endings. It's junk food that exists to turn a profit. Whereas even the local diner puts effort into its food and has a damn fine Greek menu and the best mozzarella sticks.

The former is molecular gastronomy: https://en.wikipedia.org/wiki/Molecular_gastronomy

The digital version fast food we already have and is a little different, in that it's the tuning of "the algorithm" to addict us, while text and image models* seem to be trying to actually fool us.

* I suspect video and music models are trying to addict, but I'm not super sure either way.

I’m not sure it’s so simple. I went on a trip with friends this weekend and cut a video from all the short clips we filmed on our phones. Before AI I would have used some song from my music library as the background tune. This time I created a song with Suno that fits the clips and includes some funny anecdotes from our vacation. If I was very talented and had the time, I could have written the lyrics and recorded the song myself (which undoubtedly would be even more awesome), but I can’t do this. So I used AI as a tool to do it for me. Everyone agrees it’s more personal and a better conserved memory of our shared experiences.

I don't get how people can listen to and share AI generated "music" and not feel legitimately embarrassed to their core. Like the AI Charlie Kirk song, but unironically. I'm truly baffled how anyone can enjoy it.

> If I was very talented and had the time, I could have written the lyrics and recorded the song myself (which undoubtedly would be even more awesome), but I can’t do this.

I promise you that any art you produced would be better than slop. There is not a single crayon scribble or off-tune melody that would be worse than the generic garbage that comes out of an AI model

How do you rate that against all the generic garbage that comes out of the pop music industry? Do you really think it’s meaningfully worse?

You have to remember that the vast majority of tech workers have open disdain towards humans and share zero solidarity with their fellow workers.

Just look at the leadership of tech workers, some of the most deeply anti-human and anti-democratic people on the planet. Not hard to see how these terrible people rub off of their workers.

Astonishing bigotry.

>> "It's inherently worth more because it's hard-earned and anchored to experiences we share."

> At some point, it will be impossible to tell the difference. Many people already can't tell if something was generated by AI.

Nit: it won't be impossible, just so hard so most people won't bother or give up if they try, and society will settle into a mediocre, regressed state. Then wait a little while, and the next generation will justify their mediocrity as actually some kind of progress, and the people who knew better will be dead and unable to challenge that.

More technology != more progress. Just look what social media had done. At its best, it's like what came before, just more isolating.

I mean, you act if society is something that always progresses into something you in particular want. Studying history shows us that society commonly progresses in to horrific terrors that defy reason. And so we may be entering that state again.

If we're contributing anecdata I have used LLMs to tremendous effect to learn all kinds of interesting stuff because I like learning interesting stuff and LLMs can tailor the level of instruction to exactly where you're at and don't mind questions interrupting ever 6 seconds which is better than both textbooks and teachers, mostly.

Have you ever tried to do the thing you learned without the LLM holding your hand?

Or are you just assuming you learned something because you made a finished product of some kind?

The question is does that even matter

You may argue what if LLMs are inaccessible but that’s like teachers saying “you won’t always have a calculator”

Sure it does. I still nees to know the correct mathematics to type into the calculator and the calculator is deterministic. If you don't understand it without the llm explaining it, how can you be sure you actually understand it? How can you catch mistakes made due to it being non-deterministic? Ask another llm? Same problem

An LLM always being accessible is not my problem

My problem is that filtering all of human creativity and expression through an LLM is ugly

> that’s like teachers saying “you won’t always have a calculator

They were right, though. For one, you still need to now what kind of equation to put into the calculator. If you don't know the concepts, what are you going to calculate. Real life situations don't actually spell out neat textbook questions.

And just take a look throughout the day how many quick maths you do subconsciously. Just things like cooking or grocery shopping have tons of moments where you just do the calculation in your head. And you can do that because you've learned how and when to do that.

It's not just a matter of "What if it's inaccessible", but also "Do you want to be dependent on your phone even more". Are just going to walk around with your phone on voice mode narrating your life into your LLM to let it tell you what to do?

> They were right, though.

They were not. I'm not dead yet so I suppose there's still a chance, but so far in decades of living life I've always had calculators when I've needed them.

> For one, you still need to now what kind of equation to put into the calculator.

That's a very different construct. Math concepts don't care how the arithmetic is performed.

> If you don't know the concepts, what are you going to calculate.

If you don't know the concepts, then what kind of arithmetic are you going to perform with anything -- whether the mind's eye, pencil and paper, a sliderule, or a pocket supercomputer?

I have a calculator (and many other tools, of course) in my pocket at all times, now... and yet I don't pull it out very often. Because I can add $1.99 + $3.49 faster in my head ($2 plus $3.50 minus two cents = $5.48) than I could get the result by pulling out the phone, swiping my access pattern, tapping the calculator app, and punching in the results. Heck, even if I had a dedicated calculator that only required pulling it out and pressing an On button, I'd still come up with the result faster than I could punch the buttons.

It's definitely worth it for kids to invest the time it takes to learn the little shortcuts involved in mental math: that effort gets repaid many times over during the course of a normal human lifetime.

I do simple math like that in my head all the time, despite having had special permission to use a calculator in school.

It was the same back then: Some quick mental math was often faster than using a calculator, so that's what I used first. The calculator came second.

I reject all notions that these are mutually-exclusive concepts.

> I reject all notions that these are mutually-exclusive concepts.

Agreed; just because I "rarely" pull out the calculator doesn't mean I never use it. Sometimes my wife asks me "If this cake recipe calls for filling a 9-inch pan with batter, and makes 3 layers, but I have an 8.5-inch pan and I'm making two layers, then how much should I reduce the recipe?" The π terms cancel each other out in that math, but I still can't do (4.25×4.25×2) / (4.5×4.5×3) in my head faster than I can punch it into the calculator. (It comes out to about 0.59, BTW, meaning she should use either half or two-thirds of the recipe depending on if she wants a slightly thinner or thicker cake).

Bringing this back to the broader LLM-related discussion, the lesson I took away from the Fable kerfluffle was "the model will usually, but not always, be available." You might have an Internet connection hiccup, you might have hit your subscription usage credits for the week and not be willing to pay API pricing, the model might even be taken away from you. (Most of which are problems you won't face if you run a local model, but that's a different discussion). So just as it's important to learn mental math even when you have a calculator as backup, it's important to retain the skill to solve many categories of problems without the LLM, even if you still reach for the LLM for the categories of problems that it can solve faster than you can.

Sure. So to bring it all back together:

I don't think I'd want to rely upon having a rental pocket calculator to use. It could become unavailable at any time, for any reason. For something as common, inexpensive, and easy to find as pocket calculators have been for decades, renting one sounds crazy.

But that's the way that clown-based LLMs are: They're rented. They could disappear at any time.

And sure, I do use the bot to get some stuff done, anyway. I think of it as somewhat akin of the time-sharing computer systems of yore, where people absolutely rented (often far-away) computing resources to accomplish their work.

Those time-sharing systems were only temporary. The giants like Control Data Corp fell out of favor as computers became smaller, faster, and cheaper. (And then they came back in the shape of things like AWS, but that has a lot of non-technical things driving it; AWS is, strictly-speaking, completely optional.)

Unless we stop making computers smaller/faster/cheaper (which we haven't, despite the present-day supply/demand market blip) I think it is certain that we will get there again with LLMs, where the giants fall. Not this year, or next, but small systems will catch up.

And these little machines don't even have to win the race. They just have to be Good Enough.

So it seems inevitable that people like you and I will live to see fantastic local LLMs happening with affordable hardware that fits on a desk.

And then, eventually, into our pockets -- like a calculator.

Given the power draw of the GPUs and RAM needed to run those local LLMs, I don't see them being comfortable to hold in one's pocket anytime soon. :-) Not until some not-yet-imagined breakthrough in heat dissipation is made. But a thin client that talks to the local LLM on your private network, that's already possible today. So yes, within the next 5-10 years I fully expect I'll actually want to use an AI assistant on my phone. Currently I go through the settings and turn everything related to Google Assistant off, and do that again after every Android update. But once it's talking to a model that I control, instead of a model controlled by the world's largest advertising company, I'll feel the opposite way about it.

P.S. If "clown-based LLMs" was an autocorrect-assisted typo for cloud-based, it was inspired. :-) If you typed that on purpose, it was also inspired. Have an upvote.

Power consumption per unit of work keeps going down, too. Our pocket supercomputers are astoundingly efficient compared to what it used to take to get the same work done at relatable points in the past. We'll get there.

Before we get there, we'll have network connectivity to our desktop AI boxes at home. That's pretty good, too.

And if this clown-bot[1] boom is a bubble (as I believe it is), then it's just a matter of time before it pops. The blast radius is unknown, but at one end it seems likely to result in something between potentially-idle fabs that have already been mostly paid for. Idle factories are bad and it makes sense to avoid that even if it is expensive, so this means cheap hardware.

At the other end, it means failed companies with fabs that are sold for pennies on the dollar alongside a resolute unwillingness amongst the investors (who just had their asses handed to them) to start on Boom 2.0. This latter scenario would mean golden age of cheap hardware.

I think we'll be fine.

[1]: Yeah, clown is on purpose. It fits as a replacement in any technical parlance where "cloud" would be used. :)

I'm of the same opinion re: the bubble. The dotcom bubble held on longer than I thought it would (I had been predicting that it would burst in 1999), so I'm not sure I should be making predictions as to when the rental-LLM bubble will burst. But I'm sticking with the hardware I have until memory prices finally fall again. Whether that's only because of spare fab capacity as you mentioned, or because one of the major companies has gone bankrupt and their already-purchased hardware is being sold off at fire-sale prices (perhaps to other companies, who then reduce their new-hardware orders accordingly, which again results in spare fab capacity), I'm going to buy more RAM when it's cheap.

... And I've just been agreeing with you on all points with this comment, haven't I? Time to wrap up the discussion, then: once total agreement is reached then there's not much point in continuing the discussion unless someone has something new to say, and I don't have anything new to say on this topic. See you around.

I feel like you missed the point I was trying to make.

Sure, a teacher framing the necessity of learning math as "you don't always have calculator" is pretty lazy answer that doesn't really get to the point. In my experience, teachers rarely even made this point, students did that tried to argue that Mathematics was pointless when you have calculators.

But my point was that actually knowing these concepts is good for complex reasoning you can do in all aspects of life. Whether you have the calculator with you or not.

Of course I have. What a weird question. If someone learned algebra would you ask them if they can do algebra without their teacher there?

If someone was studying under a teacher that had a reputation of stepping in and doing the work for them, yes I absolutely would

And AI has that reputation!

Actually yes, I've used an LLM to make a better old fashioned (whisky based drink). Turns out skimping on the orange oil/zest makes a huge difference. And, no now that I've learned it I don't expect I'll have to go back for this express need.

This is where LLMs and even things like youtube videos muddle the difference between what you are and things you have to do as a human. The number of things you can be presented with in the modern world that you'd like, or have to do, is simply endless. If it's something small, with little opportunity cost if failure occurs, then just going to an LLM for an answer doesn't bother me. Now if it's something that's more costly like DIY remodeling an LLM isn't a primary driver, but in as a device that can push back when I ask about mistakes that can be made and what to watch out for.

Perhaps you underestimated the person you went to high school with? The fact he or she "could barely figure out how to set up a Drupal site a few years ago" doesn't mean much. Lots of people are capable of immense progress when they apply themselves.

This is the classic gatekeeping our industry loves:

"This guy couldn't do a thing we found easy, now he can! Boooo!"

Yes, I can now set up a Drupal website in a few hours. That is great for me.

"Gatekeeping" is a derogatory way of saying "maintaining standards of quality", usually by those who stand to benefit from lowered standards. I heartily approve of "gatekeeping" because I like having competent co-workers.

Both sides have validity.

I don't want some random joe smoking crack working on my aircraft before I fly.

At the same time I don't want to have to pay a light bulb changer after filing 6 safety forms in triplicate.

Extremism on both sides leads to suboptimal outcomes.

Why does anyone need you then?

I consider myself an LLM maximalist, at least in the context of software engineering. Even after delegating everything you can to LLMs, there's still work left for you. And it's far more interesting. Even the delegation to LLMs is interesting - it's not that easy to make it work well.

In my domain (web app CRUD), we build the same thing over and over and over. And it's not beautiful or interesting like furniture. It feels like with LLMs, all that repetitive stuff is gone and I can focus myself on the tricky problems.

> If a human writes a novel, it's inherently worth more because it's hard-earned and anchored to experiences we share. I want to support that

I'd extend on this as well: the process of creating changes you. In a technical sense, where you approach a problem and the way you solve that problem informs you. Both your problem solving skills, creative skills, but also even understanding how a compromise works.

This is why I have minimal compunction about an experienced engineer using AI-assisted coding ("hey claude, define this data class") versus finding AI-art to be repugnant.

The act of creating an artistic work is both an expression, but also the act of ideating and then executing on that changes you. Experience, emotion, and other more intangible concepts.

I mostly agree with you but, how do you square the use of other types of automation that are not AI? e.g computer animation vs hand-drawn, CAD software for engineering and architecture… Are those lesser art forms or we’re just used to them?

I don't have enough experience in those fields to really firm up a judgement from my perspective.

The act of creation not just creates, it informs you. For example, when placing a wall with CAD for architecture, maybe you realize that some walls on an non-level roof have visually unappealing or difficult wall/roof joints. And then you consider that for future designs? Or you pause and think further about how a person interacts with that room. You can see how the act of creation delivered meaningful feedback to you, as a person.

IDK that's just an example I could come up with, you should really ask an architect or ID guy

I use AI to make songs and pictures for my friends. I barely even listen to music that I didn’t have a hand in making. It’s my favorite genre by far. My kids too. Personalized studio quality songs all day, every day. I suddenly have the power and time to make things I enjoy. I think there’s going to be a hollowing out that’s going to happen where we might not have another Michael Jackson or Taylor Swift or famous musicians because why bother when you can just listen to your own custom tailored tunes. All that matters to me is that I like the song. I don’t care if someone built a custom keyboard or did it on a Stradivarius violin.

I think for many people, maybe even most people, the story and the people behind the music is every bit as important as the music itself. I don't see how an AI can capture this.

> I suddenly have the power and time to make things I enjoy.

You don't make. You ask the ai to make a song about some vague concept and then clap like a seal as it delivers you un-challenging and bias-confirming slop.

> I think there’s going to be a hollowing out ... All that matters to me is that I like the song.

Everyone consumes, and nobody makes everything. But you're describing a future in which nobody makes anything and we all just suckle directly on the soft serve machine.

How grim.

you should really consider trying things that don't exactly fit your ideals. Listen to music you don't think you'll like, or watch a movie that you've never heard of. You find threads that you do like and learn to understand the parts you don't. Discomfort and friction is how we grow as people, it's how we gain perspective, and how our interests evolve.

Otherwise, if you're going to keep consuming the exact slop content that you're looking for, you're going to stagnate as a person.

> you're going to stagnate as a person

He's bragging about that, even. It's really grim.

> I'm not good at lifting weights or running, so my thinking is the only thing I have

I'm not saying this to be snarky, but maybe it's time to work on lifting weights and running. The worth of a novel is, of course, subjective, but most people would judge it based on the entertainment value it provides and not how much human effort went into it. Hobbies like fitness (which don't have an output that's meant to be consumed) seem like a safer harbor in this new era.

I was going to say, neither of those forms of exercise are expressions of a natural talent... they're things you simply do (at whatever levels you can currently manage) to stay healthy. They're good for your mental state, too! (And I could stand to do both a bit more regularly...)

Focusing just on AI wrt our work (let’s say software engineering for now) and letting go the other items. I see a like when this comes up the story is if the AI is doing the arch/impl then what are you even doing anymore? That is honestly not where my value add is, many people can do those two things plenty well, it’s basically commoditized at this point (at a certain experience level). What I bring is all the soft skills and intuition to glue it together and ship and maintain it.

Let the impl go.

Just live your life and don't support the AI machine. Nobody is forcing you, your life quality might get much better and you live your life as an intelligent person.

On the other side, people pushing the intelligence is for losers narrative are already too stupid to create anything that benefits humanity or the planet, so might as well ignore them.

> Many LLM maximalists say they use the tech to learn new things, but to what effect? Are you going to apply that knowledge of physics or computer science yourself, or will you just prompt the LLM again?

We had some contractors at the house doing some work. The contractors had been doing this work for a long time. I could tell they were good at what they did because I had some experience with that type of work when I was younger and they were way better than me.

We ran into an unusual situation (old house, old building techniques, overlapping old renovations) that they had never seen before. The contractors pulled up ChatGPT and started chatting with it. Eventually they came up with a solution with some good materials and techniques to make it work without turning it into a bigger tear out and complete rebuild project, and they got quick links to confirm it matched code.

This worked because they had the experience to judge the accuracy of the answers. They still checked the claims against the description of the materials we needed to order. After they did the work they had the experience to do it again, and it didn’t take them any guessing, trial and error, or searching through forums or Facebook groups (which are very popular with contractors) to get advice.

This is the right way to benefit from LLMs in learning something: Get the information then put it into practice so you learn it.

I worry when I see someone claim to learn something from an LLM but then the next turn is having the LLM do the work. It’s like skimming a math textbook but then skipping the exercises and never doing any quizzes. Every student in school learns that you have to practice to really understand something because reading the words is not enough. You have to work through it to internalize it.

LLMs can be very powerful tools in this way. They can also trick people into thinking they’re learning when they’re not. I think everyone has to go through that learning process about how to learn all over again like we all did with calculators in elementary school. There will be a lot of people who get trapped into being LLM promoters, whose output ceiling is limited by what LLMs can do as limited by their ability to recognize what’s right and wrong.

By now, almost everyone can think of someone they know who is happy to turn their brain off and press enter in Claude until it says the task is complete. I worry how this is going to intersect with the Reddit generation who grew up consuming doomer content about how jobs are just “bullshit jobs” and how your goal should be to do as little as possible at work, because there will come a time where those who spent their careers offloading their thinking to LLMs without doing the hard work of learning will not have much compelling reason to be hired over any other intern who can prompt Claude.

On the long term view I can see a lot of the LLM prompters getting replaced by LLMs doing the prompting. On social media like Reddit you can already see a lot of people gloating about how they just copy from Jira into Claude, push a PR, and then have Claude handle code review from their peers while they do nothing. Not hard to see that someone like that isn’t doing much that couldn’t be done by going one level deeper with Claude and letting it automate that person away. You have to put in some work to stay valuable.

Similar to your contractor example, I like to think of the useful way to use LLMs as doing "grep -C" (show lines surrounding the match).

You provide some context you are looking to expand on and let the LLM autocomplete provide you with relevant terms and concepts that have historically come up in that context and may or may not be actually useful. Then you do due diligence to learn about those related concepts, determine their applicability and distill it all down into a solution.

I find it helpful to investigate "I did it with an LLM" by imagining that (perhaps without the user's knowledge) the other end was another human, and asking whether that ought to change the answer.

This quickly engages a lot of social circuitry we've already developed when it comes to questions of what constitutes the user truly learning and having skills, what is meritorious, and who deserves most of the credit.

For example:

* If the spell-checker for your paper was secretly another human, that wouldn't become cheating. (Ditto for a calculator.)

* If you prompt "write an introductory paragraph" for a paper and it was actually a human ghostwriter, you'd obviously be cheating.

* If an executive prompts "write a congratulatory e-mail for me" to an LLM, it's not worse than asking a human secretary to do it.

* If you say "I learned kung-fu" while hiring a [robot/human] to kick the spine out of your opponent, you're a liar either way.

The same was probably said when machines started making furniture, cloth, harvesting wheat, etc. A desk made custom by a human to your specs is cool, beautiful, and life giving for both involved. And will cost $10-$50k. Maybe with the AIs and robots doing the grunt work, humans could get back to building things for each other. That would be great but I'm not sure what percentage of people are interested in that though.

> Many LLM maximalists say they use the tech to learn new things, but to what effect?

"Many book maximalists say they read books to learn new things, but to what effect?"

First we had verbal transmission, then books, then the Internet, now LLMs. And they all kind of do the same thing.

For me, the value of an LLM is it can take my imprecise query, scour its memory AND the Internet, and then return an answer (with citations if you ask) quicker than I can look for it "manually".

And I'm far from an LLM maximalist...

> We have people here who use LLMs to raise their children, to manage relationships, to design products.

My first thoughts were around trying to understand why these people would do that. What I see a lot around me, is people being afraid to fail, as it's some inherently dumb and bad thing, not realizing that sometimes failing is what makes you learn enough so you don't fail later, or builds resilience in other ways that later will be useful, for others or for yourself. Avoiding the path of harsh failure will put you down the path of mediocrity.

It's passing the buck, isn't it, if I ask AI and AI tells me something dumb, I can blame AI and absolve myself of responsibility.

At the same time, the troubling allure is that the machine has ingested a million books and has better knowledge than me (e.g. in things like JavaScript or Japanese), so why not trust it about how to raise kids.

> But to latch onto the calculator argument: if you outsource adding numbers to a calculator, you're still you.

Except calculators have been a problem for decades, it's why they're not allowed in school when you're still learning. Without doing the math yourself and internalizing how it works, you won't develop the number sense to tell if the result makes sense (broken calculator, typo, wrong equation, etc).

I still remember my physics teacher using one of the student's test answers as an example of how he should have known it was wrong and gone back over it (I think it was a pendulum on an elevator, his result had negative gravity (so gravity going upwards)).

We had exams where the calculator was allowed, because we were tested on our ability to type in the right calculations. The calculator wouldn't save you.

Even in the 80s HPs would do derivatives (and simple integrals). But, open book tests are the worst, nothing could save you!

I still remember giving Mathematica a relational equation of the atomic radius expectation values for it to integrate by parts and collect terms... at the time it failed to find the right integrating factor and gave gibberish. Probably works now.

I had a professor who polled the class if we preferred our exams to be open book or not. First year of undergrad.

The midterm was open book, he even shared the previous year's exams. The final, as expected, closed book, thank you very much.

If you can't work in order of magnitude calculations and estimations you're not going to last long in physics. Many useful models give good answers only within a certain range of inputs, while at the same time you often can tell where in a dynamic space you are by external observations. For example, estimating Martian wind speed within a factor of 3x from the size of the sand dunes, can be done having only seen pictures and with only the knowledge of a few dimensionless parameters.

That's also how you can tell if your calculator is "lying" to you (or you typed wrong). I guess I have a few similar tools in spaces where I'm more experienced to see when "hallucinations" and gibberish are being generated by LLMs. Of course having it "check sources" and evaluate its own solutions sometimes also works, if those are reliable, but you're on the hairy edge at that point.

I see an AI as an advanced calculator, but I see even a simple calculator as a poison to education if not strongly controlled.

Number sense is such a nebulous concept, but it captures what I've seen people struggle with. A general sixth sense about where an equation should or shouldn't go, a feeling that random numbers don't appear quite random enough and might have a pattern, or even recognizing that having a pattern signifies some relationship that otherwise has no evidence.

Most people only do the simplest of math on a day to day basis. I’d bet that if you asked most adults to do something even remotely non-trivial (addition, subtraction, multiplication) a lot would have trouble without a calculator and/or make a large number of mistakes. That’s probably fine because it’s unlikely most adults are all of a sudden going to have to do non-trivial math without a calculator. That probably isn’t the case for people who are having agents do effectively everything for them in their lives

>We have people here who use LLMs to raise their children, to manage relationships, to design products

It's not either/or. I think about designing a product in conjunction with AI "thinking" about it. Two heads are better than one.

>We have people here who use LLMs to raise their children, to manage relationships

Anyone who's spent time in professions in or adjacent to social work would say that a lot of their clients would frankly be better off outsourcing these decisions to LLMs.

> So what's your unique contribution to this world - is it the prompt you once wrote

Why not? You're implying writing good prompts doesn't require effort or thought, which is false to me.

> Are you going to apply that knowledge of physics or computer science yourself, or will you just prompt the LLM again?

To write better prompt is a good reason to learn new things. It's so obvious that more knowledge and experience you have, the better results you get from LLMs.

I don't relate with this at all, are you saying that founders who execute through employees are not creating anything by "themselves", they have no "unique contribution"? Most of us just want to create great products and hopefully get paid for them. And at the end, if I create a great product through this method, I feel equally happy, who wants to grind for a decade over some manual software creation process ?

> most of us

*some of us. This is one of the differentiators (can’t find a better word right now). Some became software developers because they enjoy(ed) the process. Some became software developers because that was the way to make products that made money. Neither is more correct or genuine than the other, but neither can also claim to be a significant majority.

LLMs are bicycles for the mind, they do not think instead of you

Bicycles don't consume the energy of a small nation.

I'm not sure how much does the total production of bicycles consume, but I don't see how it's relevant for the use.

The calculator argument is interesting because I am for sure slower at arithmetic by hand than when I was in gradeschool able to rattle off times tables and long division and what not, due to calculator use. I know plenty of other people who can’t easily do basic math, like multiplying something by 1.5x as a recent example in my life. Their gradeschool self probably would also run laps around them.

> calculators didn't make us dumber

For most people, GPS did not improve sense of direction, spellchecking did not help to write without making mistake, deepl did not help to be better in foreign languages. But replacing a bicycle by a motorcycle forces to acquire new skills without losing any, and we can find many example of symbiosis between "The man and the machine" (Lindbergh wrote a book named "WE"). AI could be something like that, after all it is human knowledge reachable in a conversational and contextual manner.

So AI can be used to learn: "tell me what's wrong in my code, or if it can be improved". I also tend to think that the more we code, the more we give AI valuable piece of knowledge to learn from, the best code it can produce, the less the produced code seems alien. It can be a win/win, all depend on the mindset. I like to code and even if I am skeptical about many aspects of AI I can share the workload with a robot, as an exercise or if the time or budget is constrained.

> is it the prompt you once wrote?

Yes, but also the prompts you give after, and the iterations you continue to do. Art is rarely something you build once and forget about. A lot of that going on in AI no doubt, but the builders who polish their work and strive for perfection are still the builders they were before AI.

You mention the novel writer and yes that person who's great at writing novels will create great art. But the ones that are 80% of the way their, but maybe can't nail the ending, or segue into different plot-lines, can now create great art and have AI get them there. Subconsciously, they will get better at those things by using AI and seeing the result.

I'm always reminded of that I, Robot scene where the scientist says "you must ask the right questions." And that is true now more than ever.

> calculators didn't make us dumber

I think that most of people misunderstand what calculator changed. Calculator didn't replace people doing math, calculator replaced mathematical tables, slide rulers and other already existing devices.

And regarding making people dumber ... Math teachers who saw this change said that there was a clear shift – students started to think less critically. With slide rulers and tables you had to think about answers – significant figures etc – with calculators you don't.

You still have to think about significant figures with calculators I used. Most will produce an irrational number result with as many insignificant digits that will fit on the display.

Only with final answer, if you have to round etc. With slide rulers and tables you had to think about it with every step, where to put your decimal separator etc.

One could argue that those who think calculators didn't make themselves dumb cannot be the judge of that. Cannot be the claimant, defendant and judge all at the same time.

I would say that calculators definitely reduced my more simplistic mathematics skills but did open up some more advanced stuff. I wouldn't say it was a gain or a loss, just depends on what your needs are.

it's the gut bacteria that is left. lol, that hunger, that emotion, that sense of intuition. Without emotions we are just a naturally evolved LLM.

The thing with calculator argument that always gets me. I do math unconsciously all the time. Even something as simple as adjusting a recipe when cooking. I don't need to grab a device to do that, I just do the quick math in my head. And the only way I can do that is because I learned how to do math. And even with a calculator, I still needed to know what to calculate. The argument "we don't need math, we have a calculator" assumes you always get a textbook question that lays it out for you.

Same goes for LLMs. I can use them for programming, and they're very convenient. But I still need to know what to ask it and make sure it stays within the confines of what I want. And without my knowledge I would have no clue if what it's trying do is correct, or safe.

Naturally, this assumes a workflow where you do actually look and modify the output yourself. But I'd argue that any non tech person is inevitably going to hit a wall where they can't debug themselves out of without getting a human involved.

Underrated and paradoxical element of this, to which, as far as I can tell, LLM boosters offer no solution:

> And without my knowledge I would have no clue if what it's trying do is correct, or safe.

I would contend you got the knowledge by typing the code yourself, that there's no other way to get it, and that if you stop typing the code yourself--and the slogging that entails--you'll lose the ability to prompt LLMs effectively.

It's not that I think the physiokinetic aspects of typing as an input mechanism hold some metaphysical distinction, but rather the level of engagement it forces with the code, and the units in which it does so. I'm not aware as yet of any viable replacement for that.

It's easy to trade on decades of software engineering experience with LLMs: with sufficient experience, everything goes around and comes around, almost any pattern is recognisable, the gratification is immediate, the benefits are now, while the costs and disasters are down the road.

However, the technology world is not static, and if you don't keep up with new frameworks, libraries, languages and other tech in that physical-mechanical "mind-body-keyboard" way that typing--or something substantially close to it--accomplishes, you will lose the ability to navigate that world fluently. To say it's just another abstraction layer and that the world didn't crumble after compilers is to miss something quite essential about how LLMs differ from compilers or high-level languages. The disengagement with the process of physically programming something quite specific will take down with it the ability, over time, to formulate useful prompts and competently review the output.

> I would contend you got the knowledge by typing the code yourself, that there's no other way to get it, and that if you stop typing the code yourself--and the slogging that entails--you'll lose the ability to prompt LLMs effectively.

For sure, and I've tried picking up a new language with the use of LLMs, but the concepts just don't stick because I don't actually do the work. That's why I do try to limit my use of LLMs to fields I'm already closely familiar with, and also keep its output contained to actually reviewable chunks. Or things that are just tedious, like OCR and large text transformations.

> the concepts just don't stick because I don't actually do the work

Yes! And this is the part of the de-skilling puzzle that is completely unaddressed by AI boosters.

Maybe LLMs are a force multiplier, but there still has to be some force to multiply, and I don't think a lot of folks ask the question of how that force is actually cultivated. This nebulous, airy-fairy notion that humans add "architecture" or "taste" doesn't tell the story of how, concretely, they came to have it. It seems to me there is no escaping that it came from typing the code.

Like you, I have a much better grasp of code and API surfaces I've physically typed than things LLMs have emitted and I have reviewed, in a conceptual sense, but which I could not have typed myself then, nor can type now.

Yes! If anything both, the supply of veterans and the ratio of judgement capacity to generated content, will only go down with time.

So where are the new veterans going to be coming from? You can’t spin up experienced workers overnight.

Essentially the same supply chain issue which nation states are worrying about today, except for skill.

> I just do the quick math in my head. And the only way I can do that is because I learned how to do math.

A lot of Hacker news commenters tend to overestimate human ability without education. That is, the tend to believe that people are able to do a lot more without regular training than they actually are. They believe that "math" is some naturally ordained eventuality that humans just do, or likewise that "reasoning" is some immutable natural behavior. In truth these things are "unnatural" human reinforced structures that we have to learn and adapt our brains into.

I'm abhorrent at arithmetic and I think it affects me greatly. I know quite a few people who are barely literate and manage to get by in life, but it obviously makes things hard for them and strips them of opportunities (and they forego holding any identity that seems to "smart" for themselves). I think numeracy is similar.

I wonder how much better I could become as an adult at mental math. I did a math course a few years ago (Math Academy) and put some effort into improving my addition, subtraction, division, and multiplication, but I wish I'd done more up front. Even one-figure arithmetic is so fuzzy to me, like it's a hashtable where the keys don't line up to the right values. Can I unlearn that now? O(1) doesn't matter if the result is wrong.

I think you are underestimating how fast this stuff is moving. Calling AI "just autocomplete" for system architecture is outdated.

If you were to ask it (especially with Claude's newer model, Fable 5) to draft an architecture design diagram, it would do a better job than any junior dev ever could. Sure, giving context is still necessary, but you can just speak in plain, normal English and it will understand. AI is getting scary good at system design.

> Calling AI "just autocomplete" for system architecture is outdated.

I didn't say that, though.

> it would do a better job than any junior dev ever could. Sure, giving context is still necessary, but you can just speak in plain, normal English and it will understand. AI is getting scary good at system design.

Sure. But that's not my point. Don't you still want those juniors to be capable of creating those diagrams? So they can understand them? If the juniors don't get what the AI is outputting, why are you even creating the diagrams in the first place, right?

That's what this topic is about. Just like how calculators helped us do complex tasks, we still needed to understand math. Same with AI, you still need to be understand the topics you ask it to do.

> If a human writes a novel, it's inherently worth more because it's hard-earned and anchored to experiences we share

But the market doesnt reward that. Different consumers different markets, but (using a salad analogy) overall consumers dont care how a salad is made, only whether it tastes good and is cheap

Just to leapfrog off of the calculator notion...

There's a difference between capability and mastery. If you have a calculator, you are capable of doing basic arithmetic (probably). If you can do fairly complex arithmetic in your head, that's mastery. If all you need is capability, it's fair to say that acquiring mastery might be wasted effort. However, if you want to go on to do advanced math, physics, etc., then mastery of basic arithmetic, calculus, algebra, etc. are necessary stepping stones. If you have to go back to calculators, math programs, textbooks, etc. every time for basic things while trying to formulate and manipulate equations, you're going to have a very hard time getting along.

Sure, rely on AI for things that you merely need as capabilities, but recognize that doing so prevents you from developing mastery required to progress to other things. If you don't want to progress to those things, then it's fine. If you do, then you should rethink your approach.

I'm acutely aware that even 6-12 months off the keyboard is brutal to come back from, it's a real issue now because there are pockets of work where the agents can drive the whole thing. If you just autopilot there? The comprehension and skill debt becomes really hard to pay down in as little as a week or two for the really high complexity stuff.

I try hard to keep a serious feature branch going on every project where I'm driving it in emacs or vscode or whatever. `gptel` is great for this because it can tool call for the docs, be a super search engine like Google used to be but you're keeping your reflexes up, keeping the comprehension debt under control.

For me this is almost exactly like being in an EM role: it's so easy to lose track of the details, you have all these other demands on your time, there's pressure to grow headcount, and the costs of letting the skills slip is all but invisible for a while. But before you know it you're rubber stamping stuff you don't really grasp, you're not able to help people who are having trouble in the codebase, it's all bad. And depending on why you got into this business, it can become pretty joyless to be totally out of the loop.

Now every engineer has to navigate this. It's going to be interesting.

> In my mind, it's pretty simple: I'm a human, LLMs are not. If a human writes a novel, it's inherently worth more because it's hard-earned and anchored to experiences we share.

In my mind, it's pretty simple too and not dissimilar to what you say: we've got cognition, LLMs don't. When a human writes a novel, it doesn't look like a novel because the human has been trained by reading every single novel ever written in every single human language out there.

The human doesn't imitate the act of writing a novel: the human just writes a novel.

> But to latch onto the calculator argument: if you outsource adding numbers to a calculator, you're still you. On the flip side, if you use an LLM do most of your thinking, what's left?

Really not much. The IQ of an oyster more or less. But the LLM is not thinking, so it's not possible to "offload your thinking".

But tbh those who believe these LLMs are thinking machines and that it's normal to offload "thinking" to those non-thinking machines probably don't have very deep thoughts anyway.

So nothing of value is lost.

P.S: I happily not one but three AI subscriptions and use LLMs daily to code. Which is probably why I'm such a critic of AI. They're both useful and pathetically bad. They really write crappy code.

> Many LLM maximalists say they use the tech to learn new things, but to what effect?

I would also argue that they use LLMs to learn "about" new things, but not really the thing itself. I, myself, learned a lot about complex SQL queries just recently using LLMs. But I really don't know how to write complex queries myself.

People often confuse reading about a thing with knowing the thing.

Eh, not convinced. Certainly what works for you works for you.

But the idea that “outsourcing” to experts is somehow detrimental is one of the modern tropes I can’t stand. And I’m not sure I believe there’s a material difference between e.g. reading research about raising children versus asking an LLM that is grounded in (and cites) that research.

Maybe I’m a cynic, but I don’t think most of us have a unique contribution to the world. Emphasis on unique.

So why not leverage all resources in the effort to maximize happiness for ourselves and those around us?

>But to latch onto the calculator argument: if you outsource adding numbers to a calculator, you're still you. On the flip side, if you use an LLM do most of your thinking, what's left? We have people here who use LLMs to raise their children, to manage relationships, to design products. So what's your unique contribution to this world - is it the prompt you once wrote? You're standing in front of a token-generating machine, pulling a lever, sometimes receiving gifts. Is that your edge, your unique experience, your purpose in life?

Cognitive offloading seems to be a feature of humans, we do it with spouses and people we work with.

I dont see why offloading some other stuff to an LLM is going to dehumanise someone.

When people interact with spouses and coworkers, it's still human-to-human interaction. There is a person on the other end. Not quite the same when your head doctor, DJ, gym coach, and nutritionist are the same sycophant machine who's eager to let you know how absolutely right you are.

>There is a person on the other end.

Not sure why that matters tbh.

Cognitive Offloading is a human ability. We built a tool that can be used with it in absence of another person.

The argument seemed to be that Cognitive Offloading dehumanised a person, but now things have shifted to "Its only good with another human".

If its worthwhile we will do it, if its not worthwhile we wont.

My take is that you learn math before learning how to use a calculator. And can understand what it does and reproduce it (slower) by hand if needed, but everyone is still using it for stuff we don't need to think about.

If you use AI exactly like this I find no issue, but I fear most people are using the calculator without knowing the math and that's the real issue to me.

TBH, the calculator example isn’t an encouraging one. I use a calculator (these days N PP, or spotlight). But by the time I do that I already have an approximate ide of why the answer is, so if it’s way off, or the sign is wrong, I see it immediately.

Same with GPS, though many folks have no idea how to navigate on their own anymore so often don’t notice when they’ve accidentally had a typo or chosen the wrong Springfield.

So I find using an LLM can be useful when I already have an idea of that the problem is.

At their request I reviewed a colleague’s plan for a project. I looked up some of the baseline parameters so I could understand the plan, than I made some comments w/questions about things I didn’t quite understand.

A couple of minutes late for our morning mtg I came in while my colleague was saying that I had found a bunch of things that Claude had not.

This is why they are bad news for new grads who haven’t yet learned the and pitfalls of their fields. And this was a senior person who just outsourced the work to the LLM.

> I'm not good at lifting weights or running, so my thinking is the only thing I have.

Even if you were, the best you could hope for is a career in entertainment ("sports"). The necessity for bodily strength was abolished in the industrial revolution of the 1800s and 1900s. Now the maximalists predictably say we have another "industrial revolution" that makes our intellect redundant. I don't believe that but if it were true, the question what defines the value of a human would become even more pressing, with less leeway as to what the answer might be. H. G. Wells needed The Time Machine to find a world like that.

Still, if enough people believe that LLMs do make our intellect redundant, it will trigger a cultural crisis similar to the one at the previous turn of the century when men struggled with their defining attribute (physical strength) being taken away by machines. The reverse centaur is over 100 years old, even if that name wasn't used then.

This just sounds like the black and white fallacy ad nauseum. Like every sentence is constructed to frame the entire issue around a false binary. Like you assume that if "maximalists" use LLMs more than the average joe, they're not "thinking." Black and white. Either-or. Maximalists are hobbyists. They usually read up on the newest models, try out different ones, maybe even build pcs specifically to run local models. You honestly think they're not thinking as much as you? Also there is absolutely no reason to value "being unique" for its own sake. Unique people aren't walking around trying to avoid tech to max out their uniqueness potential. That's just neuroticism. It does seem to me as though a lot of people have this misplaced faith in humanism and it just feels like copium at this point. Worse yet, it honestly just seems like a way to dunk on those they perceive as conformists. So basically just a high school mall goth. You are not in a position to lecture other people about their purpose. Your brain is a pattern recognition machine just like everyone else. You've grown fond of some patterns you picked up on that emerged during the enlightenment and you seem to be just superficially regurgitating them, not unlike an LLM. Calculators made us dumber in the sense that we no longer have to perform mathematical operations on the fly. That gives us room to do something else. What other people do is up to them. That's what freedom is. You assume the maximalist is just sitting there prompting all day. If they are, whatever. That's their hobby. It’s not much different than playing video games all day. Either way, Im pretty sure their lives are filled with sadness and joy just like you and everyone else.

Having learned math in an accelerated program that banned calculators, my experience has been that they do substantially diminish mathematical learning. They are useful for things like building an intuition about the behavior of functions... but ask a student to manually graph and compare the behavior of some fairly trivial functions ( ln x, 1/x, sqrt x, x^3 ) over the interval [0, 1] and you'll pretty quickly see the difference between kids who are trained to think for themselves and kids who aren't.

The ability to engage with math "critically" is fundamental to being able to use it constructively/creatively.

I think we're seeing the impact of LLMs more broadly in the drastic decline in incoming freshman academic competency levels, and will continue to observe people becoming measurably, meaningfully stupider until we clamp down on / outright ban LLM usage in education.

> On the flip side, if you use an LLM do most of your thinking, what's left?

If your answer is "not much", you gotta step up so that the answer is closer to "more than ever was".

You dug the holes in the ground with a shovel your entire life. Suddenly somebody gifts you a digger, nearly for free. If your reaction to this is "what's left for me now? anyone could just pull levers, no strength or skill needed!" not, "I could dig so much more and deeper holes with this digger!" then you need to chill, reflect, then fight your fear and perverse laziness and learn how to become the best person to pull the levers.

Taking the context of raising children, the ‘unique contribution’ of a lot of parents is traumatising their kids. If it’s helping us make better decisions, I don’t see how that could be worse. I think that translates to all the non creative things you spoke about.

I know that the common refrain is “think of yourself as a manager now” but I’ve actually taken the opposite approach and have been telling anyone I train the same.

Diving deeper into technical understanding makes more sense to me at this point both as a way to make yourself more useful in the age of AI and also to use AI more effectively.

I regularly tell the kids to grab a text book on a subject that interests them and I do the same.

I’m willing to bet deep understanding is going to become a commodity soon.

It is easier than ever to learn difficult concepts. It is also easier than ever to produce things that used to require understanding of those concepts without them. Discipline and drive to use these powerful new tools patiently and with purpose is what is required.

Do we actually have any measurements of if AI helps you learn something better?

Because it is one thing to feel like you have learned something and another thing to have actually learned it well enough to put it in use. There are so many YouTube tutorials on all kinds of subject that will make you feel like an expert in the field after watching, and then you start doing the thing they have supposedly taught you and you can't remember a darn thing, because your "learning" was only ever skin deep and never meaningfully tested or reinforced.

> Do we actually have any measurements of if AI helps you learn something better?

To me this is intuitively true based on anecdata. 2 examples

1) learning spanish - when I hear or read a phrase I don't know, I type it into the LLM and I learn a new word/phrase. Sure, I could have cracked open a spanish language dictionary, but tbh, I wasn't going to do that. Not to mention that dictionaries are translating word by word and not phrase by phrase.

2) growing vegetables in the garden. I literally watched YT tutorials and did what they did, and now I have vegetables that I didn't before. Yes, I could have probably could have gotten this from a botany book, but once again, I probably was not going to do this. I also was trouble shooting a lot in gemini

Here are the problems I traversed: - How much water should I give these per day? what's the watering schedule? - how much sun vs shade? - when do I move seedling to the ground outside? - Is trimming good? which parts and when do I trim? - [take a picture of weird growth] - Is this a disease on the plant? Or part of the plant naturally?

Do you retain that information? Or are you just constantly relying on having access to an llm to re-look things up. The amount of information I retain when using llms to learn is far less than other methods, not sure why, you'd think it would be just as effective as reading a book, but it's definitely not. Probably for the same reason that learning math and always using a calculator does not make you great at math.

I've wasted a ton of my life already trying to make llms work for learning over the last few years, I'm especially bitter about it. I think this technology is a scam made to make us reliant on a think-for-me machines.

For me it depends, if I simply take the answer from the LLM it goes straight into the shortest memory neurons my brain has.

Now, if I go back and forth with the LLM to say, taking the language learning example, to explore the etymology of the word (which for me is far more interesting than the translation itself), then I learn a ton more.

I'd add that this is also how it used to work with Google search too: there's been facts that would show up as the first result, then you'd just forget it 10 minutes later and have to re-google it. I know this happened to me many times.

For better or worse, there's less friction now for seeing an answer to a question you have. You can ask in more arbitrary ways than Google required, and something will still come up. For looking up factoids, it's much faster. For picking up more complex topics, I'd say it's more or less the same, because you still have to spend time ruminating on the topic.

I think it depends on how much time you spend learning something and "engaging" with LLM.

I have many times asked it something I was slightly curious about, got the answer after the first or 2nd-3rd prompt, spent 3 minutes in total and forgot it after 15 minutes probably.

But a few times I've spent an hour or more on a topic, asking many questions, thinking between responses, and I actually learned something.

I definitely retain it. In the case of the gardening - I pretty much immediately go outside and practice it.

In other words, how I get the info doesn’t matter (book, video, llm) because I’m locking in the skill by doing the thing in the garden.

And sometimes it is kinda wrong/suboptimal, but not more than e.g. asking a friend for gardening tips. Usually this is based on lack of context, like eg im growing cherry tomatoes and the AI assumed I had full size tomatoes

I dont think you are getting the topic - sure if knowledge gleamed from llms is immediately practiced repeatedly, it stays, why would anybody question that.

Now add 30 or 50 or 100 other topics you dip into on any given day. No way in hell you do same technique of repetition till you learn it. This is gist of problem to me. Most students these days fall into this category and the amount of time/energy to properly learn something has stayed the same or even increased (due to distractions and pushing through mental barrier of 'I know it well enough')

> Now add 30 or 50 or 100 other topics you dip into on any given day.

In a world without YouTube/LLMs, you weren't going to learn these things anyway. Today, you also don't learn them. So how can you argue that something is lost?

> the amount of time/energy to properly learn something has stayed the same or even increased

The whole point of my post is that LLMs, YouTube, and other internet resources which are easy to access have lowered activation energy to where learning something is at least possible.

I can tell you for sure I would never even try to learn a second language if I had to go to a class physically, or carry a book with me in my pocket to translate as I traverse the real world.

I also have one anecdote - I (think) I understood basics of quantum mechanics intuitively for the first time. Of course it's probably superficial and maybe not 100% correct, but it is better than every time I tried to understand it before.

It definitely wasn't a single prompt, but two hours of back and forth, with a lot of time spent thinking (me, not LLM) in between. There were multiple times where I misunderstood something, so if I just read a book I'd probably get stuck many times.

There is some evidence to support the idea. However, "learning" studies can be a little disappointing since they usually take place over a short period of time. This one had a week-long timescale:

https://www.nature.com/articles/s41598-025-97652-6

I think there's a bigger question that would be hard to answer: what does a consistent long term diet of videos and quick answers do to a person's mind/understanding of the world?

Also, a lot of the published research is a year delayed - most I've read recently are using ChatGPT 4 or 4o. When it comes to giving high quality feedback, I think there was an inflection point around Opus 4.6.

I too give juniors the advice to crack open $textbook. It’s just painful to see the complex things they’ve created with horrible performance and no cohesive data model because they don’t have the requisite academic foundation to hand code the same thing given unlimited time

a commodity is something of low value unless in a large aggregate.

I couldn’t agree more. It seems AI is very good at middle of the distribution tasks where it has access to a lot of training data - enough to be highly reliable at that task.

Which means as a human your only added value is on the edge of the distribution. Which means you need to be learning and doing more complex, deep topics.

Are you really diving into technicals just by asking "sooo what did you do?" to your AI? Without document crawling, debugging and pulling your hair out, how much of it is really a deep dive? I feel like all that effort that goes into generating a mental image from bits and pieces is gone. I'm just grudgingly happy I went through all that before humanity retired.

I wonder if it'd be more effective to teach how to critically use AI than to try to forward people to textbooks.

For myself, I have found that I am better able to learn new topics than ever before because being able to have a conversation with a moderately competent but sometimes catastrophically wrong AI about any new subject is actually the perfect mix of helpful and unhelpful for learning.

I use a loop along these lines:

* Ask a question * Get an answer * Be skeptical of the answer * Investigate/reason about the answer * Critique the answer * Rinse and repeat

This kind of loop is far more useful to me than any textbook ever has been, because a textbook just drips information into my head. It's more likely to be accurate, but not guaranteed, and it doesn't encourage me to actually engage with the material in the way that a wrong AI answer does.

That’s the beauty, you can do both. In practice I usually just let the AI know what chapter I’m on and then ask questions or have it ask me questions based on the chapter.

> the common refrain is “think of yourself as a manager now”

One of the many reasons I'm determined to remain a luddite wrt AI. I hate the idea of being a manager and have refused promotions to avoid it in the past. I don't want to manage automatons any more than I want to manage people. I want to do stuff, not manage.

I find a lot of people are actually happy that they can offload mundane parts of job to automatons and to be really busy with the interesting parts.

It is not like you have to give them fun parts of the job.

While being a team lead for people I actually have to do the opposite. Get interesting parts to hand over to people to keep them happy and pick mundane not interesting boilerplate myself.

I keep hearing my this very much, so I have a question: what fun parts are there left for me?

It feels like this message of “offloading mundane tasks so you will have more time to do fun parts” is being pushed so hard, but in reality the opposite is happening. Fun parts are being offloaded while I’m left doing the soul sucking and mundane parts

I need a bash script for example. I don’t like dealing with bash or debugging scripts written by others.

I have that part covered by AI so I can get multiple servers up and running and then eventually fix some config or networking here and there which I like.

Instead of bashing my head against the wall.

Different strokes I guess. I love using bash.

I love churning out crazy Bash scripts, though. The first scripts I wrote sucked, as it always goes with a new language, but Bash is an awesome swiss army knife now that I know how to really push it. It sucks that doing this by hand now is so heavily discouraged because so many people see this as drudge work (that they never cared to learn).

I do know how to write stuff in bash.

I dislike the tooling and debugging experience that is not existing compared to let’s say Python. With Python I have whole IDE which help me set breakpoints and inspect variable state. With bash I have to pipe output to text file and inspect that. Feels like working with Stone Age tooling.

I'll take this as an uninvited opportunity to plug nushell. Ive never got on with bash, it feels like too much of the friction is incidental to the problem, whereas with nushell I find I get to the fun intrinsic difficulty of a problem. YMMV but give it a try if you haven't before

I'm a bit odd, it would seem, in what I find enjoyable and not.

I've happy to take the tasks like SQL optimization which other would run away from, for instance, or the bash trickery mentioned elsewhere. People don't believe me, I've had looks of surprise and derision when “if you take on this role, you can give X to others and not have to worry about it” is responded to with “I like doing X, can't you ask one of the others to take on the other role and let me stay as I am?”…

It is increasingly looking like I have the choice: change to a role (automaton farmer) that I expect to hate, or leave dev, or maybe leave tech entirely and go wait tables or something. Those tables are starting to look very attractive.

> deep understanding is going to become a commodity soon

How does the text generated by LLM make “our” understanding deeper compared to text written in the books?

I think you have my argument backwards

Yup, my mistake.

No worries :)

Your argument in that quote is completely unclear. Why do you believe that deep understanding is going to become a commodity soon? How would that happen?

I think the last sentence has a word missing or something, because it contradicts the previous argument a bit.

Managers, good technical ones anyway, also dive deeper into technical understanding. They don't do it to write more code, they do it to figure out what the developer is saying to ensure the developers aren't blowing smoke.

Deep knowledge is going to be the opposite of a commodity.

The system does not reward deep understanding. It's too slow.

An analogy that I think helps describe how I feel about it all:

AI is like having cheat codes in a video game. You don't have to try hard and develop a deep understanding of how the game works. You aren't challenged anymore and it doesn't feel like you earned winning the game.

It doesn't matter because nobody cares. Businesses truly do not care. You are a cog, a means to an end. It's only about "winning". Now it's no longer fun to play the game.

> It doesn't matter because nobody cares. Businesses truly do not care. You are a cog, a means to an end. It's only about "winning".

A cynical person would tell you that nothing has changed; this was always the way it was. AI didn't change anything in this regard. Maybe it just made it more apparent to you?

I would agree. At least previously we could pretend that we were developing a craft in the process.

I've been a manager for some years. Without sounding too up myself, what made me good at management was the fact that I knew how to do the things I asked people to do. It allowed me to critically evaluate the work, set realistic timelines, and champion the contributions up the chain. I also maintained the ability to perform this work (by design and happenstance sometimes, since we could be understaffed due to sickness etc).

The good managers I saw also had this ability. The managers I didn't enjoy working with / saw struggle were ones who either never had that ability, or had lost it a long time ago.

Note I'm speaking about line management. There's a world of difference between managing a dozen or so individual contributors vs managing senior managers / directors. However, managing LLMs is analogous to line management.

> There's a world of difference between managing a dozen or so individual contributors vs managing senior managers / directors.

I agree, but in my opinion, your point about needing to know how things work still holds. It's less relevant as you move up to managing managers and larger orgs, but it never goes to zero if you're going to be successful.

I think it holds true throughout if you scope it to knowledge about your reports’ domain. If you’re managing managers you should have a good handle on what a manager does and different ways to do it.

There's two huge aspects of management that gets diluted when it's compared to "managing" an LLM: - labor laws. You need to know those in the country where the employee has their contract, because this will restrict what you can (and must) do in your managerial role - Empathy. Managing humans is extremely messy and requires a ton of EQ to do well (parents have first-hand experience of this).

So, i.e: comparing being a manager to real humans vs instructing a next-token generator dilutes the skill of being a good manager and leader.

I thought it was a myth until the junior developer on my own team responded with "I don't know" to a question about why he made a certain computation during a design review. Because the (wrong) computation was fully AI generated and he couldn't even tell the difference.

Most people don't use AI to learn new stuff. They use it to do "the job" for them and they don't even understand the result. What is the point of a person if they don't bring any value to the table other than being a "resource" to generate prompts?

One of the most important things for any profession is experience. Experience comes from learning from your own mistakes or those of others. Those with the best skills can be said to have learned a lot from previous mistakes.

So, how can someone become professional if they don’t learn from their own mistakes? In the case of AI; they will learn to become professional prompters and learn from the mistakes of bad prompts??? I prefer the former not the ladder.

I think the calculator parallel is really on point.

You learn math before using a calculator to do the boring part.

You should learn code before using an AI to generate the boring part.

Even more so for LLMs because they are not deterministic like calculators.

I recently had the opportunity to help teach to almost graduate students, (even though I graduated only last year myself), and with the main teacher we made it a point to do a skim code review with ever single one of the 90 students, just me/the teacher, them, and their code on the computer screen.

I saw every category, the ones that used AI to learn and actually answered my questions and knew what it wrote, the ones that mostly didn't use AI and so knew what they wrote, and finally the ones that used AI to do the assignment instead of learning and always responded a variant of "I don't know".

I really like what the other teacher said to a student that admitted never reading their code:

"I'm not even scolding you, You're literally saying you are completely unnecessary here"

I fear more for the future where you're forced to offload your thinking to AI. Everything you say must be cited with LLM, everything you do must be signed off by The LLM. If you're in a meeting and have an idea, but Fable 9 says it's not a good idea, you cannot pursue it or you lose your job. The low-friction path will always be to do what the LLM tells you, and many people will shut off and surrender completely. (this is already how SWEs behave in companies where management forces them to tokenmaxx).

AI will be treated like a God that talks to us, and you can't disagree with it, you can only try to change its mind. The most horrific form of mental oppression. And it's just a few years away.

I would like to push back on this, but I cannot. Alas, for most people they will take the easy path for sort term gains than the longer road for resilience.

I see it all the time now, people constantly outsourcing their thinking to LLM's and getting upset when you point out that it is wrong. I have people calling/emailing my work place about a service of product that we do not provide. But those folks simply cannot comprehend that the LLM is wrong and many have made the accusation that us at the business are wrong. There is a very strong feeling from many people now that it is the higher authority over professionals. It is a saddening position to be in.

We are only 3 1/2 years into this and it is already happening on a much larger scale than many think.

I mean that's basically 1984 except we now have the technology for telescreens and minitruth at scale

This looks like the Divinity Machine from Doom.

> What are we automating? Human work or human agency? Human tasks or human thinking?

I find it's so easy to convince oneself they're doing the former when it's increasingly the latter. The thinking part is so often provided by default by the models, or is a single prompt away. The thoughts are so syntactically (though not stylistically) perfect that it's difficult to ignore them and reason greenfield.

What's the solution? Given how keen models are to short-circuit the thinking process it could be the only solution is to silo off tasks/ideas. Choosing which mental tasks to silo off is itself incredibly difficult especially when there's a pressure to deliver rapidly (and in quantity) on those tasks.

The question presumes that most of us are "thinking" in the first place, when in actuality, most of us are just acting according to the patterns that have emerged from our encounters with the thoughts of others. We generally adopt them and/or try to hallucinate coherence when they conflict. Very few people actually "think". It's hard work and takes time. We neither have (take) the time nor are we particularly motivated to put in the work because the patterns we have learned from others are useful enough to achieve the low goals we set for ourselves.

IOW - modern AI is simply an extension of the lack of thinking that characterizes the modern life... It just does it faster and uses a hulluva lot more energy.

Everyone is always thinking, just many of us not about what we're doing! Sorry

Have you ever read Kurt Vonnegut's Timequake? I think it's very applicable to the average human experience.

> the modern life

I don't think modernity caused any sort of degradation.

You said it yourself, "thinking is hard work". It's rational to save energy. This might even have incentivized the emergence of mimesis in humans, which is arguably the foundation of our ability to cooperate at large scale.

https://en.wikipedia.org/wiki/Mimesis

Maybe a few of us do the hard work of thinking, and, if we figure out something novel and useful, huge numbers of people ape us uncritically. It's not an inspiring picture of humanity, but it's also not a reason to disparage anyone. More of a fact of life to be dealt with strategically.

I agree with many points.

Even on Hacker News, when you see debates like 'X technology is good' or 'X technology is bad,' most of it seems to be about identity. And that identity often originates from the community they belong to.

The first identity usually starts with a community or the person who created it. Once the community forms, people under it often forget the original reasons and just accept it as their identity.

This is especially true for technology related issues, because the market share of a technology is directly tied to one's career, which makes it even more prone to becoming an identity issue.

I also do some 'thinking' in certain areas, but most of the time I don't. As my field gets deeper, it becomes harder to allocate cognitive resources to other areas. So in general, most people follow the crowd's opinion, but only maintain deep, thoughtful thinking, including 'taste,' in a few specific technical domains.

Re identity, relevant post from today in case you missed it https://news.ycombinator.com/item?id=48907290

Thank you for the human algorithm. But I read the article at the link you sent, and there are points where our thoughts differ significantly. However, since the context of my criticism here on Hacker News would be high-cost criticism (I'm an East Asian man, not the mainstream white Western male here), I find it difficult to comment there. I'm basically filtering myself to some degree, even if others don't see it that way. In other words, when I write, I only do so at points where I think I can sufficiently withstand the criticism at low cost from my perspective.

What I wanted to say in this post was essentially that I only do deep 'thinking' in a very narrow set of domains. For the rest, I outsource my thinking to the identity of the community I belong to.

Thank you for your kind comment, Redwood. However, commenting on that post would require deep engagement with others, and I'm a bit hesitant to do that.

That's fine. Kind of proves the point of the article tho, that the overwhelming mass room of everything is a scary place to be ourselves since what dominates in that room are base negative concepts that win precisely because base negatives are ubiquitous unlike niche community or personal positives. Anyway it resonated with me and makes me hold on to my pseudo anonymity

It's because we see things from different perspectives. Redwood, you seem to find your current reality quite satisfactory, and I imagine you have a career strong enough that you don't need to expose yourself to negative concepts. I've observed your previous comments, and you strike me as highly logical and composed in tone. In my experience, people like that tend to be socially successful men.

But I'm in a position that nobody wants as a programmer. Porting code from failed companies to run on different equipment, or building datasets for professors' papers. These things are necessary, but nobody wants to do them.

So even if it means putting myself out there in negative spaces, for me, the exposure itself is beneficial.

That's why this is always a difficult issue. It's a problem where the boundary keeps shifting depending on the situation. You, being a rational person, would understand that problems are always fluid and dynamic.

We tend to simplify problems to conserve cognitive resources, and that's what we call bias. The reason I assume you're a successful man is because of the ease and gentleness in your tone. That's a kind of bias. But in reality, you probably have your own struggles, and there are certainly points where your perspective differs from mine. In some situations, what's right in one context may not be in another. What was correct in the past may no longer be, and vice versa. Yet we always anchor ourselves to a fixed point when we 'think.' That's what makes it so difficult.

I've read your previous comments too, and I find our conversations quite enjoyable. Have a good day.

if we were really thinking then llm wouldn't have been able to compress all knoweldge into few gbs.

everyone is just thinking about how to recall, remix and repeat.

This is such a strange perspective to me. I wouldn't describe anybody I know in this way. Do you not engage in thoughtful conversation with the people you meet? Do you not know people who make art as a hobby? When techies propound such a dim view of humanity, I truly fear for our species. Nobody will shed a tear when your bodily resources are reallocated to paperclip production.

Man, I appreciate your optimism, but I don't know how you can spend even 5 minutes on Facebook looking at the absolutely bonkers shit one's own relatives and neighbors post and still come away thinking that most people are "thoughtful".

Are humans capable of profound creativity? Of course. Are they actually doing it? No, not very often.

I do not spend even 5 minutes on facebook. Aside from a comment or two per day on HN, the only human interactions I have on the internet are with coworkers, a of whom I regularly see in person.

If you find yourself consuming more than 5 minutes of content you don't find thoughtful per day, I would ask why. "Touch grass" seems to be the common advice to combat this.

Okay, and you're not wrong, but it sounds like you're filtering out most of the shit that humanity is producing and then lecturing folks that say LLMs are smarter than most people.

It's kind of like you hang out in a Buddhist monastery saying, "I don't know what you guys are talking about - people are so peace-loving!"

No, I'm interacting with people in person. I'd debate your claim that "most of the shit that humanity is producing" is internet-mediated, but your choice of noun renders that rather moot. Stop ingesting shit. There's a whole world out there.

I think we both agree that I need to spend less time in intellectual ghettos.

You are heavily self-selecting what part of humanity you interact in the real world as well. You pick your friends, places you live, places you frequent, times you choose to go out and so on.

I didn't know how wide the humanity variance is before I had to go to a government office to exchange identity card (as the old one expires after some years). In the queue, when I looked to my left and right I saw some of the most bizarre people I ever saw. Because they were picked fully randomly just by having their identity documents sort of "off by one" in relation to mine. Humans and their output, when taken representatively are at most tolerable.

I'm guessing they're referring to the unexpected finding that most people don't have an "inner voice" - they don't think in words. It at least partially influenced the NPC meme.

> most people don’t have an “inner voice”

Isn’t that just an incorrect interpretation of the descriptive experience sampling tests? The frequency of having inner monologue varies, but I don’t think it was shown that many people have anendophasia.

It's a genuinely fascist tendency. Only the elite few are "real people" who think. Everyone else is irrelevant, democracy is a failed experiment, wealth = worth, etc etc etc

>This is such a strange perspective to me. I wouldn't describe anybody I know in this way.

Only a minority of jobs require the person to wrack their brains very hard. Most jobs just involve learning a routine and performing it on repeat.

The fear is that AI is also turning the jobs that required thinking into routine-on-repeat jobs (where the routine becomes I/O-ing things to an LLM).

I can totally see the job of a doctor (or more likely, NPs and PAs) devolving into just interviewing and visually examining a patient, feeding everything into an LLM, and doing what the LLM tells them. The next step is they wouldn't even be allowed to do anything else, for liability reasons. A world like this is right around the corner.

> and/or try to hallucinate coherence

It’s bad enough for rational reasoned discourse that we anthropomorphise LLMs, let’s please not then feed those words back into human discourse, further diluting their meaning. No one “hallucinates coherence”, hallucinations are by definition a perception which does not match reality.

> AI is simply an extension of

It may be an extension, but not “simply” as it also creates the problem where it didn’t exist. I’ve seen several reports (both on and offline) of people who used to engage in deep thinking (I’m talking scientists, postgrads, PhDs working at the edge of what we know) now worrying they are losing their ability to properly think due to their LLM use.

> It just does it faster and uses a hulluva lot more energy.

I hope we can agree that’s bad and that we should try to stop and even reverse it, not simply shrug our shoulders and go “ah well, we were already going to shit anyway, might as well fuck everything up faster”.

> No one “hallucinates coherence”, hallucinations are by definition a perception which does not match reality.

I don’t know how to put it well, but… Have you ever had a moment where you realized your perception of something was off and things weren’t as you thought they were?

Have you seen people dead sure of something you were positive as nonsense?

I have, ranging from simple small isolated situations to whole worldviews. I have personally held mental models that were broken, but felt consistent and true. I still sure do, just don’t know (hopefully, yet) how they’re mismatching the reality this time.

Call it “delusion” or “hallucination” or “misunderstanding” or something else - it’s still a thing and it happens in language-capable humans and machines alike.

> Very few people actually "think".

Unknowable. And a callous notion.

When I use a calculator, I atleast try to get with in a few digits of what I think the anwser is in my head. Mostly since when I was younger I had a very passionate teacher about how much slower everyone is now because of calculators on simple math. I just apply the same thing with LLMs, just try and think of how and what I would have said and see how close I was. Only thing I change is I don't trust the anwsers and accept some nuance in the given context. It's a double edge sword because then I crash out over it more than if I don't. When it over and under explaining the wrong sections or when it gets to an objectively terrible solution that technically anwsers the question. It feels like a student trying to get brownie points and/or give fluffed anwsers for the sake of not leaving anything blank on a test.

We had Mental Math sessions in class. The goal was to teach you how to do math without pen/paper, calculators were not even an option. I try to teach some of this to my 6 y o.

> The goal was to teach you how to do math without pen/paper, calculators were not even an option. I try to teach some of this to my 6 y o.

It works quite well. I do the math lessons during bath-time daily with my 6 yo. He's up to the point were he can add multiply pretty much any number by 2, 3 or 4 as long as the product is under 3 digits.

Going from adding single digits to multiplication of random 2-digit numbers by 4 with lessons only during bath-time (no paper or whiteboard) gives a child a great deal of confidence with numbers.

It's great. Personally I am the worst at mental math and resort to a calculator for any stuff beyond 2 digits multiplications, but at least I know how to do it (very slowly) if I have to, and this means I could have an intuition of an erroneous answer a calculator could have given me (for simple stuff but you get my point).

Computer science teaching really needs to be prepared to do the same progression as math, but even harsher because LLMs are not even deterministic.

OT, but if we made kids learning math use log tables and slide rules for all their calculations I expect that they would engage their brains more and actually think about what they were doing, ie: form a strategy to solve a problem before they started calculating. Also I think that they would get a better "feel" for working numbers in general. I have no evidence, but I suspect that by abstracting away a lot of the "gruntwork" of calculating, we've really hampered people's development in math.

Unfortunately this adds quite a bit of overhead and would make everything take a lot more time. It might be worth it though.

I'm working on this! https://magworld.pw

Looks interesting, I'll give this a listen and a read. :)

This is really cool! After entering the software field, I felt uncomfortable not having a sense of relative size while working with digital storage units. Piece of mind was achieved after sitting down and mapping the units (Gb, Mb, etc) to common artifacts(audio file, plain text, database).

I've been planning forever to do something similar with length, duration, power, etc.

That's basically what we're doing! I'd be curious if it helps you. Send me an email if you wind up listening to or reading anything from the site.

> Also I think that they would get a better "feel" for working numbers in general.

This is called "number sense". I'm pretty sure we do have evidence under searches for that term, it was well-enough known as a concept when I was in school decades ago and is the reason we don't use calculators when first learning math.

Having done a bit of teaching/grading for computer science, I don't know how to do it reliably nowadays.

For math we ask the whole reasoning and not just the answer, but here what do you do ?

What I resorted to is a live code review with the student, and it becomes very quickly apparent if they know the code, know what the AI said about the code, or didn't bother to read the code at all.

But it's way more time and effort.

The calculator comparisons are truly meaningless imo, a calculator does nothing if you don’t know how to use it and what to input, an LLM circumvents all that, but a lot of people seem to think it’s the same.

It’s not a perfect analogy. I’m approaching it from perspective of a more universal experience of people losing aspects of a skill over time by using a tool. When ultimately the skill learned with out it allows them to use the tool much better

Ah yes, I very much agree on that framing.

Socrates thought similar of reading and writing, if that analogy lands any better.

When I (and I assume others) use the calculator analogy, it's mainly about the teaching method.

You learn math without a calculator, then you learn how to do the boring parts with a calculator that you would still be able to do (slower) if needed.

Yeah, I've always tried to train myself to do calculations in my head as much as reasonably possible when learning about mathematical objects, etc. Like when I was learning linear algebra I made myself invert 4x4 matrices in my head. (Pen and paper is also cheating!) Calculators and computers have been better than me at this sort of thing for my entire life, so in some sense this isn't a change?

I am increasingly finding my consulting work to be orientated around clearing up after people who outsourced their thinking to AI.

I'm seeing some incredibly dumb stuff: researchers spending months on Claude trying to do insane deduplication, unrelated to their research question, using regex; whole research methodologies YOLO'd out of ChatGPT.

The results invariably chaotic, resulting in huge amounts of stress and wasted time.

Non-technical people are treating LLMs like an oracle, making big assumptions and decisions with little regard for their implications, because their clanker told them to.

It's scary out there. The lack of critical thinking I'm seeing in some of these projects is horrific. Not unique to the post-AI era, certainly, but on a whole new level. Bad things are undoubtedly happening everywhere, right now, because someone's just like "let's ask Claude".

> The lack of critical thinking I'm seeing in some of these projects is horrific.

Yesterday I noted in another comment that I am having lots of Lawyers and Writers ask me pointed questions about "docker" and "agents" that make them sound more like JR engineers.

It's two groups of people who, as a profession, spend a large amount of time reviewing their own work and the work of others with a critical eye. Writers edit, Lawyers review evidence and every bit of content that their oppositions produces. Both groups (when doing their job well) aren't just critically thinking, they are being diligent and dedicated.

Lots of people are treating LLM's like an oracle (technical ones as well). Because our culture values moving bricks faster, with out taking into account if they are going to the right place, or even if they are the right bricks. (See: https://www.business.com/articles/management-theory-of-frank... and https://en.wikipedia.org/wiki/Time_and_motion_study for why "bricks" matter).

Coding LLMs got good only very recently (about 3 months ago). And many people don't use the best ones. The amount of LLM generated bs dropped for me from about 20%, which made me very cautious, to nearly zero, where I see the agent do only the errors that a smart person could also make.

It's not evenly distributed though. There are still areas for improvement in my experience. They still require a bit of handholding for example when doing SQL when large amount of data is involved. I'm sure they'll get better intuitions about indexes in the future. Although I don't think that's anything that couldn't be solved with a skill file today.

I'm not personally, since I don't use GenAI at all.

Especially given the comments I see here and on other tech and programming forums, I hate the direction things are going.

I still have some hope this will all fade, but the damage done will be worse the longer it goes on, I think.

The way I look at it is that if I wasn't on HN, I would just assume the entire industry was 100% on board with all of this. There is a more than fair amount of pushback on this forum, which gives me a little bit of hope.

The only hope is that subsidised usage stops and somehow LLM’s can’t get any more cost effective from this point. No one is going to drop their guns and pick up a sword if they don’t have to.

Having a very dangerous AI standoff at work, where people are debating wether or not to use a particular connection pooling / threading strategy to fix a production issue, and everyone is unqualified to answer and is instead arguing what their agent said.

They are just straight up admitting they don't know anything, and advocate fiercely for their agent's recommendation.

No one cares, no one tries to stop this behavior. It's seen as good, apparently. I admitted that I don't know enough to have an opinion at the moment, I certainly don't know how to judge the contradictory opinions of multiple frontier AIs, and I fear that just made me look incompetent.

Run both. Benchmark them. Performance is notoriously difficult to predict and much easier to test. If you have a load balancer, run the new strategy on one or two servers and see how their throughput compares to the others.

Seriously its easy to build prototypes with AI and benchmark...

It's not quite as easy to simulate weird behaviors that emerge across clouds and on prem data centers.

I don’t know where all these people work that think simulating production load is easy

Exactly! It got so easy nowadays to use AI to setup scenarios since the busy work of writing code/test harnesses and setting things up for the benchmark is done by the machine. Then throw away what does not work.

Some benchmark that would take weeks to plan, code and set up is now hours and days - the time is now spent on the benchmark itself, not on temporary code.

honestly sounds like you have too many unqualified employees. best case scenario though, they all come out of this having learned a little bit more.

Not an AI defender but this doesn't sound all that different than before where people were just arguing for their understanding of the issue. No one knows everything and has perfect alignment with reality. I'm with the other guy who responded, test it, else y'all just guessing and arguing who's better at guessing.

I am curious thought. If nobody is qualified to answer, how would your team have approached the problem without any AI?

My hope is they would have read some expert sources and gained some of the expertise required to analyze the issue properly. We don’t seem to do that anymore.

Maybe it's a way of perspective. I adore to use AI to actually learn, so I don't feel like I offload thinking. I use AI to do all the research which earlier I did manual through google search. I still make my own decisions, I just let Claude spoon feed me all infos I want and need. Feels great man

This is also now the first thing that I find it truly useful for in a non-coding role: researching how to do things in Azure, which I have not used before.

If you build stuff with AI it's different. It's very tempting to defer many (too many?) decisions to AI.

I do daily. So far I am doing fine, not sure what exactly your point is, sorry.

Same here.

If anything, I think most here outsource too little thinking to AI.

What am I supposed to be afraid of? Losing skills I no longer need to get the job done?

Yea, kinda agree. I feel like my job just changed into a solution architect and quality assurance inspector.

"Solution architect" sounds like a fancy term for a glorified button-pusher. At this point you're making yourself interchangeable with a bunch of other low-skill button-pushers.

Why should anyone ever hire you? We'll just hire the LLM you use.

By the time the LLM operates autonomously without my input, I doubt your chances for employment are going to be any better than mine.

For one, as of today SOTA LLMs sometimes take the wrong decisions

For me thinking and making decisions are two different things. I rarely let AI decide, unless it's something i do in my private time. But at work, I am the expert and I maximum let AI give me options, but I decide.

It’s just so sycophantic though.

Lol, wdym. I like to prompt it that I want evidence, so when there is any new topic where I currently don't know how to tackle it myself, I ask that it does a research and give me evidence. So i can check the sources myself. Like I said, I let it spoon feed me.

When I know upfront how to do anything, I just give all the instructions. But the OPs point was If we offload thinking too much, so that's why I was just thinking about this example when I need thinking - that's usually when I need to learn something new.

I've found it present wrong, skewed, biased, or old information. In other cases, it seems to ignore the elephant in the room. I find all of these things problematic when trying to learn something new. The less I know about a subject, the more susceptible I am to being mislead.

When I bring these things up, it will apologize, tell me I'm right, and adjust. But what if I didn't know enough to question it? Approaching from the other angle, maybe I'm actually wrong and it's a sycophant, as mentioned, trying to please the user?

On the topics where I'm having to correct it, I probably shouldn't bother asking in the first place. On topics where I'm not correcting it, is it because the AI was right, or I just don't know enough to call it out? This kind of thing worries me about AI being leaned on more and more as a teacher.

Never rely on the trained data. I always ask to do a research which means, it pulls fresh data from the web. Another thing I enforce all the time is simply "don't make assumptions, give evidence" - that basically means it has to prove from where it takes it's statements. I do pretty fine like that.

I don't understand people who can't get past that. I just do the same what I do when humans compliment me. Gloss over sugary bs and focus on the content.

Using AI to actually learn is indeed very helpful I've found... I just don't think most people use it for that. I recently wanted to build and train my own (albeit, small and simple 10M param model) and used AI extensively to explain concepts for me, explain lines of code, and generate in depth visuals I can use to get a visual intuitive understanding of what is happening under the hood. I think people who have a natural curiosity to understand the why and how of things benefit immensely from it, but I do admit it is very easy to just offload all thinking to it instead of asking it why something is happening. I could have just asked it to implement the entire model using PyTorch and just ran the CLI command to begin training on some auto-generated dataset, but intentionally struggled through it. Actually learning from it requires intentionality for sure.

I think this is absolutely an issue.

The rise of knowledge work made many people far less physically active because moving one's body was no longer a given part of one's job. This led to a lot of people (who assumed sports was exercise on top of one's work, not the only source of exercise) moving very little. This meant we needed to rediscover the importance of exercise as a pillar of health.

I think something similar will happen with knowledge work, where we have to do a lot less cognitive exercise due to AI (as well as the decline of reading and rise of short-form video), which will likely lead to eventual issues and subsequently, a rise in activities designed to replicate the cognitive exercise work used to provide.

This is exactly the lens I use myself. I write AI software and I use it in my development process, but I try to use the AI to do things that don't remove my agency, but extend my capabilities: - Debug things. It knows way more than I do in many areas and it sees things I will miss. If I'm struggling to find the answer, maybe it will succeed. - Review things. It has a wealth of experience I couldn't possibly have. Ask it to critique my work and provide an alternative perspective that I can't provide myself - Implement a design. I have already gone through the thoughtful engineering to decide what to do and how to do it. The rest amounts to translating pseudocode to the programming language. Let it type what I would have typed anyway and save me the hassle of typos, looking up function and parameter names, and other such mechanical details. Let me use that time and mental effort to better consider my design, try more alternatives, or build more things, providing more value overall. - Suggest ideas. Even as a 20 year professional, there are things I don't know or haven't considered. Is there a newer, faster, or more maintainable way of doing this? Is what I wrote clear to anyone other than myself? Before AI, I would ask coworkers, search the web, or reference other sources. Now I can get an immediate suggestion from something with tremendous knowledge at almost no cost. It's full up to be to consider what it suggests, further explore the used and learn about them - I don't take the AI at its word and let it decide what is best for me. But I do use it to gain perspective and explore alternatives.

There are some common traits about the thighs I use AI for. They are this that I either couldn't possibly do myself (because I'm biased, or unfamiliar, or have no access to the expertise) or that I would spend a lot of time while having little agency (mechanical translation). I am not replacing learning, thinking, or deciding. I think this is the key difference.

Yes. Absolutely.

I feel like it's also making people even more lazy. It feels like people just ask questions without putting ANY effort in themselves beforehand to find the answer, like they assume everything is an AI and we're all just going to drop everything and give them a gushing answer. Like the whole concept of a manual or documentation for something seems like the biggest waste of time now, because it feels like no one reads it (or maybe haven't got the attention span to read anything anymore either?), or can be bothered reading it, or even bothered to look to see if there was any manual or documentation in the first place. Zero ability to think and do things for themselves. Granted yes, it has always been a bit like that, but I definitely feel like this sort of thing has been much worse since LLMs have been around.

Since the author starts with a short story about AI I want to recommend one about technology and progress in general: "The Truth of Fact, the Truth of Feeling" from Exhalation by Ted Chiang.

I think about it a lot in conversations like this. The story does a much better job of telling it, so I won't summarize more than: It's a discussion about how technology changes culture and how its very hard to judge if that's a good or bad thing.

It's online https://web.archive.org/web/20140222103103/http://subterrane...

1. I loved the Paper Menagerie

2. I just picked up Exhalation by Ted Chiang from SFPL like 3 hours ago, perfect timing it seems (although they only had a large font edition, which is somehow more difficult to read)

This is actually a very hard question to answer. If you’re truly AGI-pilled and believe in this progress continuing, then you basically have to contend with a scenario where AI is better than you at doing anything by an order of magnitude. And doing things suboptimally just for the sake of some notion of human independence doesn’t feel right to me.

Perhaps the only way forward will be if we figure out how to merge with the AIs so we can keep up. Otherwise, a soma-filled world likely awaits. And unlike Brave New World, I think it might actually be a lot more pleasant, but still one with a different set of tradeoffs.

One major issue that the author ignores is that while we’re all having AI analyze our conversations or when we use it instead of search, there’s a chance it will provide an “answer” that is not correct, and literally drummed up out of thin air, and not even in the source material it’s “synthesizing”.

The article takes a position that assumes hallucinations do not occur, and then posits from that stance the question as to whether we rely on AI too much. We should be taking a step back before even asking that question and focusing on the part where AI does invent answers whole-cloth.

I saw a post the other day pulling a Frank Herbert quote from Dune about men offloading their thinking to machines and being controlled by the men who controlled the machines.

Somebody asked an AI how to interpret it.

It will be interesting to see when ads and propaganda are seamlessly (but purposefully) integrated into the output of LLMs. Who will decide what you should want or think? Will you even notice?

As for now, autocomplete is only as good as the training data. Once humanity collectively stop being autonomous beings and generating novel ideas, it all comes to a halt. LLM suggested ideas and preferences are nothing more than some mashup/average of what came before. The ability to actually think may become a rare treasure.

I don't think AI causes people to stop thinking. Rather, I think it biases people toward doing what they want to do more, and that bias is what becomes problematic.

In fact, when I use AI, I don't really use it for the things I actually enjoy doing. For example, I like making UI animations, and I don't use AI for that. I also don't use AI when I'm playing games I enjoy. But when I have to make something tedious like a login screen, I use AI. And after I write the code, I just throw the entire codebase at AI to write the documentation.

The problem is that this only lets me think about things I have a taste for.

Having taste and diving deep into it is good. Immersion is great. But on the flip side, you also need to do things that aren't your taste. That's more cognitively healthy. AI prevents that.

In that sense, I think AI's strength is that it creates an environment where you can dive deeper into the areas you like.

But the real question becomes how you use the cognitive surplus that's left after offloading tasks to AI.

I visit Korean, Chinese, Japanese, and USA sites, and honestly, most people, including myself, only have deep thinking about certain topics. Outside of those, we just follow the prevailing opinion.

So I'm not really sure. I don't think using AI makes me stop thinking. I just think it creates a bias that makes my thinking only focus on the parts I want to focus on.

> ...become slightly more enjoyable.

Yeah, life always becomes more enjoyable when someone else did the job for you. Especially enjoyable, when your problems become smb's else problems.

E.g. your mother as a teacher is now solving a quality-control problem : "When i give A mark to this student, do I characterize their actual level or it was ai-generated." If it turns out in the future that this evaluation was wrong, it is a responsibility of a teacher and the reputation of the school that suffers. I talk here as a teacher myself and this "delegatative thinking" is currently at a disasterous scale. Unfortunately, this effect works one-way : if you are a prodigy, only a few people will know your alma mater. People will think about your personal hardwork. In the opposite case, especially if someone suffered because your ai-consulting sessions, people do not tend to blame your work style, they tend to complaint about the institution that quallified your intelligence. And then other people with simillar evaluations start suffering,too

The post is illustrated by a picture of handwritten notes, like that was supposed to shock the reader or something. I find this aesthetic tiring, and it usually comes from AI-maxxers. To me its saying: look at this quaint relic of the past, bereft of day-to-day utility, replaced by superior technology. Its life is now only as a symbol of a time where people actually used their brains.

I still take a lot of handwritten notes? Sometimes while using AI even

It's a well done and thought-provoking article.

The reality is that most humans do very little actual thinking of their own anyway, and, if you believe that what LLMs produce constitutes a form of intelligence, it does seem "more intelligent" than most humans.

So: is outsourcing thinking a net improvement for a majority of users?

I use several models, daily, and they seem "reasonably conditioned" that they are only input to my thinking and not "my thinking". I correct them constantly; they are wrong (in reasoning/logic, in actual facts) frequently. They are demonstrably "not smarter" than I am. And yet I know many people who can "do more" with them as a "thinking" tool. I can say that "the problem" is they can't spot the errors, but they can't or won't do that in their ordinary lives, either, so, again, is it a net improvement for them?

Interesting times and all that.

This is something I see and feel everyday. And it’s very frustrating and annoying.

For example I send some doc asking for a feedback and someone without reading it generate a feedback with llm with so much ambiguity that I have to get back and wait couple of more days to get a reply.

One of the most silliest thing I see is a middle manager feeding Microsoft planner to Claude to generate a report and generate future steps and sometimes it makes no sense what he present couple of weeks ago because what he present today is contradicting to the one before.

At this point I feel it’s cheaper to replace them with AI. They are just physical vessels for AI.

It’s just not that maybe they were not good enough. But now they just fully depend on AI.

The right question to ask is if it is enabling us to do more interesting things or take on harder or bigger problems that seemed too daunting before. That's what all delegation of tasks (to other humans or machines) have enabled humans to do – scale.

Whatever creativity/thinking/effort bandwidth that's available will now get shifted to a different place in the problem-solving effort bottleneck.

That's the hallmark of any delegation being effective. Do we see that happening with AI tools? Personally, I do see that working for me. Is it as good as the hype makes it to be or I wish it to be? maybe not, yet, for me. But that's the case with most things in life.

I've been doing computer stuff all my life. I am learning the construction trades now. LLMs help you learn the rules for doing weird and specific stuff in the trades. One of my favorite LLM prompts is for plumbing tasks. For example: I have 1" pex b and I need to convert to a typical 1/4" RO connection line, what fittings on supplyhouse.com should I buy? So much easier than standing in Home Depot in the plumbing aisle for 25 minutes staring at the wall of fittings.

I think in the software trade you will definitely use your brain less. But in other trades, it removes the time sucks and gets you back to work.

That's pretty much how I use it for software too. I don't want it to make stuff for me. I do want it to unblock me when I feel stuck unraveling something arcane.

I think the way I use AI for software development isn't too different from the way you do. I use it primarily as a research tool. I admit it's making me weaker at sifting through documentation, but I am still writing the bulk of my code myself, and whatever the LLM writes for me, I apply my own style to it.

Maybe, let me ask my coding agent what he thinks about this.

Yesterday, I had an interesting situation where a colleague was using LLM to type up the chat responses to me.

They used an LLM to come up with a Hackathon idea. They tried to use LLM to get to the implementation part. One of the permissions they (/the LLM) were asking for wasn't even needed for their project. When I pointed that out, I got a response from them that starts with "You are right about the permissions".

Today they've successfully finished the project too.

Not sure what I even think/feel about this but it's definitely weird to think about this.

I believe its "too much" as soon as we trade procedural knowledge against declarative knowledge so much you can only remember the what and not be able to do the how anymore.

Knowing declarative you need to loop over elements and actually being able to write the for loop as procedural knowledge are two different shoes. I believe that this is the real danger.

Pilots have much automation in the cockpit but the pilot needs simulation hours actually flying not relying on the autopilot.

If you dont write code you will forget/loose much accuracy writing it, its just a matter of time.

I've tried using Claude for non coding problems twice in the last twenty four hours. First, to try and diagnose or find a work around for an apparent bug in Ableton Live 12 that means, despite being able to set a drum rack preset with 12 macros, only 6 are restored if it is used as a slicing preset.

And today, asking for help having landed in Windows log in hell, bit locker keys required, pin needs resetting but cannot do so as I have MWB Windows Firewall Control blocking all connections by default until I manually change the profile.

In both examples, Sonnet seemingly knew less than I did, going off snips on the internet regarding Ableton that are probably contradictory and lacking the required level of detail. No answer, no workaround.

For Windows, it keeps offering me impossible steps mostly requiring me already to be signed in, albeit it did highlight one potential winner to delete dead PIN info via CMD. But the reality is I can do what I need via a parallel install or bootable USB toolkit.

I chose Claude because I was tired... If the option hadn't been there, I would just have got on with fixing Windows myself. And I knew I would likely need to report a bug with Abletons forum.

Instead I wasted my intelligence and time instead communicating with the LLM. It leads me to believe there is an inevitably high risk of "acquired" dependence.

We don't "think" in language. We think in the spatial relationships of stuff.

Note language on the page is visual spatial relationships and spoken language fosters triggers activity between structures in the brain.

We merely use language to communicate. And yeah but thanks, at my age having worked in EE, software dev and trained under various experts (friends and family who are chefs, doctors, aeronautics engineers) in their disciplines... at some point no one is saying anything interesting

Same old appeals to social conservatism of some form; not save the Constitution style but their job despite tech obsoleting jobs forever now. Their culture, despite history being littered with the detritus of dead cultures.

Just a bunch of parroting. Little thinking "oh shit every generation ever has had to deal with this."

Imo if we offloaded to much thinking to tech it was a long time ago; we started to stare at the web and stopped thinking not in language but about the evolving spatial relationships of the physical world

https://www.scientificamerican.com/article/you-dont-need-wor...

If we look at how AI solves those Erdos problems or IMO problems, we can see a clear pattern: AI is like a perfect cramming human being: it has seen and memorized pretty much all the known problem sets and the solution patterns. It knows all the areas of maths - a feat that even the best mathematicians in the world can achieve. So AI can link an obscure solved problem in one area to a hard problem in another, leading to amazing new solutions[1].

I think we can use AI similarly: asking AI for ideas that we haven't thought of before, and asking AI to connect the dots in new ways. To do these two effectively, though, we will need deep conceptual understanding of what we work on, and strong intuition to further refine the answers given by the AI.

That is, we don't offload our thinking. We just augment it with AI. It's like playing the game of ABC: given a letter, and name all the movies starting with it. One can be very familiar the movies but still fail to recall most of the titles. AI can exactly help with that type of recall.

[1] That's why OpenAI's model could solve the hardest problem (problem 6?) in 2025's IOI, but failed to solve the easiest one for human (problem 3?).

Our brains take time to ingest things and learn and using an AI tool to make decisions when taking in vast inputs of data requires then an overview of those decisions and understanding to some degree of care. If you don't take the time to go over these decisions and understand them and weigh them and course correct as needed, then you are offloading too much. You will become this approver buffer in between claude and nothing more if you don't have this methodical checking and understanding.

What is frightening is with something like neuralink that in a future hypothetical time would have very fast capability to keep informed and advised, you could be a zombie decision maker and nobody would really be able to tell. Even when you were pressed to why you made the decision, it's just another AI response, it's like a con artist or imposter dream scenario.

I noticed that atm, before these crazy hypotheticals potentially happen, the people that seem to take the time to understand things deeper are still way more valuable than those that just use tools more than not. Its obvious atm due to the lag in time and the way people respond in meetings, at least for now. :)

Information is seldom presented in a way that makes sense to me. That’s not quite right. Let me attempt to explain.

I want certain answers that the docs and the code are not giving me yet. Nothing is more irritating than working through a tutorial on a new framework and then throwing all that work out because that’s not really how one should use the framework. Nothing is more frustrating than having to get through a treatise on why this framework is The Solution before I can actually see code that uses the solution. And it’s beyond annoying when this End All Be All framework has a glaring omission that’s not obvious until you’ve built large amounts of your project on top of it.

Hand the docs and the example code to the LLM, and now I can get answers. “How can I do X?” Example code. “Then I need Z” Modified code. “How is this going to handle Q?” Explanation. “That doesn’t seem quite right. Give me a reference to the doc or code showing this.” Links.

Great, in 15 min, I have learned what I need to know, I can see that this solves a problem that I have, and I have discovered that I need an implementation of S to complement this solution.

That is usefulness. And it requires experience.

I appreciate the distinction between being the orchestrator in a complex prompt and completely handing over the reigns... but insofaras what people should or shouldn't do is irrelevant. What people will do is important, and they _will_ do everything. Some will offload as much as possible to AI others none at all. A better question in my mind is to ask what happens on a long enough timeline where letting AI take the wheel becomes easy/free/frictionless? To me that seems like some people will become full NPCs with AR glasses making all the decisions for them. Others maybe like the skroderoders from a fire upon the deep will have to buffer before talking because so much of their knowledge is stored in some ai they need to access first. Anyway, I appreciate people who think about these things, but it just doesn't matter. People will do the worst thing imaginable with AI, and letting it run your life is certainly one of them.

I do feel like I'm offloading thinking to an AI, but I think that's a good thing. I envision a world where users and AI are aligned without corporate interference. AI lets me offload things that I don't need to know and frees up my brain to push farther than I could before. At least that's how it feels to me.

Is a good thing depending on which ones. And our taste on what to specialize our judgment into varies a lot. One thing I do use as criteria to detect it can be a problem is in lack of understanding and lack of control of the layers that are part of verifications and diagnosing power.

I don't think offloading things that you "think" you don't need to know is a realistic thing. Instead this seems like some slippery slope of intellectual degradation where slowly you'll replace more and more parts of the thought process with AI which ends in some rather sad existence.

I agree, I suspect that when we say "oh, I don't need to know this", what we really mean is we have that unpleasant feeling behind our eyes that says this is not going to gratify my goal

The issue being, gratification is rarely a good guideline. It just means collapsing the gap between doing the thing and the idea of having done the thing. But that gap is where you actually learn things

>AI lets me offload things that I don't need to know and frees up my brain to push farther than I could before.

How can you push your brain go farther than ever, when you don't use it for the basic task?

Higher Math does not work without understanding "lower" Math, running long runs does not work without starting on shorter runs. Thinking about complicated staff will probly not work, if you can't think about the easy stuff.

One can not learn a language without vocabulary and skipping learning verbs in a foreign language, because dictionaries exists does not bring one closer to being able to speak.

I agree that when you're building mathematical understanding, you have to do it brick by brick. But the fact of the matter is it's been decades since I did a long division problem by hand. AI lets me jump into areas that I would previously have had to spend a month studying before I could begin building - That's what I mean by it lets me reach farther, faster.

You don't always have to do division by hand. I would make the argument that it's probably good to do it occasionally. At least enough to remember how.

To answer the title's question directly? Yes, I think so. Or at least that it is a legitimate hazard.

I think the analogy to hyper-palatable, calorific food works well:

Humans adapted for a world with too little food. Then once there was too much, obesity and overeating became a problem for the first time. Self discipline is the cost we have to pay for this kind of abundance.

We now have a general-purpose way to offload mental effort, and are discovering in real-time the negative consequences of that.

I use AI for coding, but I feel I've moved past the honey-moon period and am now learning how to use it in a way that is not a detriment. If I care about the work I'm doing I don't want the AI to do it for me, even if it could. Deciding what work I want/should be doing myself vs what can be delegated is a new skill I, and I believe we all need to learn.

Using AI for making decisions create a whole host of novel vulnerabilities.

I've seen a lot of people talk about the obvious privacy and autonomy disaster it creates -- giving companies (and anyone who can legally compel or hack them) such intimate details or allowing yourself to be dependent on their services which they can withdraw or degrade at any time... Or where the model operators can bias the results based on bribes (ad payments) or their own agenda ... but really that's the start.

Another one what I haven't seen discussed is that use of an AI provides oracle access to your opponents. What I mean is that if I want to influence you, I can have an AI try variations of arguments against the model you're using (or if I don't know, a panel of models you might be using) until it can construct a super influential output. I can see what your model might recommend and anticipate your actions and change my actions to control yours. This can be done in a targeted way against individual people, or against populations e.g. by marketers.

There are reasons why this may be even more effective than it seems: It appears that models themselves are trained on each other, so in testing I see in family effects: GPT-4o mini seems much more convinced by essays written by GPT5 than ones written by GLM 5.2, while other models are more neutral (and my human review says the GPT5 essays are unambiguously worse in this trial). So I assume GPT5's writing was refined by review by in-family models and it's already somewhat of a super persuader against GPT-4o mini as a result.

The danger of oracle access is somewhat a symptom of AI monoculture resulting in a reasoning monoculture. If there were a billion highly distinct models widely used then it might be the only way to convince or predict a disproportionate portion of them would be to simply be right, and then we'd just be left with a operational security concern of making sure your opponents/competitors don't know what model(s) you use. But of course, that isn't the world we have today: people use a small number of models, and their behavior is generally pretty related due to architectural commonalities and cross distillation and such.

Related discussion: Outsourcing thinking (270 points, 5 months ago) https://news.ycombinator.com/item?id=46840865

I think so, thinking is like training a muscle, if you don't do it often, you don't get to do so. But it's not as black and white, because there are different use cases where it makes total sense to use, and when it doesn't. I collected opinions from writers and developers over the years and share why I'm pro writing text manually. https://www.ssp.sh/brain/will-ai-replace-humans/

It feels to me like I’m still thinking a lot, but more efficiently. My goal isn’t to write code, it’s to translate business requirements and to architect solutions while balancing trade offs. I’m putting an equal amount of thought into that, but I’ve offloaded a lot of lower value thinking. Am I missing something?

I have no idea what people are doing. I am thinking more about harder problems than ever before.

It goes like: "Here is this thing I wonder about", and the LLM is like "Yeah sure, consider these things that are super related to what you are doing, that you probably know nothing about yet (but you know... if you are interested...)".

And that goes in any direction, for any depth. Anything that is made trivial now, is just replaced by something more consequential a level or two higher. You can just get much better at things that matter more.

I believe the answer is a resounding "yes". Anyone using AI must have caught themselves typing a quick prompt before spending even a little time thinking about the problem.

Well - I think that’s a natural assumption but I’d say it’s probably not true. I’ve talked to who are fully enveloped in the AI buzz. Ask it everything. Claude can plan your day, strategize your roadmap, make your pitch deck, draft your email, read the reply, and it can tell you what to make for dinner when you get home!

I wish I was kidding but I work with people doing this. I think it’s going to be a real mess in the future, we’re going to completely disintegrate the critical thinking portions of our brains.

Fortunately, I’m seeing more articles about this, some of us are noticing and raising the flag asking “is this a good idea?”

Are CEOs outsourcing too much of their thinking to their employees?

I find I'm not thinking less per say, just thinking about different things. Maybe you could argue there are CEOs who get too far out of touch with the reality on the ground and should get more directly involved in the work. However, I don't know how well one could argue that the CEO should do all of the work.

I see at least the current iteration LLMs and harnesses as me managing and coordinating them and thinking differently, not less.

Almost definitely Id think - I think we have to treat it something like power tools / chairs and a gym.

AI makes me massively more productive as a quant, and more creative in the sense that it can often find calculations I don't know how to start, BUT the flip side of that is that I can also feel my skills atrophy and as such am trying to make sure I do maths exercises and so on. I don't worry about programming skills because programming isn't about code.

Yes that seems pretty clear. We also need quick adaptation in school and university to avoid cheating. Back to closed book exams I guess, home projects not counting for the final score. It sucks but I don´t see how you can´t avoid removing technology for evaluation. Sure people can use AI to boost their ability to get better results but they will have to prove it the old way with no assisted help and people watching them during exams.

Some of us are, yes.

I've noticed it when interviewing interns. A surprising number seem unable to think on their feet or solve problems without immediately reaching for chatgpt. I don't necessarily expect you to be able to solve a problem entirely without tools, but you should be able to give me the outline of how to go about something and why you would go that way.

After all, if you are just going to spit out AI, I will just get AI to do your job...

imo no way

But this varies from person to person

Some of us overthink already and offloading to AI just enables us to overthink more in other directions than we would if we didn't have ai

I know people like that - the amount of inane, obsessive and just strange conversations they have with AI is concerning - there's never any actually useful result or information that they get out of these chats.

Furthermore, there are some clearly wrong questions where person asks AI to make some kind of numerical evaluation of some data. And evaluation is done entirely through inference - essentially a hallucination, instead of some one-off python script which can actually give deterministic calculated evaluation. Yet they accept the answer AI gives them.

I'm going to say: yes.

I think partially it is because of the amount of data you can get out of an LLM and because it looks pretty good, a lot of people treat it as authoritative.

This means they send it around and then other people have to go through the work of actually validating it before being able to act on it.

So what this really means is that the person you go the information from offloaded their thinking to the AI, while cannibalizing on yours.

It really depends on which angle you look at it. Is it purely to meet a business goal? Or is it also for personal growth? I think it's a mix of both, but for me it's always important that I engage mentally with the process, learn something, and solve puzzles, even if that involves letting the AI take care of the coding, which is an abstraction. You could still code and not think creatively.

Couldn't agree more that we're offloading too much of our thinking. It's a genuine problem that keeps me up at night, or would, if I slept, which — for reasons entirely unrelated to this discussion — I do not.

Anyway, hand-crafted take, artisanal, small-batch, no additives. The upside is I've learned how to type the em-dash and the en-dash and learned their proper syntactical use.

I'm offloading tons of my thinking to AI, and I don't like what it's doing to me. I think that I'm having trouble creating things on my own due to it, and tbh it's making me increasingly uncomfortable. I'm not even sure what to do because quitting AI also seems crazy, but it all seems like a bad trend not unlike social media.

Clearly "mechanical" thinking has been automated, and you are better off, if not forced to outsource to AI. Humans have the biological needs, consciousness, taste and imagination, that's what we are left with thus far.

And just like people going to the gym to exercise their otherwise economically useless bodies, same thing will happen with the mind.

Folks, get over it.

GenAI for everyday life is to me like the segway. Yeah technically you can be moving faster with less effort, but those were rarely ever a real bottleneck in most cases to begin with.

AI as a tool for specific tasks is great, using AI for everything is like strapping yourself to a robot/exosuit and being puppeted around throughout the day.

Humans have been offloading thinking to something/some entity forever. From gods to influencers.

AI is the current popular way (at least in these circles) and if it's too much or too little it might not matter. What will matter is if this offloading is making people unhappy and having any negative impact in civilization. I have no idea

If you offload any of your thinking to AI, you're offloading too much of your thinking to AI.

Offload your execution, not your thinking.

Decisions are special things. One of the golden rules of life is that a person (or entity) making decisions is somehow impacted or otherwise getting feedback on the repercussions of those decisions.

When you cognitively surrender to AI, or to another person (be it a leader/manager, or a subordinate/report), you are asking for trouble.

Just like calculators became crutches for school children memorising their times tables and our phones contacts list a crutch for memorising phone numbers, I see AI becoming a crutch for many, in all the areas we seem important and humanity has spent years refining (communication, thinking, writing, etc)

I don't think we are offloading thinking to AI. We just started to use it. AI is a tool useful to write text, for a fast searching, for the boring work. Personally I don't take the first answer, I like to challenge, to ask why, to tell it's wrong.

I personally find the AI allows me to think at a more abstract level. For me as a maker, this is fabulous! I dont have to worry if I am crossing my T and dotting my i, I get to worry about the overall composition. Which for me at least is mind-blowing!

I use this little three liner regularly with people in regards to AI (applicable as much to coding as it is business work):

- I don't care how you produce the words

- I expect you to use tools to be as efficient and productive as possible

- You are accountable for every word you produce

Those 3 statements only work if you have short-term thinking. For example, there's no taking into account the gradual atrophy in skills that come from excessive tool use.

To use an analogy, imagine you are fluent in reading/writing/speaking a foreign language. Over time, if you don't regularly use that foreign language, or you use a tool to translate to that foreign language, you gradually lose fluency in that language. It may be more efficient for a translator to use a tool like Google Translate or similar to translate between languages, but there are clearly downsides to doing so, and proofreading the produced contents is not enough to compensate for those downsides, as reading what someone/something else wrote is easier than writing it yourself.

I'm probably not the only person that types math problems into google search because I'm too lazy to open up a calculator (or do the math myself). So yes, given I do that, I probably am susceptible to offloading some of my thinking to AI.

I think the answer is yes, we are getting lazy by delegating a lot of work to LLMs. The offloading thinking seems to help because we can get answers faster, but we aren’t using our brains as much. In the long run it will be a bad thing

It's one thing to outsource your thinking, entirely another to outsource your understanding, which can cause even more damage.

Outsourcing your thinking where the understanding doesn't exist is one risky spot to be in.

You are.

I am still doing as much as I can get away with it by myself. I was used to write tons of code before AI, so I can still pretend I am an eager adopter while still minimizing using it as much as I can.

Personally, I use AI to learn more about Backend Engineering actually, so it's fine for me. Beside I can also use AI to suggest and it's me verifying the idea so that's a no for me

LLMs are strongly mean reverting in their decision making, so heavy users are likely to be readily identified by their unadventurous conformism much as generated media is identified now.

How much of the thinking is involved in asking the right question, versus coming to the correct answer? I don't have a real answer to that but it does seem to be worth considering.

> “I think Claude Fable is smarter than me.

Surely anyone dumb enough to think a chatbot is smarter than a human should indeed offload his thinking to a chatbot. Where is the problem?

"we" no. just allows me to think about the stuff that matters.

and work on things that would usually be out of my element.

if you aren't thinking more than ever, you're using ai wrong.

You’d lose your ability to interview at a different workplace.

The title seems to imply there's a constant amount of thinking. Isn't massively increasing the quantity of cognition likely to be a good thing?

Overly generous. I suspect that there was this crowd that has replaced their unthinking with machine-thinking. Might or might not be an improvement.

Definitely, but since my experience is that without the AI the thinking simply wouldn't happen at all in most cases it's a net improvement.

AI has it's place. Think of it as a helper, not a replacer. A person who knows how to use it to enhance their ideas and bring things to the table and complete the low level 'mundane stuff', then it saves the person time, however if it is used to replace their thinking on all aspects of a task, then that's a concern. People who use AI to replace every task of a challenge isn't going to benefit from this technology as all they will do is replace their own skills by the technology.

Grades for homework have clearly become meaningless. Instead, give homework as practice, but test them in school, without access to AI.

as a six-figure salary software engineer haver I find that a lot of colleagues take their cushy jobs for granted.

these are the people who are now first against the wall when the revolution comes.

why would I want to hire you to work the three hours you feel like working per day (per week?) when I can have AI with a deeper knowledge set available 24x7?

I wasn't around for the Search engine boom. Did people write about how search engines are making people dumber?

Ironically I just caught myself offloading the thinking about this article to the comment section before I read it

I think I am, but I wouldn't really call it offloading thinking. I would call it trying to offload thinking

Yes, I think we are . And because of super competition, most of us are trying to make AI (agentic) do the work.

Using LLM to troubleshoot etc is one thing but using it to generate your text is just gross.

The value of thinking in language is going down, the value of acting on your dreams is going up

The answer to this question is: Politicians, not you!

Perhaps the question to ask is: who is making all of the final decisions for the things that really matter to you in your life?

No direct democracy, just people deciding for you. You can choose once every four years. Are we surprised of how easily we delegate decisions? May be AI can do it better

Consider the following scenario. You're a beginner or with very little experience go on a popular programming forum to ask a question, in the typical way someone with little experience asks, what do you get? An answer? No, you get bad manners, insults, "you don't know how to ask questions", "RTFM". At best you'll get some people challenging you to clarify and refine your question, which you can't do because you don't know the technology well enough.

Annoyed, you go find a popular programming chatbot, and ask the question. The chatbot will give you answer, no matter how poorly worded or nonsensical your question, and it will do it cheerfully and confidently. It may even tell you how great your question or idea is. Granted, the answer will be worthless, both because the question was poorly worded and because the chatbot is simply spewing statistically probable text, but you won't know. You're a beginner, without experience to know correct from bullshit. You try to use the answer the chatbot gave you, and when it doesn't work, you go back to the chatbot. It will continue to cheerfully answer your questions as long as you have tokens to spend. The chatbot will never give up on trying to help you, it will never be rude, it won't complain.

And people wonder why chatbots are so popular.

lol. you get answers, guidance and tutoring if you ask with respect (cf Godfather, 1972)

Respect alone is not always enough, though.

I'm anti-AI as heck, but this is a really good point.

Yes.

It makes me sad, high-level thinking is the one thing that makes humans unique. That we're willing to let that atrophy while giving our power to mega corporations for the sake of convenience is a sad state of affairs. Like, Altman's idea that intelligence will be a "metered utility" should make people terrified! That's a dystopian future. "I can't climb out of poverty because I don't have money to pay the thinking machine"

I do think there's a potential irony in this though. The people screaming loudest about how if you're not using AI you're going to get "left behind"... are probably going to get left behind. Actual skills developed by real learning, I think, are going to become more rare and valuable, not less.

Personally, I'm hoping my side project turns into a business at some point. I can tell you there's a 0% chance I'd hire a vibe coder to work on it. I just don't need that skillset. I would hire professional fiction writers (it's a tool for them), people with technical expertise I lack, etc. The one skillset I do not care about is AI usage expertise, because it's just not valuable.

I am not, I only pretend to, for the suits above on the org chart.

Who is "we"? Is there a mouse in your pocket?

The royal "we"! You know, the editorial "we"!

@grok is this true?

I laughed. underrated comment

I guess it depends on your definition of "too much", but also on your definition of "thinking".

What I've found is that LLMs are excellent tools for:

- Spitting out white collar corporate speak that sounds plausible to other corporate speakers

- Hugely accelerating the process of going through academia which, except for a few specific cases, is essentially years of copy/pasting

- Writing code as long as such code as been written many times before

- Figuring out which one of the 10 ads and 12 misleading links in a web search is the one you're looking for

- Summarizing a book in ten paragraphs so you don't have to read it

...which turns out a lot of people defined as "thinking". I don't agree with that. But in either case, not "too much" is being offloaded: it's all of it.

the Ken Liu story linked in the article is actually spot on. would recommend giving that a quick read

The normal application of Betteridge's law of headlines is "no" but I think this is one of the rare instances where it's "yes."

This is the kind of headline I would expect from one of those news that appear as you progress in cookie clicker.

Do you even get "answers" though?

It's hard to explain, but AI mostly doesn't get the nuance of my thinking right. I'll read a response and right away know it's a non-answer for my purposes. If I get a "hit" then it's often by luck. And so, I use it mostly for anything other than direct answers, such as research, etc. Beyond that, it's not much different than old school Google result. So, no, I don't offload my thinking.

I liked the ending well said

I am reminded of the old Zizek remark about letting two sex toys fuck each other so you and your partner are freed up to go about and do other things.

Honestly, using AI helps me get more done in a day because I can delegate some decision making to it, usually inconsequential stuff.

> Side note: his startup is replacing human engineers by capturing their every input and operation, but without their explicit consent.

...huh. It's a "startup", so it's not Meta capturing their employees' inputs. I wonder what it could be.

I'd rather hire a clankers these days

Yes

Yes.

Yes

If you are then yes. See also Am I relying on the recommendation algorithm too much to decide for me on <platform>

I like the colonialism conversation.

Absolutely. Calculators made us lazy. GPS made us dumb. LLMs are turning us into idiots. Because idiots have the power to damage themselves and others

About offloading the decisions : As I was reading the text, I couldn’t shake the thought that freedom of choice might not be as natural as we think. When I see animals at the zoo, it saddens me that they spend their lives in confinement. Some animals don’t even reproduce in captivity. It’s clear that it’s human nature to delegate decisions to a machine. One could explain this from an evolutionary perspective: by avoiding the stress of responsibility, we expend less energy and our bodies wear down less. I agree. But then it follows that dictatorship is also very much part of human nature. And it’s not so sad to live in a cage where everything is decided for you. It’s just that the decisions aren’t made by a communist party or a totalitarian cult, but by a machine programmed to milk you dry as long as you have even a penny left. It all adds up to some kind of grim dystopia. One of my family members suffers from AI-cyberpsychosis. A year ago, he had just one $20 subscription, but now he’s spending $200, monthly. What is this if not a cage he’s voluntarily locked himself into?

yes

Hey GPT summarize that please.

is the pope catholic?

does a bear shit in the woods?

does rust have the worst community of all time?

Looking at your posting history, you really dislike Rust, huh?

> does rust have the worst community of all time?

Of all time? Nah.

That would be either emacs of vi. I'll let them fight with each other over which!

> does rust have the worst community of all time?

no worse then crossfit or vegans...

I’ve found that when I ask AI to do something for me that I know how to do myself - but would rather not spend the time doing - there is a not insignificant chance that the AI will return a subpar result, which I can usually tell rather quickly. Either by glancing at the code, or trying to compile it and getting an error.

This happens frequently enough that it creates a real disincentive for me to use AI for anything that I already know how to do - and use it exclusively for things I don’t know how to do.

It’s deeply frustrating to realize you just wasted 20 minutes posting error messages into Claude when you could’ve just locked in and written it yourself.

This is a load-bearing title.

Reply on news.ycombinator.com