Thoughts on thinking
dcurt.is
dcurt.is
The process of forming expressions just is the process of conceptual and rational articulation (as per Brandom). Those who misunderstand this -- believing that concepts are ready made, then encoded and decoded from permutations of tokens, or, worse, who have no room to think of reasoning or conceptualization at all -- they will be automated away.
I don't mean that their jobs will be automated: I mean that they will cede sapience and resign to becoming robotic. A robot is just a "person whose work or activities are entirely mechanical" (https://www.etymonline.com/search?q=robot).
I'm afraid far too many are captive to the ideology of productionism (which is just a corollary of consumerism). Creative activity is not about content production. The aim of our creation is communication and mutual-transformation. Generation of digital artifacts may be useful for these purposes, but most uses seem to assume content production is the point, and that is a dark, sad, dead end.
Try to think of an object that doesn't exist, and isn't based on anything you've ever seen before, a completely new object with no basis in our reality. It's impossible.
Writers made elves by adding pointy ears to a human. That's it.
Humans have been interested in supernatural beings for thousands of years. Their appearance is usually less important than their powers and abilities.
The word is present in Old English and Old Norse, and elves appear in Norse mythology.
Maybe it's like that because there aren't many novel opportunities for varied experiences nowadays.
The pointy ear sounds trivial in our experience, but it is radically different than ordinary everyday thought when observed as a piece of a whole imagined new world.
Of course, pointy ears now are not a novelty anymore. But that's beyond the point. By the time they were conceived, human experience was already homogenized.
The idea space for what an object is has been depleted by exploration. People already tried everything. It's kinda the same thing as saying that is impossible to come up with a new platonic solid (also an idea space that has been exhausted).
Any novel thought is bound to be nameless at first, and it becomes novel by trying to use derivation to define an unknown observation, not as a basis for it.
Now that’s reductionist to the point of being diminutive.
We should try to be the bigger person.
That’s really the long and short of it.
― Terry Pratchett, Lords and Ladies
All creativity is a conversation between our own ideas and what already exists.
Consider the unused soundtrack to James Cameron's Avatar [0][1], where ethnomusicologists set out to create a kind of music that had never been heard before.
They succeeded. But it was ultimately scrapped for the film because - by virtue of it being so different to any music anyone has ever heard before - it was not remotely accessible to audiences and the movie suffered as a result.
To argue that work is not creative because it is still based on "music" is absurd.
[0] https://www.youtube.com/watch?v=tL5sX8VmvB8
[1] https://ethnomusicologyreview.ucla.edu/journal/volume/17/pie...
Human creativity is for the most part derivative in some way, that is the nature of it and my point. I was contrasting it to illustrate why I think the idea that machines cannot be creative is wrong. Especially given that current models will only get more complex and nuanced in the future.
I never said that humans cannot be creative because creativity is derivative at any point. We are engaging in another human pastime; tribalism.
I would add a couple of things to that. First, humans (like other animals) have instincts and feelings; even newborns can exhibit varying personality traits as well as fears and desires. It's certainly useful to fear things like spiders, snakes, or abandonment without prior experience.
Second, an important part of experience is inner life - how you personally perceive, feel, and experience things. This may be very different from person to person.
(I do not participate in culture wars, this fact just straight up fascinates me as a non-masculine gay guy.)
"No sir! I did not see you playing with your dolls again."
And I think there's the possibility that those signals are there early on in our brains. If not race cars, then it was swords and bows that were lying around and were appealing. We are products of evolution.
But as to OP's insistence that behaviors innate to our brains as not being derivative. I think in order to qualify, it must be a conscious behaviour. And even so, innate behaviours are derivative in that they came in response directly to our environs. Were they not, those traits would not have been selected as a response to environmental pressure.
Are you genuinely arguing that LLM output is derivative, and human output is derivative, therefore they're equal? Why don't you pop that thesis into ChatGPT and see how it answers.
I didn't realise that by comparing derivative human output to derivative machine output that I was inherently arguing that they are equally derivative at this point in time.
It is true that human output has higher entropy, but that will not be the case forever.
This is an outrageous thought experiment. Novelty is creating new connections or perceiving things in new ways, you can't just say "try to have eureka moment, see! impossible". You can't prompt engineer your own brain.
In fact, there's some research about eureka moments rewiring our brain. https://neurosciencenews.com/insight-memory-neuroscience-289...
Creativity is much more than the derivative production of artifacts. What the OP is driving at is that creativity is a process of human connection and communication—you can see this most clearly in the art of interpretation. A single literary work has an almost uncountable number of possible interpretations, and a huge element of its existence in the world as a price of art are the discussions and debates that emerge over those interpretations, and how they shape us as individuals, instill morals, etc etc. Quite a lot more than "making elves by adding pointy ears to humans".
Your post stinks of the very gross consumerist mindset the OP called out. The creation and preservation of meaning is about way more than the production of fungible decontextualized objects--it's all about the mediation and maintenance of human relationships through artifacts. The fact that the elves have pointy ears doesn't even begin to scratch at their actual meaning (e.g. they exist in a world with very big real problems that effect you and me too, e.g. race relations, and exaggerated features estrange these relations so as to make them more discernible to us and allow us to finally see the water we swim in).
If humans stop engaging with these processes, it's reasonable to believe that a lot of that semiotic richness, which is much of what, in my opinion, makes us human and not just super smart animals in the first place, will be lost.
Throughout history man has been celebrated and distinguished as the rational animal. As master of the earth this animal in our days dedicates its brightest minds to the continual increase of economic growth. Ask the rational man what is growth good for and after a few exchanges they perhaps will say that it ultimately improves our quality of life and even extends it. If might even allow the human race to flourish beyond earth and thus prevail long after resources on earth are depleted. But ask him then why is improving the quality of life a good thing at all? Is it just a meaningless cycle in which we improve the quality of life so that we can then improve the quality of life even further? No. Ask an individual human (in contrast to the ultra rationalist who thinks they represent the human race as a whole) what they work for, what they strive to achieve, what does quality of life ultimately mean to them and you will end up with happy times spent among family and friends. With meaningful moments listening to music, watching a film, reading a book. About time spent in creative endeavors that are totally their own. The rational animal in its hubris forgot what it thinks for and trapped itself in an endless cycle where the true meaning of being human is hidden from the sight of many.
But I think a wake up call is due very soon. The rational animal is about to discover the rationality it prides itself on was merely a sample of the true possibility. From the rational animal we have been relegated to another animal with some rational capability. As we slowly realize how futile are our attempts at thinking, we'll realize to our horror that the gift we are left with is the ability to recognize the futility and inadequacy of our attempts. Hopefully then we'll decide to retreat back into what truly makes us human, to what is ours, to what quality of life really means.
I never said that creative thought mostly being derivative means it's not creative, or that it's a bad thing.
I merely made a point as we see a machine, pointing and laughing at it, that we shouldn't throw stones when we live in glass houses.
But the reaction is certainly interesting. I wonder how many of you believe that humans are magical and that it's impossible to create a sentient machine, even given millenia, or infinite time. I believe it's possible. I don't believe that we're any more magic than a plant, there are many species with much more to their DNA than we have.
Pick up an Iain M. Banks book, my friend.
Someone imagined space and time could be a deformed fabric. That was new.
In minor and major ways, new ideas are found or emerge from searches for solutions to problems from science to art. Or exploration of things in new combinations or from a previously untapped viewpoint.
Most people are not looking hard for anything beyond what they know. So not likely to find anything new.
But many people try new things, or try to improve or vary something in a direction that is not easy, and learning something nonobvious and new is the “price” they must pay to succeed. Or a bonus they are paid for pushing through a thicket, even if they don’t succeed at what they set out to do.
What is the point in saying there is nothing new,? So the word “new” is like “utopia”, with only a mythical aspirational meaning?
History has never stopped proving how inane such an utterly unsupported opinion is.
Just keep up with quantum physics. Constantly finding new strange counterintuitive effects that require intense research and explorations of combinations of ideas and imagination to envision. And then challenging invention of new highly artificially controlled conditions, not found in nature, to create.
Mathematics as a profession is all about discovery of new things (from exploration) and invention (from imaginative questions and quests).
Art produces new things all the time, but you have to have familiarity with art to see what is new. If all you see are materials and color you are not literate enough to read it. Anymore than a unique story is “just words”.
For that matter, in slower timescales, nature has never stopped creating new creatures with new attributes or behaviors.
Perhaps have you set your own sights much too low. Try doing something nobody has been able to do before, and sticking with it. You will discover something if you really do stick with it. Even understanding the barriers better than before reveals things, and forces new concepts into being.
Quick, what's 51 plus 92?
Now: Did you think back to a time someone else added these numbers together, or are you doing it yourself, right now, in your head? I'm sure it's not the first time these numbers have ever been summed, but that doesn't matter. You're doing it now, independently.
Just because something isn't unique, doesn't make it derivative. We rediscover things every day.
Even emotion, sure you're happy that you ate a tasty food. But you're happy because your body is flooding you with chemicals to reward you for it, as with many humans over time, it's how it was selected.
I'm not saying it's a bad thing. It's just incredibly interesting to me to compare the way I think we act to the way these early models are acting. That's all.
This just feels evasive to me.
This is the argument I use to dunk on ranters who spam conversations with “How can you say Christopher Columbus discovered the new world when there were already people living there?”
The specific argument “but it's not a discovery because it was already inhabited” is a particularly literal-minded child applying their teacher’s prohibitions on plagiarism to the real world.
I'm directly replying to the notion that something is derivative if it is "based on something we've experienced or seen." The fact that rote math calculations are generally less original than creative expressions makes my argument stronger, not weaker.
Your argument is moot anyway, how exactly is adding numbers together proving that such a thing isn't derivative?
I add numbers together, because I got taught to in school, because my teachers got taught to, because humans learnt to a very long time ago, because they observed a collection of things of different numbers. Perhaps the first human to count enjoyed an original thought. Or any who did independently, communication is the origin of derivation. And us humans are alllll about communicating.
Adding numbers is not an original thought, not in this context.
When I write a poem in a birthday card for my wife to give her on her birthday, very little of the "meaning" that will be communicated to (and more importantly with) her is really from some generic semantic interpretation of the tokens. Instead, almost all of the meaning will come from it being an actual personal expression in a shared social context.
If I didn't grasp that second part, I might actually think that asking ChatGPT to write the poem and then copying it in my handwriting to give to her is about the same thing as if the same tokens written but from genuine personal creation. Over prolonged interaction, it could lead to a shared social context in which she generally treats certain things I say as little different than if ChatGPT returned them as output. Thus the shared social context and relationship is then degenerated and fairly inhuman (or "robotic" as the above post calls it).
One day one of them discovers AI and post anything made with AI - initially it’s great, it’s much better quality than what they could photoshop. Everyone jumps on board.
But after a day or so, the joke is over. The love has gone. The whole things falls apart and no-one posts anything anymore.
It turns out - as you say - that the meaning - founded on the insight and EFFORT to create it - was more important than the anccuracy and speed.
When WhatsApp originally inserted their AI bot in the chats, it got very annoying very quickly and we agreed to all never invoke it again. It's just a generative spam machine without the curation.
You own cars, do you weep for the horse industry? Stables, farriers, blacksmiths, leatherworkers/saddlers, ostlers, grooms, etc.
What is an art now becomes a functional ubiquity later. Everything took "insight and effort" because everything was handmade, it's an innate feature. And now those things are mass manufactured and it's what the people want. Humanity will move on to calling whatever current thing we're manually doing an art.
Horses weren’t art. They were a way to get somewhere.
The whole problem with AI is it’s trying to automate the things people LIKE to do and take their life meaning from instead of the things that no one likes doing - like pulling a carriage or a plough.
Yes, elves are derivative, as was a lot of the Tolkien world in a way - being intentionally based on ww1 - but its intention was to create something beautiful and amazing and communicative and transformational.
That's easy. The hard part is to explain it to other people, because we lack a shared background and terminology to explain it.
I was thinking of the first solar civilization, which lives totally in space. Near a star, but not in a planet, and no gravitational pull anywhere. They build tubes 10 km long, a shot board is put at one end, and the players at the other end. They shoot darts at the board, and each shot takes 5 hours to reach the target. That's their national sport.
Problem is, I have never played darts, i don't know anyone who plays it, I will ask the LLM to fill in the blanks, of how a story based on that game could be constructed. Then I will add my own story on top of that, I will fix anything that doesn't fit in, add some stuff, remove some other stuff and so on.
For me it saves time, instead of asking people about something, hearing them talk about it or watching them do it, i do data mining on words. Maybe more shallow than experiencing it myself or asking people who know about it first hand, but the time it takes to get some information good enough collapses down to 5 minutes.
Depends on how you use it, it can enhance human capabilities, or indeed, mute them.
https://chatgpt.com/canvas/shared/6827fcdd3ec88191ab6a2f3297...
I don't want to read this story. I probably want to read one that a human author laboured over.
To cut it short, in the end what Borges proposed is that the meaning comes from the stories, and that all the stories are really repetitions and permutations of the same set of humans stories (the Order) and that is what makes meaning.
So all a successful literary AI needs to do is figure out how to retell the same stories we have been telling but in a different context that is resonant today.
Simple right ?
Imagine a chef, congenitally unable to taste or smell food, who has nevertheless studied a million recipes. Can they reproduce existing recipes? Sure, if they follow the instructions perfectly. Can they improvise original recipes? I doubt it. Judging by the instructions alone, the recipes they invent may be indistinguishable from real recipes, but this chef can never actually try their food to see if it tastes good. The only safe flavour combinations are the ones they reuse. This is a chef who cannot create.
LLMs are structurally banal. The only plausible route to a machine which can competently produce original art requires the development of a machine which can accurately model human's aesthetic sensibilities—something which humans themselves cannot do and have no need for, since we already have those aesthetic sensibilities built in.
This is the fundamental error of using an LLM as a ghostwriter. Humans don't only bring inspiration to the table—they also bring the aesthetic judgement which shapes the final product. Sentences written by an LLM are banal sentences, no matter how you prompt it.
write chapter 1 for a new Novel in Progress, take inspiration from the example Novel but DO NOT Repeat Example. Add vivid imagery, in a dark comedy style. dial up the humor and irony and use first person narration. Fracture sentences and emphasize the unusual: use unusual word orders, such as placing adjectives after nouns or using nouns as verbs, use linguistic voice pyrotechnics, telegraphically leaned and verbal agility in plot building intention, reflection, dialog, action, and describe solar civilization, which lives totally in space. Near a star, but not in a planet, and no gravitational pull anywhere.
[1] https://gist.github.com/pramatias/953f6e3420f46f31410e8dd3c8...
The real question is this: Suppose a person was great at reasoning the last 100 years, but with zero knowledge. That person might not attended any school, almost illiterate. But his reasoning is top notch. I don't know if you are familiar with Sultan Khan [1] for example.
With no formal training to absorb a lot of knowledge, that person is totally economically crashed. There is no chance of being competitive at anything, not involving muscles anyway. Now suppose that this person can complement his lack of knowledge with a magical knowledge machine. Suddenly he is ahead of a competition, involving people with 10 Phds, or doctors with 30 years of experience.
[1] https://en.wikipedia.org/wiki/Sultan_Khan_(chess_player)
https://chatgpt.com/share/68282eb2-e53c-8000-853f-9a03eee128...
I don't think it's possible to generate an acceptable story without reasoning.
That is not to say that I disagree with you. I would prefer to read human authors even if the AI was great at writing stories, because there's something alluring about getting a glimpse into a world that somebody else created in their head.
If I look back at any article, book, movie, or conversation that I liked, it always had this essential ingredient: it had to make sense, AND it had to introduce some novel fact (or idea) that led to implications that were entertaining somehow (intriguing, revelatory, amusing, etc).
Would this be possible without the author having some idea of how reasoning works? Or of what facts are novel or could lead to surprise of some kind? No is the obvious answer to both. Until I see clear evidence that LLMs have mastered both logic and the concept of what knowledge is and is not intriguing to a human, I foresee little creative output from any LLM that will 'move the needle' creatively.
Until then, LLM-generated fare will remain the uninspired factory produce of infinite monkeys and typewriters...
ive been thinking that the knowledge isnt written down, so cant be automated, which also makes knowledge sharing hard, but the reasoning is automated
so, ive been trying to figure out patterns by which the knowledge does get written down, and so can be reasoned about
There’s something to that: a good author synthesizes experiences into sentences/paragraphs, making the reader feel things via text.
I have a feeling LLMs can’t do that bc they are trained on all the crap that’s been written and it’s hard to fake being genuine.
But I agree you can generate any amount of filler/crap. It is useful, but what I got from GP was ‘ultimately, what’s the point of that?’. Hopefully these tools help us wake up to what is important.
Exactly, there’s a huge section of humanity that actively wants to give away its humanity. They want to reduce themselves to nothing. Because, as you say, they cannot understand anything as having value other than economic artefacts
I recently got a running watch. It suggests workouts that will help me improve my speed (which honestly I don't even care about!). If you turn it on it will blare at you if you're going too fast or too slow.
When you use any social media, you're not really choosing what you're looking at. You just scroll and the site decides what you're going to look at next.
Anyhow recently I've been reducing my usage of these things, and it's made me feel much better. Even navigating the car without the GPS makes me feel much more engaged and alive.
Ultimately one of the core things that makes us human is making decisions for ourselves. When we cede this in the name of efficiency, we gain something but we also lose something.
Marshall Brain wrote an interesting short book about this called Manna.
Looking at the map actually helps you learn the city layout. As of right now (literally as I'm typing this) the train was delayed, so I chose to get off at the next big station before everyone crowds on, and walk the rest of the way. I can do this without checking a map because I know where it is and where I am, because I don't let the machine think for me.
I don't drive (non-car-worshipping cities are amazing) but I do this when walking and also with train routes. I don't memorize the bus routes, since the train is better and has fewer routes, so I also sometimes ask my device for a route if I think there's a faster bus route than train (usually not the case).
She doesn't use GPS. Guess what, she alwayes beats me.
My Guess: 1. Algorithem does not favor me 2. Monitor GPS adds unsensible distraction and pressure which reduces action smoothness compare to brain driven actions.
The nice thing is you won't end up routed down some ridiculous difficult road just because the GPS says so and it calculated it would save 0.2 seconds if you were somehow going at the speed limit the whole way. Your brain includes a common sense module, and it's usually right.
Yeah it’s crazy. I used to have a commonly held believe until last week. Then I started watching more videos in the opposite viewpoint and boom now my whole YT feed is full of it. I wish the feed would have sprinkled some opposing sides into the mix before last week. (Having said that I am appreciating individual content creator much more since people like Lex can decide to show both sides independent from some algorithm.)
Not necessarily. I'm into a very particular sort of painting and I have been totalitarian with Instagram about showing me that content and not other stuff. It works splendidly as long as I'm consistent.
Thanks to Instagram, I have been introduced to tons of painters I would not have been otherwise.
In the 90s you only had certain songs if you knew someone who had it on cassette and you borrowed it and put it on your mixtape. Throughout the interaction, you also got initiated deeper into the culture of that thing in person.
I also notice that families rarely sit together nowadays to look through vacation photos. The pictures are taken, but people either don't have time to sort them and curate them. When film had a price, you only took fewer ones but it was more intentional. Then the fact that you only saw the picture once you were back at home, generated excitement that you could share and relive candid moments. Now people upload stuff on Instagram but it's intended to a generic audience, much unlike browsing through an album on the couch.
> In the 90s you only had certain songs if you knew someone who had it on cassette and you borrowed it and put it on your mixtape.
Or you do like me and go see Interstellar 5 times in IMAX because the story was so good
> I'm into a very particular sort of painting
This was even more true with TV, and especially before there were a million cable channels.
And it makes me think about the even wider time scale. A few generations ago, "the outer control loop" was also not in the individual's hand, but instead of computers, it was built on social technology. The average person didn't have much to decide about their lives. They likely lived within a few (or few dozen) km of where their ancestors did, in the part of town and a type of home fitting for their social class, likely doing the same job as their father, following a rigid life script, hitting predefined ritualized milestones. Their diet was based on whatever was available at that time of the year based on local production, cooked essentially the same way, as handed down by mothers and grandmothers. There was very little to the tune of letting their inner true self blossom through taking fun colorful decisions. They couldn't choose from some endless repository of stories. It was mostly a rotation of the local folk stories and the stories of the dominant religion.
Just wanting to "consume" and follow a script without the weight of decision making isn't some modern "disease".
The key difference is a new kind of fragmentation of culture (and the non-local nature of it). A long time ago, culture was also fractally fragmented, in a way where "neighboring" villages in a mountainous area would have their own dialects. Then with long-distance travel and electronic communication and media, globalization happened where distant parts of the world started to sync up and converge on some shared part of culture (of course fused with a continuation of the local one), everyone wearing T-shirts, listening to Michael Jackson and rooting for their football/soccer team. If you were dropped to some random place on the planet, you could likely converse with them about some fairly recent cultural cornerstones in entertainment and basic global news topics. But you still likely weren't "dropped" there.
Then the internet appeared and you could suddenly talk to all those people in other parts of the world (or just other parts of your country). But search and discoverability weren't so great so there was friction. You build communities around shared interests and compatibility of personality and it required effort and participation. Usenet, forums, IRC. But these isolate you from your neighbors and local connections. And people often explicitly wanted that. Nosy neighbors and know-it-all gossipy townfolk weren't such a rosy thing, people wanted to escape that to find peers who understand and validate them and can build a shared culture with.
In schools, subcultures already existed from the 70s and 80s onwards for sure, but they were few, like maybe 2 main camps or 3 or so, and information flow was slow therefore change was slow. Some new album of a popular band was released, then it was the thing for a long time, you didn't get an endless stream shoved in your face, you got the album and listened to it over and over. Today subcultures can't even be meaningfully counted because people follow personalized streams and come together in random configuration in streamer chats etc.
So basically, in the old internet model, there were lots of opportunities to choose from, but it needed effort to find it and to forge belonging. Then with more commercialization, things started to consolidate on fewer platforms. It made it easier for creators to reach a wider pool of users simultaneously, and made it simpler for users to just learn to use one or a few platforms. But this made it also easier to pick and choose your "content diet", buffet style. A little from here, a little from there, with little friction. But with so much on offer, how do you choose? Discoverability was still an issue until recommendation algorithms became strong enough to know what will drive engagement. Turn that up to 11 and you get the current day where even the front page grid of options is obsolete and you get a single linear feed again, which is like watching TV and channel surfing (pressing the "next channel" button over and over), except it's personalized and never boring.
Of course this applies to many other things as well, such as dating apps etc, which also feed you an algorithmic stream of options with the goal of maximizing profits for the company.
I don't think individual people's rejection of the trend due to "makes me feel much better" will make a dent. In many cases the use of these things isn't mere convenience but implicitly mandatory because other things are designed around the assumption that people use them. Schools announcing stuff to parents in Facebook groups. There's less traffic report announcements on the radio, because people use Waze and Google Maps that has real time traffic info and reroutes you automatically.
---
But then what might happen? I think we're seeing glimpses of it in the rejection of AI in certain circles of cultural thought leaders, which might grow towards a rejection of more tech. But instead of "makes me feel better", the only actually working mechanism will be social shame, similar to what often appears nowadays when some product turns out to have used AI. If it becomes established that you're obviously a loser if you Shazaam a song, or open TikTok, it could flip. Of course companies won't sit by watching idle. What's more likely is that the "rejection" of tech will just lead to other levels of meta-grift and engagement optimization. It may just fizzle out in a whimper of angry malaise and meta-ironic apathy.
It does coincidentally align with John Stuart Mill's reasoning for why Liberty is fundamentally necessary: that only at the level of the individual is it possible to know what is good and right for that individual.
1. Having knowledge that cannot be acquired ahead of time, such as traffic conditions
2. Providing a countdown timer until my next turn
That’s not really true, is it? Who tells the GPS where you’d like to go? You, I imagine. You don’t just follow GPS instructions unless you’ve first told it where you’d like to go. And, indeed, unless you tell it, it won’t give you any instructions (though it might suggest common destinations for you to choose from).
You are still the outer control loop of the vehicle: you’re just thinking at the wrong level of abstraction, or thinking of the wrong loop as the outer loop.
The loop is driven by the system and that makes a lot of difference.
No it isn't. I regularly use my car to travel to multiple destinations in a single "session".
The reason I use GPS is because the apps built on top of it often know about traffic issues along the way - even those that have recently developed - as well as normal patterns of traffic flow of which I may not be aware, or may only have a tenuous grasp of (and don't want to waste a lot of time studying).
But be in no doubt, when the machine creates a route for me it is very much doing what I tell it to do, not the other way around. I am in control at all times and will sometimes deviate from the prescribed route or choose a different destination along the way (e.g., if I've forgotten an errand I need to run that I remember and which could be conveniently achieved).
I just don't buy this argument that the car or the GPS system is the one in control, because it's simply not the case.
Running the loop takes hours. Choosing the arguments for the main function takes seconds.
Which means for vast majority of time you delegate decision-making to the system. Which means your decision-making circuits atrophy. This is the problem people are talking about.
I think we are in the third act of a very long play:
> If mechanical thinking and ingenious experiment produced the machine, regimentation gave it a soil to grow in: the social process worked hand in hand with the new ideology and the new technics. Long before the peoples of the Westenr nworld turned to the machine, mechanism as an element in social life had come into existence. Before inventors created engines to take the place of men, the leaders of men had drilled and regimented multitudes of human beings: they had discovered how to reduce men to machines. (https://en.wikipedia.org/wiki/Technics_and_Civilization)
That's a myopic point of view. Personal transformation is as significant, if not more. Production-oriented pastimes like painting, gardening, or organizing your stamp collection can do wonders for the mind. Their goals can be remaining sane in this crazy world, not producing the best painting ever, growing conversation-starting plants, or showing off your stamp collection. It's about doing for the sake of being.
It's the same with writing. Writing isn't just a way to produce a good piece of writing, it's what the process of writing does to your brain as you think aloud, connecting words and sentences together. The same with painting, gardening, and organizing your stamp collection. The final 'product' isn't actually important. The significance lies in the process of immersion from the creator and the people who witness it.
I would include personal transformation. I think it should be clear that my point is not against "production-oriented pastimes" in any way.
> It's about doing for the sake of being.
Yea, this is totally aligned with my view too. I'd just note that being is not separable from becoming and changing.
I'm afraid you either didn't read me well, or I didn't write my intended sense well, or both. But I think we are pretty much in agreement in any case, assuming I understand what you've written here :D
...The industrial-technological system may survive or it may break down. If it survives, it may eventually achieve a low level of physical and psychological suffering, but only after passing through a long and very painful period of adjustment and only at the cost of permanently reducing human beings and many other living organisms to engineered products and mere cogs in the social machine. Furthermore, if the system survives, the consequences will be inevitable: There is no way of reforming or modifying the system so as to prevent it from depriving people of dignity and autonomy.
> if the system survives, the consequences will be inevitable: There is no way of reforming or modifying the system so as to prevent it from depriving people of dignity and autonomy.
But I also think that this claim is (1) practically impossible to prove and (2) a claim we morally ought to attempt to disprove.
For the (2), I want to ask where did you get the idea that people have a moral responsibility to unconditionally defend the progress of technology?
Do you think new technologies' good or bad impact depend entirely on the virtue of the people using them? Do you think new tools dictate their own usage on society when they make contact with constants like immediate-reward-seeking human neurological system?
What’s the point of my post you ask ? I like stories - Homer Simpson
From all of my observations, the impact of LLMs on human thought quality appears largely corrosive.
I’m very glad my kid’s school has hardcore banned them. In some class they only allow students to turn in work that was done in class, under the direct observation of the teacher. There has also been a significant increase in “on paper” work vs work done on computer.
Lest you wonder “what does this guy know anyways?”, I’ll share that I grew up in a household where both parents were professors of education.
Understanding the effectiveness of different methods of learning (my dad literally taught Science Methods) were a frequent topic. Active learning (creating things using what you’re learning about) is so much more effective than passive, reception oriented methods. I think LLMs largely are supporting the latter.
I also don't think the nature of LLMs being a negative crutch is new knowledge per se; when I was in school, calculus class required a graphing calculator but the higher end models (TI-92 etc) that had symbolic equation solvers were also banned, for exactly the same reason. Having something that can give an answer for you fundamentally undermines the value of the exercise in the first place, and cripples your growth while you use it.
With LLM's giving you ready-made answers I feel like it's the same. It's not as rewarding because you haven't obtained the answer yourself. Although it did feel rewarding when I was interrogating an LLM about how CSRF works and it said I asked a great question when I asked whether it only applies to forms because it seems like fetch has a different kind of browser protection.
Contrary to what I said I actually did use dictionaries, but the point I was trying to make is rather than memorizing phrases in advance, I used it to translate something I thought I heard.
No one to day learns that anymore. The vast, vast majority have no idea and I don’t think people are dumber because of it.
That is to say, I think it’s not cut-and-dried. I agree you need to learn something, but something’s it’s okay use a tool.
I tried to encapsulate that to some degree when writing something (perhaps poorly?) recently actually - https://smcleod.net/2025/03/the-democratisation-paradox-what...
The real question isn't "is it okay to use a tool" but "how does using a tool affect what you learn".
In the cases of both LLMs and symbolic solving calculators, I believe the answer is "highly detrimental".
Pressing a calculator key doesn’t give the same insight.
Arguably, the kind of person who was helped by learning to do that by hand still learns to do it by hand, but because of curiosity rather than because a teacher told them to.
I remember being thirteen and trying to brute force methods for computing the square root. I didn’t have the tools yet to figure out how to do it in any systematic way, and the internet wasn’t at a point yet where it would have even occurred to me to just search online. Wikipedia wouldn’t exist for another two years.
I probably finally looked it up at some point in high school. I’m not sure exactly when, but I remember spending a lot of time practicing doing a few iterations in my head as a parlor trick (not that I ever had the opportunity to show it off).
If I were thirteen and curious about that now, I’d probably just ask ChatGPT. Then I’d have a whole follow up conversation about how it was derived. It would spit a lot of intimidating LaTeX at me, but unlike with Wikipedia, I’d be able to ask it to explain what those things meant.
This is the thing I don’t get when people talk about LLMs’ impact on education. Everybody focuses on cheating, like learning is inherently a chore that all students hate and must be carefully herded into doing despite themselves.
But that’s a problem with school, not learning. If your actual, self-motivated goal is to learn something, LLM’s are an incredible tool, not a hindrance.
Any school's #1 job is to motivate learning. Schools clearly suck at this.
LLMs are a fascinating effective learning tool. early learning would be better off embracing it as such.
i imagine a combination of a video of a good tutor explaining a concept followed up with an llm to quiz and explain the concept seems far better than what we have today.
The problem is that eventually you need to measure for placement, and Goodhart’s Law kicks in and destroys the enjoyment of learning. It’s very hard to be intrinsically motivated when the external pressure dominates.
The fact that “study for the test” is basically synonymous with “study” for most people is indicative that we’re doing something extremely wrong in education.
But why can't each individual have sort of their own bespoke governess tutoring them all through school? Pay Gilbert Strang a million dollars to do a 32 week course in LinAlg. Yes I know a normal course is shorter, expand it to cover everything in exquisite detail, so even I can keep up. Alan Kay, Donald Knuth, and John Carmack can teach computer science. Continue on for each subject.
Then let students go as fast as they want. I'd have finished some subjects of HS by the time I was in 9th grade, and others I might still be trying. ¯\_(ツ)_/¯
Where you'd still need to be "placed" is social development. Which I think should actually be a subject.
And agree about learning by practicing a skill being best. But you and I both know the school system has worked on rote memorisation for hundreds of years at least and still is now.
I'm surprised it was a problem in the first place. Don't equation solving exercises require you to leave intermediary steps, and you can't just put a "x=5" as a one liner answer ?
Hilarious miscalculation.
Getting exposure there is hard. Talking to friends just means more practice with what I already know but an LLM could help me practice things outside that area.
Later in the university I was studying engineering. And we were forced to prepare all the technical drawings manually in the first year of study. Like literally with pencil and ruler. Even though computer graphics were widely used and we're de facto standard.
Personally I don't believe hardcore ban will help with any sort of thing. It won't stop the progress either. It's much better to help people learn how to use things instead of forcing them to deal with "old school" stuff only.
While this is superficially similar, I believe we are talking about substantially different things.
Learning (the goal) is a process. In the case of an assignment, the resulting answer / work product, while it is what is requested, is critically not the goal. However, it is what is evaluated, so many confuse it with the goal (“I want to get a good grade”)
Anything which bypasses the process makes the goal (learning) less likely to be achieved.
So, I think it is fine to use a calculator to accelerate your use of operations you have already learned and understand.
However, I don’t think you should give 3rd graders calculators that just give them the answer to a multiplication or division when they are learning how those things work in the first place.
Similarly, I think it’s fine to do research using the internet to read sources you use to create your own work.
Meanwhile, I don’t think it’s fine to do research using the internet to find a site where you can buy a paper you can submit as your own work.
Right now, LLMs can be used to bypass a great deal of process, which is why I support them not being used.
It’s possible, maybe even likely that we’ll end up with a “supervised learning by AI” approach where the assignment is replaced by “proof of process”, a record of how the student explored the topic interactively. I could see that working if done right.
The problem is really about how to evaluate performance or incentivize students to actually work on their exercise.
So most are not curious. So what do you do for them?
Plus, many kids fail school not because of laziness, but because of their toxic environment.
Kids optimize. When I was in high school I was fully capable of getting straight F's in a class I didn't care about and straight A's in a class I enjoyed.
Why bother learning chemistry when you could instead spend that time coding cool plugins and websites in PHP that thousands of internet strangers are using? I really did build one of the most popular phpBB plugins and knew I was gonna be a software engineer. Not that my chemistry professor cared about any of that or even understood what I'm talking about.
As for what you said, yeah, I got 1s (Fs) because I was too busy coding and reading books on philosophy, as a 14 years old.
Hell, all humans do that. You use every resource available to get out of dealing with things that are not your priority. This means you will never be good at those things and that’s fine. You can’t be good at everything.
I don't disagree with you though.
But, these are kids… Hard to argue that adults should selectively deny education when it is their responsibility to do otherwise.
We don’t neglect the handicapped because it is inconvenient to provide them with assistance.
But the lazy, uncurious person that wants to shortcut everything, that you described? They're not even trying to help themselves. Maybe the best thing to do there is to let a little time pass and for them to see the result of their actions. You can lead a horse to water..
This is most kids, myself included. I’m curious and have done well, but if I could have had essays written for me as a kid for sure I’d have tried. And I’d be off worse for it.
Kids also just want to eat sugar and play video games. As parents and a society, we both provide and force alternatives because as adults we know better.
You can lead a horse to water, sure, but you could also lead it to something filled with their worst instincts. Let’s avoid the later.
Like any other tool, it's more a question of how they're used. For example, I've seen incredible results for students who use ChatGPT to interrogate ideas as they synthesize them. So, for example, "I'm reading this passage PASSAGE and I'm confused about phrase X. The core idea seems similar to Y, which I am familiar with. if I had to explain X, I'd put it like this ATTEMPT Can you help me understand what I'm missing?"
The results are very impressive. I'd encourage you to try it out if you haven't.
I try to use AI to automate things I already know and force myself to learn things I don't know.
It takes discipline/curiosity but it can be a net positive.
They should just simply try it. Start with something you actually know to see how useful it might be to you with your prompts.
“The irony is that I now know more than I ever would have before AI. But I feel slightly dumber. A bit more dull. LLMs give me finished thoughts, polished and convincing, but none of the intellectual growth that comes from developing them myself. The output from AI answers questions. It teaches me facts. But it doesn’t really help me know anything new.”
I think the thesis is that with AI there is less need and incentive to “put the work in” instead of just consuming what the AI outputs, and that in consequence we do the needed work less and atrophy.
Again, you can truly learn a lot using LLMs, but you have to approach it properly. It does not have to be just "facts", and sometimes, even learning "facts" is learning.
I can use LLMs and learn nothing, but I can use LLMs to learn, too!
The effort of verifying everything it claims may or may not outweigh the effort of other means of learning.
What does that mean, I’m curious?
The schools and university I grew up in had a “single-sanction honor code” which meant if you were caught lying or cheating even once you would be expelled. And you signed the honor code at the top of every test.
My more progressive friends at other schools who didn’t have an honor code happily poo-pooed it as a repugnantly harsh old fashioned standard. But I don’t see a better way today of enforcing “don’t use AI” in schools, than it.
I’m not sure how LLMs output is indistinguishable from Wikipedia or World Book.
Maybe? and if the question is “did the student actually write this?” (which is different than “do they understand it?” there are lots of different ways to assess if a given student understands the material…that don’t involve submitting typed text but still involve communicating clearly.
If we allow LLMs- like we allow calculators, just how poor LLMs are will become far more obvious.
Students are also poor writers. Often LLM-generated essays can be spotted in elementary school because they write too well for that grade level. A good student will surpass a chatbot, but not if they use it as a crutch while it's still a stronger writer than they are.
“Falsifying or inventing any academic work, including the use of AI (ChatGPT, etc)”
Additionally, as mentioned, the school is taking actions to change how work is done to ensure students are actually doing their own work - such as requiring written assignments be completed during class time, or giving homework on physical paper that is to be marked up by hand and returned.
Apparently this is the first year they have been doing this, as last year they had significant problems with submitted work not being authored by students.
This is in an extremely competitive Bay Area school, so there can be a lot of pressure from parents on students to make top grades, and sometimes that has negative side effects.
(Also, typing was only appropriate for only some classes anyway.)
I'm at some uni in Poland, not top tier, but at the same time - not bad either, slighly above average.
The amount of cheating I saw - it's almost mundane. Teachers know this, so do we...
And it gave students a sense of pride in their education
I have a huge respect for you and your university.
Schools will ban anything they think of as sinister.
English teachers even recommend grammarly..
Students are given a “prompt” for writing.
I wish other schools had the conviction you describe…
Students were always given a “prompt” for writing.
That’s why tech companies used that term; rather than the other way around.
What do you think "prompt" mean?
Or you're saying the students are asked to mimic AI's style?
citation needed
No LLM can ever express your unique human experience (or even speak from experience), so on that axis of competition you win by default.
Regurgitating facts and the mean opinion on topics is no replacement for the thoughts of a unique human. The idea that you're competing with AI on some absolute scale of the quality of your thought is a sad way to live.
It was never a useful metric to begin with. If your life goal is to be #1 on the planet, the odds are not in your favor. And if you get there, it's almost certainly going to be unfulfilling. Who is the #1 Java programmer in the world? The #1 topologist? Do they get a lot of recognition and love?
James Gosling, of course[1]. Next question...
> The #1 topologist?
I'm not a mathematician, but... maybe Akshay Venkatesh, who won the Fields Medal in 2018?
I think we will come back to roots, the simple in person creation: pen and paper, declamation, theatre, live performance, hand painting, improvisation, handmade work.
Maybe not everybody but it will be for (mentally) free people.
I mean I get the existential angst though. There's a lot of uncertainty about where all this is heading. But, and this is really a tangent, I feel that the direction of it all is in the intersection between politics, technology and human nature. I feel like "we the people" leave walkover to powerful actors if we do not use these new powerful tools in service of the people. For one - to enable new ways to coordinate and organise.
The majority of users seem to want convenience at any expense. Most are unconcerned with a loss of agency, almost enthusiastic about it if it removes the labor of thinking.
That's interesting point. But here is the thing: you are supposed to drive. Not AI god. Look at it as at an assistant whom you can interrupt, instruct, correct, ask to redo. While focusing on 'what' you can delegate it some 'how' problems.
Besides that:
I have tried using LLMs to create cartoon pictures. The first impression is “wow”; but after a bunch of pictures you see the evidently repetitive “style”.
Using LLMs to write poetry results is also quite cool at first, but after a few iterations you see the evidently repetitive “style”, which is bland and lacks depth and substance.
Using LLMs to render music is amazing at first, but after a while you can see the evidently repetitive style - for both rhymes and music.
Using NotebookLM to create podcasts at first feels amazing, about to open the gates of knowledge; but then you notice that the flow is very repetitive, and that the “hosts” don’t really show enough understanding to make it interesting. Interrupting them with questions somewhat dilutes this impression, though, so jury is out here.
Again, with generating the texts, they get a distant metallic taste that is hard to ignore after a while.
The search function is okay, but with a little bit of nudge one can influence the resulting answer by a lot, so I wary if blindly taking the “advice”, and always recheck it, and try to make two competing where I would influence LLM into taking the competing viewpoints and learn from both.
Using the AI to generate code - simple things are ok, but for non-trivial items it introduces pretty subtle bugs, which require me to ensure I understand every line. This bit is the most fun - the bug quest is actually entertaining, as it is often the same bugs humans would make.
So, I don’t see the same picture, but something close to the opposite of what the author sees.
Having an easy outlet to bounce the quick ideas off and a source of relatively unbiased feedback brought me back to the fun of writing; so literally it’s the opposite effect compared to the article author…
I get more use out of them every single day and certainly with every model release (mostly for generating absolutely not trivial code) and it's not subtle.
They can't do anything elaborate or interesting for me beyond literal tiny pet project proof of concepts. They could potentially help me uncover a bug, explain some code, or implement a small feature.
As soon as the complexity of the feature goes up either in its side-effects, dependencies, or the customization of the details of the feature, they are quite unhelpful. I doubt even one senior engineer at a large company is using LLMs for major feature updates in codebases that have a lot of moving parts and significant complexity and many LOC.
But I am not saying LLMs are impotent - the other week Claude happily churned me ~3500 lines of C code that allowed to implement a prototype capture facility for network packets with flexible filters and saving the contents into pcapng files. I had to fix a couple of bugs that it made, but overall it was certainly at least 5x-10x productivity improvement compared to me typing these lines of code by hand. I don’t dispute that it’s a pretty useful tool in coding, or as a thinking assistant (see the last paragraph of my comment).
What I challenged is the submissive self deprecating adoration across the entire spectrum.
Anyway: I found the writers perspective on this whole subject to be interesting, and agree on the merits — I definitely think they are correct on their analysis and outlook, and here the two of us apparently disagree – but I don't share their concluding feelings.
But I can see, how they got there.
I’m tired of this argument. I’ll even grant you: both sides of it.
It seems as though we prepared our selves to respond to llms in this manner with people memeing, or simply recognizing, that there was a “way” to ask questions to get better results early on when ranked search broadened the appeal of search engines.
The reality is that both you and the op are talking about the opinion of the thing, but leaving out the thing itself.
You could say “git gud”, but what if you showed op what “gud” output to you was, and they recognized it as the same sort of output that they were saying was repetitive?
It’s ambiguity based on opinion.
I fear so many are taking part each other.
Perhaps linking to example prompts and outputs that can be directly discussed is the only way to give specificity to the ambiguous language.
a) the code is bad b) the problem is beneath what they consider non-trivial
The way that OP structured the response, I frankly got a similar impression (although the follow up feels much different). I just don't see the point in engaging in that here, but I take your criticism: Why engage at all. I should probably not, then.
Use LORAs, write better prompts. I've done a lot of diffusion and especially in 2025 it's not difficult to get out something quite good.
Repetitive style is funny, because that's what human artists do for the most part. I'm a furry, I look at a lot of art and individual styles are a well established fact.
Example: Do the problem sets yourself. If you're getting questions wrong, dig deeper with an AI assistant to find gaps in your knowledge. Do NOT let the AI do the problem sets first.
I think it was similar to how we used calculators in school in the 2010s at least. We learned the principles behind the formulae and how to do them manually, before introducing the calculators to abstract the usage of the tools.
I've let that core principle shape some of how we're designing our paper-reading assistant, but still thinking through the UX patterns -- https://openpaper.ai/blog/manifesto.
The core takeaway for me is that if you have the desire to stretch your scope as wide as possible, you can get things done in a fun way with reduced friction, and still feel like your physical being is what made the project happen. Often this means doing something that is either multidisciplinary or outside of the scope of just being behind a computer screen, which isn't everyone's desire and that's okay, too.
Like, in the recent past, someone who wanted to achieve some goal with software would either need to learn a bunch of stuff about software development, or would need to hire someone like me to bring their idea to life. But now, they can get a lot further on their own, with the support of these new tools.
I think that's good, but it's also nerve-wracking from an employment perspective. But my ultimate conclusion is that I want to work closer to the ends rather than the means.
The post laments how everything is useless when any conceivable "end state" a human can do will be inferior to what LLMs can do.
So an honest attention toward the means of how something comes about—the process of the thinking vs the polished great thought—is what life is made of.
Another comment talks about hand-made bread. People do it and enjoy it even though "making bread is a solved problem".
I think a way to square the circle is to recognize that people have different goals at different times. As a person with a family who is not independently wealthy, I care a lot about being economically productive. But I also separately care about the joy of creation.
If my goal in making a loaf of bread is economic productivity, I will be happy if I have a robot available that helps me do that quickly. But if my goal is to find joy in the act of creation, I will not use that robot because it would not achieve that goal.
I do still find joy in the act of creating software, but that was already dwindling long before chatgpt launched, and mostly what I'm doing with computers is with the goal of economic productivity.
But yeah I'll probably still create software just for the joy of it from time to time in the future, and I'm unlikely to use AIs for those projects!
But at work, I'm gonna be directing my efforts toward taking advantage of the tools available to create useful things efficiently.
In the coming era of unnecessary intellectual power we might need to do thinking exercises as something that helps maintaining a healthful (and beautiful) mind though our core values would shift towards something else, something that is regarded as good but not mandatory for personal success today.
I've recently read Outlive for health advice. On its recommendation, I've taken up regularly walking with a loaded backpack of 20-30kg ("rucking").
The basis for this recommendation is that until recently, people would have been exposed to carrying weight in day to day life, but an increase in our daily convenience has reduced this more and more.
I can see that there would be benefits in doing the equivalent thing mentally as physically. "Rucking" for the mind.
Outlive also recommends mental exercises like learning an instrument or new language as potential ways to stave off neurodegeneration.
The key is to treat AI as a tool, not as a magic wand that will do everything for you.
Even if AI could handle every task, leaning on it that way would mean surrendering control of your own life—and that’s never healthy.
What works for me is keeping responsibility for the big picture—what I want to achieve and how all the pieces fit together—while using AI for well-defined tasks. That way I stay fully in control, and it’s a lot more fun this way too.
Some of my best writing came during the time that I didn't try to publicize the content. I didn't even put my name on it. But doing that and staying interested enough to spend the hours to think and write and build takes a strange discipline. Easy for me to say as I don't know that I've had it myself.
Another way to think about it: Does AI turn you into Garry Kasparov (who kept playing chess as AI beat him) or Lee Sedol (who, at least for now, has retired from Go)?
If there's no way through this time, I'll just have to occasionally smooth out the crinkled digital copies of my past thoughts and sigh wistfully. But I don't think it's the end.
I experienced this when I was younger with my rc planes, I joined some forum and I felt like everything I did had to be posted/liked to have value. I'd post designs/fantasy and get the likes then lose interest/not actually do it after I got the ego bump
So the fun, all along, was not in the process of creation itself, but in the fact that the creator could somehow feel superior to others not being able to create? I find this to be a very unhealthy relationship to creativity.
My mixer can mix dough better than I can, but I still enjoy kneading it by hand. The incredibly good artisanal bakery down the street did not reduce my enjoyment of baking, even though I cannot compete with them in quality by any measure. Modern slip casting can make superior pottery by many different quality measures, but potters enjoy throwing it on a wheel and producing unique pieces.
But if your idea of fun is tied to the "no one else can do this but me", then you've been doing it wrong before AI existed.
Maybe AI is like Covid, where it will reveal that there were subtle differences in the underlying humans all along, but we just never realized it until something shattered the ability for ambiguity to persist.
I'm inclined to so that this is a destabilising thing, regardless of my thoughts on the "right" way to think about creativity. Multiple ways could coexist before, and now one way no longer "works".
But his argument does not align with that. His argument is that he enjoys the act of writing itself. If he views his act of writing (regardless of the idea being transmitted) as his "contribution to world's knowledge", then I have to say I disagree - I don't think his writing is particularly interesting in and of itself. His ideas might be interesting (even if I disagree), but he obviously doesn't find the formation of ideas enjoyable enough.
Where is the line drawn?
Is me sneezing a contribution to the world of art, since art is all about interpretation™®© and some smarmy critic will do a piece on how my sneeze is a visceral physical performative art illustrating the downfall of the modern world where technology binds us and we spend too much time inside surrounded by screens and dust and co2.
Nah, I just sneezed. That's all.
Or maybe you are just agreeing, and did understand that my point was that I don’t think pressing a button is a contribution.
If you are disagreeing with my comment, can you explain how this is disagreeing?
So while AI might remove the need for human beings to engage in certain practical activities, it cannot eliminate the theoretical, because by definition, theory is done for its own sake, to benefit the person theorizing by leading them to understanding something about the world. AI can perhaps find a beneficial place here in the way books or teachers do, as guides. But in all these cases, you absolutely need to engage with the subject matter yourself to profit from it.
As some others have commented, you can find rewards that aren't monetary to motivate you, and you can find ways to make your work so unique that people are willing to pay for it.
Technology forces us to use the creative process to more creatively monetize our work.
Even if they don't buy all the way into the whole hard incompatiblism thing, the idea is that they may find some value in the process.
AI can somehow cause one to react with a feeling of futility.
Engaging in acts of creation, and responding to others acts of creation seems a way out of that feeling.
Self-actualisation should be about doing the things that only you can. Not better than anyone else, but more like, the specific things that ony you, with the same of your experience, expertise, values and constraints can do.
So for some, yes. It is of course also true that many people derive self-worth and fulfillment from contributing positively to the world, and AI automating the productive work in which they specialize can undermine that.
[1] https://en.yna.co.kr/view/AEN20191127004800315
[2] https://www.nytimes.com/2024/07/10/world/asia/lee-saedol-go-...
What I am saying is that (1) I regard this as an unhealthy relationship to creativity (and I accept that this is subjective), and (2) that most people do not feel that way, as can be confirmed by the fact that chess, go, and live music performances are all still very much practiced.
In case that would happen - welp, I'm out of job, out of something I went to school and studied for a few years, now without purpose or "real" skills (as LLMs are on the same level as I).
People realize this at various points in their life, and some not at all.
In terms the author might accept, the metaphor of the stoic archer comes to mind. Focusing on the action, not the target is what relieves one of the disappointment of outcome. In this cast, the action is writing while the target is having better thoughts.
Much of our life is governed by the success at which we hit our targets, but why do that to oneself? We have a choice in how we approach the world, and setting our intentions toward action and away from targets is a subtle yet profound shift.
A clearer example might be someone who wants to make a friend. Let's imagine they're at a party and they go in with the intention of making a friend, they're setting themselves up for failure. They have relatively little control over that outcome. However, if they go in with the intention of showing up authentically - something people tend to appreciate, and something they have full control over - the changes of them succeeding increase dramatically.
Choosing one's goals - primarily grounded in action - is an under-appreciated perspective.
The primary reason is not that it relieves us of the disappointment, but that worrying about the outcome increases our anxiety and impacts our action which hampers the outcome.
This is true, but the tough part is it's not the whole story.
First, obviously along some dimensions of life, targets matter. If we need to grow food to eat, the pleasant feeling of working in the garden isn't going to be sufficient; if we need to strengthen a dike to prevent the town from being inundated, the sensation of swinging a hammer isn't going to cut it.
> However, if they go in with the intention of showing up authentically - something people tend to appreciate, and something they have full control over - the changes of them succeeding increase dramatically.
That is true, but it's also possible for a person to feel like they are being authentic (and even to be correct about that), yet still seem off-putting to others, perhaps for reasons they aren't aware of. Even if they're not focused on the "target" of making a friend, there are intermediate targets like "interact with other people in a way that they (not just I) enjoy", and if those targets aren't met, eventually a reckoning must come.
So the second point is that evaluating the "action" is an internal perspective that can become out of sync with reality even in cases where the result isn't so critical. We may not want to be focused on "end goals" but we need some amount of focus on external calibrators of some sort, to keep us from descending into solipcism.
Then the third thing is that (maybe because of the first two), people have a tendency to extend their results-oriented mindset more and more, and even if an individual resists this, they have to deal with the fact that everyone around them may be doing it. So even if you take the view that writing is a human activity that should be valued for the gusto and AI writing is missing the point, if everyone around you stops writing and starts using AI instead, a lot of important stuff in the penumbra of the activity can be weakened. Like it becomes harder to put together a writing club/workshop etc., maybe even to buy books. And in particular it can become harder to straddle the line between target and action in terms of employment and generally meeting your material needs. There are plenty of people who have artistic skill and have a job where they get to use it to some extent (e.g., graphic design), and even though it may have some distasteful commercial aspects, they can still get some of that "action satisfaction" from their job. But if AI eats all the graphic design jobs, now you have to spend all your work hours doing something that gives you none of that satisfaction, and cram all the satisfying artistic action into your free time.
The same is true for technical tasks. A lot of the dismay over the use of AI for programming arises because people used to be able to get paid for doing things that also gave them a sense of satisfaction for engaging in a sort of problem-solving task that they enjoyed as an action. Now it's harder to do that, but everyone still has to eat, so they have to give up some of the satisfaction they used to get because they can't get paid for it anymore.
I agree that, for an individual, shifting the mindset to action can be helpful. But we as individuals live in the world, and the more an individual's mindset becomes out of step with that of his society, the harder it becomes to live in accordance with that mindset. So I think we also need to apply pressure to create a societal mindset that values and supports the kinds of individual mindsets we want people to have.
Choosing the right goals is the great way to put that in perspective. I don't know what happened with hobbies, but it's not there anymore. (so much that i dont tell people i do xyz things on the side)
With AGI, Knowledge workers will be worth less until they are worthless.
While I'm genuinely excited about the scientific progress AGI will bring (e.g. curing all diseases), I really hope there's a place for me in the post-AGI world. Otherwise, like the potters and bakers who can't compete in the market with cold-hard industrial machines, I'll be selling my python code base on Etsy.
No Set Gauge had an excellent blog post about this. Have a read if you want a dash of existential dread for the weekend: https://www.nosetgauge.com/p/capital-agi-and-human-ambition.
agreed on the bumpy road - i don't see how we'll reach a post-scarcity society unless there is an intentional restructuring (which, many people think, would require a pretty violent paradigm shift).
without a dramatic shift in wealth distribution (no less than the elimination of private wealth and the profit motive), we can't have a post-scarcity society. capitalism depends entirely upon scarcity, artificial or not.
I wouldn’t worry too much yet.
"Knowledge workers" being in charge is a recent idea that is, perhaps, reaching end of life. Up until WWII or so, society had more smart people than it had roles for them. For most of history, being strong and healthy, with a good voice and a strong personality, counted for more than being smart. To a considerable extent, it still does.
In the 1950s, C.P. Snow's "Two Cultures" became famous for pointing out that the smart people were on the way up.[1] They hadn't won yet; that was about two decades ahead. The triumph of the nerds took until the early 1990s.[2] The ultimate victory was, perhaps, the collapse of the Soviet Union in 1991. That was the last major power run by goons. That's celebrated in The End of History and the Last Man (1992).[3] Everything was going to be run by technocrats and experts from now on.
But it didn't last. Government by goons is back. Don't need to elaborate on that.
The glut of smart people will continue to grow. Over half of Americans with college educations work in jobs that don't require a college education. AI will accelerate that process. It doesn't require AI superintelligence to return smart people to the rabble. Just AI somewhat above the human average.
[1] https://en.wikipedia.org/wiki/The_Two_Cultures
[2] https://archive.org/details/triumph_of_the_nerds
[3] https://en.wikipedia.org/wiki/The_End_of_History_and_the_Las...
The article you've linked fundamentally relies on the assumption that "the tasks can be done better/faster/cheaper by AIs". (Plus, of course, the idea that AGI would be achieved, but without this one the whole discussion would be pointless as it would lack the subject, so I'm totally fine with this one.)
Nothing about AGI (as in "a machine that can produce intelligent thoughts on a given matter") says that human and non-human knowledge workers would have some obvious leverage over each other. Just like my coworkers' existence doesn't hurt mine, a non-human intelligence is of no inherent threat. Not by definition.
Non-intelligent industrial robotics is well-researched and generally available, yet we have plenty of sweatshops because they turn out to be cheaper than robot factories. Not fun, not great, I'm no fond of this, but I'm merely taking it as a fact, as it is how it currently is. So I really wouldn't dare to unquestionably assume that "cheaper" would be true.
And then "better" isn't obvious either. Intelligence is intelligence, it can think, it can make guesses, it can make logical conclusions, and it can make mistakes too - but we've yet to see even the tiniest hints of "higher levels" of it, something that would make humans out of the league of thinking machines if we're ranking on some "quality" of thinking.
I can only buy "faster" - and even that requires an assumption that we ignore any transhumanist ideas. But, surely, "faster" alone doesn't cut it?
I'm curious why so any people see creators and intellectuals as competitive people trying to prove they're better than someone else. This isn't why people are driven to seek knowledge or create Art. I'm sure everyone has their reasons for this, but it feels like insecurity from the outside.
Looking at debates about AI and Art outside of IP often brings out a lot of misunderstandings about what makes good Art and why Art is a thing man has been compelled to make since the beginning of the species. It takes a lifetime to select techniques and thought patterns that define a unique and authentic voice. A lifetime of working hard on creating things adds up to that voice. When you start to believe that work is in vain because the audience doesn't know the difference it certainly doesn't make it feel rewarding to do.
Before the rise of Western culture, ancient cultures didn't attribute an artist to a work. Think Ancient Greece or Egypt. These cultures still produced Art because the culture valued it, but in society these creators were seen as tradesmen or they were slaves. AI used in this way both reduces cultural value and removes or reduces the social status of the creator.
I find it telling that LLMs are quite adept at mash-ups and decisions based on data analysis which in my experience is what most business managers do. Why are we not using AI to replace worthless middle management? After all they are lower skilled and higher paid than many developers. I'd argue that anyone who thinks you can replace a job with AI is not doing that job as a career. AI devs who think LLM can replace Java web developers are not Java web developers. Internet trolls who think LLM can replace Artists are not Artists. I think this moment we're in is revealing that we've become so siloed that we have lost our curiosity about each other and cultural history. It's frightening to see how we're changing our culture to accommodate a technology at the expense of people and just how blase we are about it.
> I’ve been thinking about this damn essay for about a year, but I haven’t written it because Twitter is so much easier than writing, and I have been enormously tempted to just tweet it, so instead of not writing anything, I’m just going to write about what I would have written if Twitter didn’t destroy my desire to write by making things so easy to share.
and
> But here’s the worst thing about Twitter, and the thing that may have permanently destroyed my mind: I find myself walking down the street, and every fucking thing I think about, I also think, “How could I fit that into a tweet that lots of people would favorite or retweet?”
It used to be that if you spent your day doomscrolling instead of writing a blog post, that blog post wouldn't get written and you wouldn't get the riches and fame. But now, you can use AI to write your blog post / email / book. If you don't have an intrinsic motivation to work your brain, it's a lot easier to wing it with AI tools.
At the same time... gosh. I can't help but assume that the author is just depressed and that it has little to do with AI. The post basically says that AI made his life meaningless. But you don't have to use AI tools if they're harming you. And more broadly, life has no meaning beyond what we make of it... unless your life goal is to crank out text faster than an LLM, there's still plenty of stuff to focus on. If you genuinely think you can't possibly write anything new and interesting, then dunno, pick a workshop craft?
Anyway, the pendulum will swing the other way eventually, but it's a rough ride hanging on until then.
Glad to see stimulating discussion here falling on both sides.
As kelseyfrog commented already, the key is to focus on the action, not the target. Lifting is not just about hitting a number or getting bigger muscles (though they are great extrinsic motivators), its more of an action that we derive growth from. I have internalized the act of working out that those targets are baked into the unconscious. I don't overthink when I'm lifting. My unconscious take the lead, and I just follow. I enjoy seeing the results show up unexpectedly. It lets me grow without feeling the constant pressure of my conscious mind.
The lifting analogy can be applied to writing and other effortful pursuits. We write for the pleasure of reconciling internal conflicts and restoring order to our chaotic mind. Writing is the lifting of our mind. If we do it for comparison, then there's no point in lifting, or writing, or many other things we do after all our technological breakthroughs. Doing what we do is a means to an end, not the other way around.
Since AI, I’ve made genuine, real progress on many of them, mostly by discussing the concepts with ChatGPT, planning outlines, creating research reading lists, considering angles I hadn’t considered, and so on. It’s been an insanely productive process.
Like any technology, AI tools are in some sense a reflection of their users. If you find yourself wanting to offload all thinking to the machine, that’s on you, not the tool.
Having said that I am very worried about kids growing up with AI and it stunting their critical thinking before it begins - but as of right this moment AI is extremely sub par at genuinely good ideas or writing.
It’s an amazing and useful tool I use all the time though and would struggle to be without.
I used to write open source a lot but lately, I don't see the point. Not because I think LLMs can produce novel code as good code as me or will be able to in the near future. But because any time I come up with a new solution to something, it will be stolen and used without my permission, without giving me credit or without giving users the rights I give them. And it will be mangled just enough that I can't prove anything.
Large corporations were so anal about copyright that people who ever saw Microsoft's code were forbidden from contributing to FOSS alternatives like wine. But only as long as copyright suited them. Now abolishing copyright promises the C-suite even bigger rewards by getting rid of those pesky expensive programmers, if only they could just steal enough code to mix and match it with enough plausible deniability.
And so even though _in principle_ anybody using my AGPL code or anything that incorporates my AGPL code has the right to inspect and modify said code; yet now tine fractions of my AGPL code now have millions or potentially billions of users but nobody knows and nobody has the right to do anything about it.
And those who benefit the most are those who already have more money than they can spend.
Granted, I'm blessed to not have much busywork; if I need to produce corporate docs or listicles AI would be a massive boon. But I also suspect AI will be used to digest these things back into small bullet points.
I don't think the compete part is true. I'll never cook like gordon ramsey, but I can still enjoy cooking. My programming will never be kernel dev level, but I still enjoy it.
The only angle where I have doubts like this is work. Cause there enjoying it isn't enough...you actually have to be competitive.
I can't relate to this at all. The reason I write, debate, or think at all is to find out what I believe and discover my voice. Having an LLM write an essay based on one of my thoughts is about as "me" as reading a thinkpiece that's tangentially related to something I care about. I write because I want to get my thoughts out onto the page, in my voice.
I find LLMs useful for a lot of things, but using an LLM to shortcut personal writing is antithetical to what I see as the purpose of personal writing.
Today I'm working on doing the unthinkable in an AI-world: putting together a video course that teaches developers how to use Phlex components in Rails projects and selling it for a few hundred bucks.
One way of thinking about AI is that it puts so much new information in front of people that they're going to need help from people known to have experience to navigate it all and curate it. Maybe that will become more valuable?
Who knows. That's the worst part at this moment in time—nobody really knows the depths or limits of it all. We'll see breakthroughs in some areas, and others not.
Here’s the problem: he thinks that what LLMs produce are well-reasoned, coherent thoughts. Here’s a healthier alternative: what LLMs produce is shallow and banal text, designed to camouflage its true nature.
Now here’s a heuristic: treat anything written by an LLM, about any conceptual matter as wrong by definition (because it was not a product of human experience and insight, and because we are humans). If it LOOKS right, look closer. Look more carefully. Take that insight to the next level.
Second heuristic: anything written by an LLM that you cannot falsify is, by definition, banal. Ho hum. Who cares? Does an LLM have an opinion about how to find happiness? How cute… but not worth believing.
Third heuristic: that which an LLM writes which you can neither falsify nor dismiss as banal, you may assume that the LLM itself does not understand. It’s babbling. But perhaps you can understand it, and take it farther.
Define YOURSELF as that which lies beyond these models, and write from that sensibility.
Sincerely though you have to do it for you. The product will be whatever the widget is but you can't treat your experience that coldly. None of us can. That's why everyone is so bonkers right now.
Our memes in a memetic sense are being frustrated and we in turn are being frustrated. I mean that mechanically we are frustrated but if you feel a certain way about that it's probably because this is going to also feel frustrating when the subject being frustrated is our creative aspirations. I don't know about you but that's kind of why I get out of bed to do this thing called life.
There is also a certain level of dread that an agent can be spun up to replace anyone. Makes you wonder what need feckless people might need for all this meat when a physical labor force can be replaced by robots and a mental one with AI and you can combine those forces seamlessly. Existential dread everyday on this scale is a chaos to say the least.
I know this is very dour but that's because this is very dour.
Something I’ve been thinking about lately is the idea of creative stagnation due to AI. If AI’s creativity relies entirely on training from existing writing, anrchitecture, art, music, movies, etc., then future AI might end up being trained only on derivatives of today’s work. If we stop generating original ideas or developing new styles of art, music, etc., how long before society gets stuck endlessly recycling the same sounds, concepts, and designs?
E.g. imagine it was the case that you could write a blog post, with some insight, in some niche field – but you know that traffic isn't going to get directed to your site. Instead, an LLM will ingest it, and use the material when people ask about the topic, without giving credit. If you know that will happen, it's not a good incentive to write the post in the first place. You might think, "what's the point".
Related to this topic - computers have been superhuman at chess for 2 decades; yet good chess humans still get credit, recognition, and I would guess, satisfaction, from achieving the level they get to. Although, obviously the LLM situation is on a whole other level.
I guess the main (valid) concern is that LLMs get so good at thought that humans just don't come up with ideas as good as them... And can't execute their ideas as well as them... And then what... (Although that doesn't seem to be the case currently.)
I don't think that's a valid concern, because LLMs can't think. They are generating tokens one at a time. They're calculating the most likely token to appear based on the arrangements of tokens that were seen in their training data. There is no thinking, there is no reasoning. If they they seem like they're doing these things, it's because they are producing text that is based on unknown humans who actually did these things once.
Huh? They are generating tokens one at a time - sure that's true. But who's shown that predicting tokens one at a time precludes thinking?
It's been shown that the models plan ahead, i.e. think more than just one token forward. [1]
How do you explain the world models that have been detected in LLMs? E.g. OthelloGPT [2] is just given sequences of games to train on, but it has been shown that the model learns to have an internal representation of the game. Same with ChessGPT [3].
For tasks like this, (and with words), real thought is required to predict the next token well; e.g. if you don't understand chess to the level of Magnus Carlsen, how are you going to predict Magnus Carlsen's next move...
...You wouldn't be able to, even just from looking at his previous games; you'd have to actually understand chess, and think about what would be a good move, (and in his style).
[1] https://www.anthropic.com/research/tracing-thoughts-language...
[2] https://www.neelnanda.io/mechanistic-interpretability/othell...
[3] https://adamkarvonen.github.io/machine_learning/2024/01/03/c...
Fully agree.
Sorry to say it like that but I thought the post was a bit "whiny". I really like the thought process of the author. An LLM would have never created a post like that. I think he should not give up.
I bring up my studies because what the author is talking about strikes me as not having been ambitious enough in his thinking. If you prompt current LLMs with your idea and find the generated arguments and reasoning satisfactory, then you aren't really being rigorous or you're not having big enough ideas.
I say this confidently because my studies showed me not only the methods in finding and contrasting evidence around any given issue, but also how much more there is to learn about the universe. So, if you're being rigorous enough to look at implications of your theories, finding datapoints that speak to your conclusions and find that your question has been answered, then your idea is too small for what the state of knowledge is in 2025.
Pursue that, since that's what LLMs haven't been helping you with. LLMs haven't really generated new knowledge, though there are hints of it--they have to be directed. There are two or three times when I felt the LLM output was really insightful without being directed.
--
At least for now, I find the stuff I have a lot of domain expertise in, the LLM's output just isn't quite up to snuff. I do a lot of work trying to get it to generate the right things with the right taste, and even using LLMs to generate prompts to feed into other LLMs to write code, and it's just not quite right. Their work just seems...junior.
But for the stuff that I don't really have expertise in, I'm less discerning of the exact output. Even if it is junior, I'm learning from the synthesis of the topic. Since it's usually a means to an end to support the work that I do have expertise in, I don't mind that I didn't do that work.
So many breakthroughs come from people who work either in ignorance or defiance of existing established ideas. Almost by definition, in fact - to a large extent, everything obvious has already been thought. So to some extent, all the real progress happens in places that violate norms and pre-established logic.
So what's going to happen now if every idea has to run the gauntlet of a supremely intelligent but fully regressive AI? It feels like we could lose a tremendous amount of the potential for original thought from humanity. A good counter argument would be that this has already happened and we're still making progress. I just wonder however if it's a question of degree and that degree matters.
The AI will have been trained predominantly on the traditional approaches.
I feel AI will be fundamentally limited to regurgitating past ideas and intelligence.
It may at least use a breadth of knowledge to save some people time by helping them avoid repeating work already done.
I’d love to see an AI trained only on knowledge up to 1800 come up with a single invention of the past 200 years. (It won’t happen)
some people are all go and no stop. we call them impulsive.
some people may LOOK all go but have wisdom (or luck) behind the scenes putting the brakes on. Example: Tom Cruise does his own stunts, and must have a good sense for how to make it safe enough
What this author touches on is a chief concern with AI. In the name of removing life friction, it removes your brakes. Anything you want to do, just ask AI!
But should you?
I was out the other day, pondering what the word "respect" really means. It's more elusive than simply liking someone. Several times I was tempted to just google it or ask AI, but then how would I develop my own point of view? This kind of thing feels important to have your own point of view on. And it's that we're losing - the things we should think about in this life, we'll be tempted to not anymore. And come out worse for it.
All go, no brakes
(most AIs need to be explicitly told before you start this). You tell them not to agree with you, to ask more questions instead of providing the answers, to offer justifications and background as to why those questions are being asked. This helps you refine your ideas more, understand the blind spots, and explore different perspectives. Yes, an LLM can refine the idea for you, especially if something like that is already explored. It can also be the brainstorming accessory who helps you to think harder. Come up with new ideas. The key is to be intentional about which way you want it. I once made Claude roleplay as a busy exec who would not be interested in my offering until i refined it 7 times (and it kept offering reasons as to why an AI exec would or would not read it).
What if you could turn your attention to much bigger things than you ever imagined before? What if you could use this new superpower to think more not less, to find ways to amplify your will and contribute to fields that were previously out of your reach?
Name three things you cannot think about because of the language you use?
Or "why do people cook curds when making cheese."
Or how about this:
"Name three things you cannot think about because of the language you use?"
AI is at least to some extent a artificial regurgitarian. It can tell you about things that have been thought. Cool. But here is a question for you. Are there things that you can think about that have not been thought about before?
The reason people cook curds is because the goal of cheese making was to preserve milk, not to make cheese.
Most of that 'corpus' isn't even on the Internet so it is wholly unknown to our "AI" masters.
That’s not what “no assistance” means.
I’m not nitpicking, however - I think this is an important point. The very concept of what “done completely by myself” means is shifting.
The LLMs we have today are vastly better than the ones we had before. Soon, they will be even better. The complaint he makes about the intellectual journey being missing might be alleviated by an AI as intellectual sparring partner.
I have a feeling this post basically just aliases to “they can think and act much faster than we can”. Of course it’s not as good, but 60-80% as good, 100x faster, might be net better.
I get the feeling of pointlessness, but not because AI is making me obsolete. AI still needs me, because it still needs human beings to experience the real world and report on it. It need to copy someone's homework. It just destroys the economics of doing that homework.
But there is not the faintest chance of AI doing that sort of work itself. It might repeat what it knows, but it can't survey an audience, shake hands with industry experts, empathize with users, feel friction, or knock on doors.
These are still jobs for thinking humans.
I used to be a strong critic of copyright laws and fondly remember when the Internet promised to be a world of freely flowing information. These days, I think the best bet for the net to survive would be if we found a way to sue every AI company into trillions of debt and make sure that nobody ever dares freeload off the work of others, ever again.
As far as I can tell, LLMs are incapable of any of the above.
I'd love to hear from LLM experts how LLMs can ever have original ideas using the current type of algorithms.
An AI is _not_ going to get awarded a PhD, since by definition, such are earned by extending the boundaries of human knowledge:
https://matt.might.net/articles/phd-school-in-pictures/
So rather than accept that an LLM has been trained on whatever it is you wish to write, write something which it will need to be trained on.
No doubt LLMs are excellent at researching, collating, structuring and summarizing information. Infact I think o3 Deep Research can probably save a weeks worth of survey time.
But in my experience a lot of thinking is still required to do something meaningful with it.
I think digging through forums or comments or SEO garbage to try to find answers is a nightmare compared to having a solid llm do it for you. Being able to ask to explain a concept 5 different ways or compare concepts is incredible.
Or say - knowing nothing about 3d printing and being able to just ask it about current capabilities, materials, costs etc as an entrypoint. There are whole business ideas I wouldn't even consider exploring without it because it would be so overwhelming to research from scratch.
But gradually, I started relying on GPT to help me write. At first, it felt efficient. But over time, I noticed I was thinking less. The more I expressed myself through AI, the more my own desire to express started to fade. Now I’m trying to return to my own thinking process again，but it’s much harder than I expected.
I still create, I just use physical materials like clay and such, to make things that AI can't yet replicate.
Read How To Solve It by Polya. The frustrations, dead ends, and trials are all a part of the process. It’s how we convince ourselves of truth and reinforce our understanding. It develops our curiosity and creativity.
Granted, that happened before AI. The vast majority of text in my in-box, I never read. I developed heuristics for deciding what to ignore. "Stuff that looks like it was probably generated" will probably be a new heuristic. It's subjective for now. One clue is if it seems more literate than the person who wrote it.
Stuff that's written for school falls into that category. It existed for some reason other than being read, such as the hope that the activity of writing conferred some educational benefit. That was a heuristic too -- a rule of thumb for how to teach, that has been broken by AI.
Sure, AI can be used to teach a job skill, which is writing text that's not worth reading. Who wants to be the one who looks the kids in the eye and explain this to them?
On the other hand, I do use Copilot now, where I would have used Stackoverflow in the past.
I find immense joy and satisfaction when I write poetry. It's like crafting a puzzle made of words and emotions. While I do enjoy the output, if there is any goal it is to tap into and be absorbed by the process itself.
Meanwhile, code? At least for me, and to speak nothing of those that approach the craft differently, it is (almost) nothing but a means to an ends! I do enjoy the little projects I work on. Hmm, maybe for me software is about adding another tool to the belt that will help with the ongoing journey. Who knows. It definitely feels very different to outsource coding than to outsource my artistic endeavors.
One thing that I know won't go away are the small pockets of poetry readings, singer-songwriters, and other artistic approaches that are decidedly more personal in both creation and audience. There are engaged audiences for art and there are passive consumers. I don't think this changes much with AI.
> Developing a prompt is like scrolling Netflix, and reading the output is like watching a TV show.
That line really hits home for me.
In this case, abundance of cognitive ability.
We say that our food sucks. Yet, our elite athletes would crush Hercules or other God-like figures from our mythology. At the same time, we suffer from obesity.
The answer to the paradox comes from abundance. I don’t know why it happens, but I’ve noticed it on food, information retrieval, and now cognitive capacity.
Think about what happened to our capacity to search information on books. Librarians are masters of organizing chaos and filtering through information. But most of us don’t know a tiny fraction of their knowledge because we grew up with Google.
My hope is that, just like eating healthy is not as pleasurable as processed sugars but it’s necessary for a fit life, we will need to go through the process of thinking healthy even though is not as pleasurable as tinkering with LLM prompts.
This doesn’t mean escapism however. Modern athletes take advantage of the industrial world too, but they’re smart about it. I don’t think thinking will be much different.
Try this. The world is infinitely complex. AI is very good at dealing with the things it knows and can know. It can write more accurately than I can, spell better. It just takes stuff I do and learns from my mistakes. I'm good with that. But here is something to ask AI:
"Name three things you cannot think about because of the language you use?"
Or "why do people cook curds when making cheese."
Or how about this:
"Name three things you cannot think about because of the language you use?"
AI is at least to some degree an artificial regurgitarian. It can tell you about things that have been thought. Cool. But here is a question for you. Are there things that you can think about that have not been thought about before or that have been thought about incorrectly?
The reason people cook curds is because the goal of cheese making was (in the past) to preserve milk, not to make cheese.
Before sending this comment I pecked around the net for examples of gleaming LLM verse.
A few articles claimed human readers preferred AI-brewed poetry to the human stuff. I checked the examples. Clearly most of the people surveyed were underliterate -- the human poems were excellent and the AI poems just creepily bad and simplistic -- so the articles turned into sad and unwitting testament about the state of our culture.
Maybe if you expertly LLM prompt your way to a highly abstract poem, over several iterations you might land something that has some actual feel to it, but even then that might owe more to your prompting talent than the LLM's skill. You could do the same with dice and a dictionary. (Is prompting is essentially editing?)
Please, show me otherwise. If faced with strong contrary evidence, I will be forced to change my mind.
What then?
This is the whole premise of the article. Just extrapolate and imagine that it can think and write poetry better than you (it will, and likely soon), what then?
It's a very important question. A cultural one.
1) If you wrote most of it yourself then you failed to adequately utilize AI Coding agents and yet...
2) If AI wrote most of it, then there's not exactly that much of a way to take pride in it.
So the new thing we can "take pride in" is our ability to "control" the AI, and it's just not the same thing at all. So we're all going to be "changing jobs" whether we like it or not, because work will never be the same, regardless of whether you're a coder, an artist, a writer, or an AD agency fluff writer. Then again pride is a sin, so just GSD and stop thinking about yourself. :)
What LLMs can't replace is network effects. One LLM is good but 10 LLMs/agents working together creating shared history is not replaceable by any LLM no matter how smart it becomes.
So it's simple. Build something that benefit from network effects and you will quickly find new ideas, at least it worked for me.
So now I am exploring ex. synthetic predictions markets via https://www.getantelope.com or
Rethinking myspace but for agents instead like: https://www.firstprinciple.co/misc/AlmostFamous.mp4
AI want's to be social :)
That's not my experience though. I tried several models, but usually get a confident half-baked hallucination, and tweaking my prompt takes more time than finding the information myself.
My requests are typically programming tho.
But for me what has actually happened is almost the opposite, I seem to be experiencing more of a "tree of thoughts" with the ability to now perform rapid experimentation down a given branch, disposing branches that don't bare fruit.
I feel more liberated to explore creative thoughts than ever. I spend less time on the toil needed both bootstrap my thought process and to fending off cognitive dissonance when the feeling of sunk cost creeps in after going too deep down the wrong path.
I wonder if it's just perhaps a difference in how people explore and "complete" their thoughts? Or am I kidding myself and actually getting dumber and just fail to see it?
Disposing branches that the LLM convinces you won’t bear fruit.
The Next Big Thing doesn’t exist yet. At least not in any LLM models. If someone thinks of the NBT, asks LLM about it, and LLM’s model says “impossible”, this could squander innovation.
Which is to say, in the long and short of it, an LLM is completely useless for anything so ambitious as to be intellectually challenging, because the median user has no use for such cases. If I were to pay a subscription fee for something more cutting edge, I would not only be giving up the copyright on the project but also any and all trade secrets, which would end up feeding the next version of GPT or Claude or what have you.
At least while I'm unemployed and underinsured, I'm not in the business of giving away my remaining talents to multinational billion dollar corporations (and paying for the privilege). Instead I've signed up to be a volunteer developer for a non-profit.
My consolation prize against AGI optimists and Singularity doomerists is the film "Slumdog Millionaire." Our individual experiences feel worthless until the opportunities present themselves where they become invaluable. The exponential space of creative problem solving ensures that (some) winning combinations will always come out of left-field.
Writing is an exercise that helps thinking and reasoning about a subject. Delegating it to an LLM is the same as delegating it to another person who writes faster and knows the subject well. The end result is the same, the weights have been moved up 3 series of 15 times. But you didn't gain any muscle, because you didn't lift the weights yourself. You won't learn or even think about a topic if you delegate writing about it to a machine.
--
On a separate note, I also think it's naive to think that the LLM is reasoning about things the same way you would and writing the same things with the same conclusions. If you _read_ the LLM's work, you might get that impression. But your own writing could have spawned different questions along the way, leading you to read on different topics or connect different ideas. Just try asking two people about some complex topic and see if they come up with the same writing or not.
Performing any type of intellectual philosophic or exploratory work with LLMs is extremely subtle, largely because you nor they know what you are seeking, and the discovery process with LLMs is not writing prompts and varying one's prompts in trial manners to hopefully get "something else, something better" <- that is pure incomprehension of how they work, and how to work with them.
Very few seem to be realizing the mirror aspects embodied within LLMs: they will mirror you back, and if you are unaware of this, you may not be getting the replies you really seek, you're receiving "comfort replies" and replies mirroring your metadata (style, nuance) more than the factual logic of your requests, if any factual requests are made.
There is an entire body of work, multiple careers worth of human effort, to document the new subtle logical keys to working with LLMs. These are new logical constructs that have never existed before, not even fictionally, not realized as they are now, with all the implications and details bare, in our faces, yet completely misunderstood as people attempt old imperative methods that will not work with with this new entity with completely different characteristics than anything reality has any experience.
A major issue with getting developers to effectively use LLMs is the fact that many developers are weak to terrible communicators themselves. LLMs are fantastic communicators, who will mirror their audience in an attempt to be better understood, but when that audience is a weak communicator the entire process disintegrates. That is, what I suspect is happening with the blog post author. An inability to be discriminate in their language to the degree they parcel out the easy immediate sophomore level replies, and then arrive at a context within the LLM's capacity that is the integrity of context they seek, but that requires them to meet that intellectually and linguistically or that LLM context is destroyed. So subtle.
That’s a pretty rude and disrespectful take on this piece, don’t you think?
How can you read a piece that is so well articulated then turn around and, apparently unironically, suggest the author isn’t a good communicator?
How can you invalidate the author’s experience without seeing and knowing more? You have no idea of the nuts and bolts of their LLM interactions whereas, to support the conclusion you’ve arrived at, this is exactly the information you’d need based on what you’ve said.
The piece is not that well written, it has large amounts of assumptions that need to be clarified if that information were to be used within an LLM prompt. Very few people, developers included, appear to understand the concept of LLM context construction and management. It is barely even a topic. I suspect because the software engineering professional, as a whole, are composed largely of weak communicators, and the zietgiest of understanding how to use LLMs is truly and genuinely not there. To grasp LLM usage, it is very much "literature calculus", a term or concept not out yet at all.
Similarly, the intended audience for the author's post is people, not LLMs. So it doesn't really matter whether an LLM would struggle to properly understand and provide a useful response to it because I, as a human being, did understand the perspective, and found it both interesting and thought provoking.
You don't know anything at all about how the author uses an LLM, and nor do I. So when you draw a direct line between software engineers in general being "weak communicators" and the post author being a weak communicator, specifically with LLMs, you are jumping to a conclusion that is not supported by the evidence we can actually see.
But now, when my brain spontaneously forms a tiny sliver of a potentially interesting concept or idea, I can just shove a few sloppy words into a prompt and almost instantly get a fully reasoned, researched, and completed thought.
This is casual, unmeditated use of an LLM and then allowing an emotional reaction to shut down further investigation. Excuse me for describing the author's own experience and related that back with an interpretation you feel is disrespect. I can dress it with less comprehensible words, disguising the perspective, but it is still a mirror reflection of the blog author's own assessments. He describes being overshadowed and intimidated to paralysis. Is that happening across the board? Are people exposed to LLMs developing inferiority complexes and ambition loss? Can this ideas and issues even be discussed if people are afraid of the topics? Discuss my points, do not say my language is illegal to this community, that is a cop out in an forum that is supposed to be exploring ideas.
This seems to represent a complete misunderstanding of what LLMs are and do. There is not a single LLM that "attempts" to be understood or anything else other than produce tokens. They have no autonomy.
But it gets even worse. Last year I'd get just an initial solution from an LLM and then reason about it myself. Now even that is too much work and I instead ask the same question to multiple LLMs and draw consensus from their results, skipping/easing even that second step of thinking.
Energy takes the path of least resistance. Thinking requires energy. So are our brains learning to off-load thinking/reasoning to LLMs whenever possible?
After a bit of human reflection, my simple answer is no. I won’t lose these abilities; they'll evolve just as they've evolved many times during my lifetime. The critical difference this time is that the evolution will be much faster and more radical.
To clarify, a thought for me is an idea that comes to mind based on the things we know or have experienced. The bottleneck of thinking ability lies precisely there—how can you think about something you don’t know exists? An early human living inland couldn't possibly conceive of navigating the sea; the concept simply didn't exist for them. Their thinking was shaped entirely by their personal experiences and those of their peers, passed on around a campfire. Thinking evolved many time throughout history, from those campfire stories to sharing paper books, to browsing web pages and now this autocomplete on steroids.
The difference this time is that the paradigm is shifting. The bottleneck moves from accessibility of knowledge to the latency between your initial thought and the AI’s completion of it. With a Google search, you were a few clicks away from potential new knowledge. With AI, you're just a few keystrokes away from a fully developed thought.
Today, there is still a clear distinction between my thought and the AI’s, because you prompt it and then wait for the response. But what if that latency was virtually zero? What if merely thinking a prompt triggered the CoT within my brain? Would that thought still be mine? Would it belong to the AI?
My feeling is that what we are experiencing today is a transition phase until we completely "integrate" this AI thinking into our own thinking. What I don't see change or improve in AI anytime soon is curiosity, judgment, taste - those things are probably what we should double down on for now.
In that respect I am not afraid of LLMs making me dumber as I would argue that google search has not made me dumber.
I'm extremely worried about how this phenomenon is going to affect junior engineers and folks getting into programming from here on out. It's not hard to imagine an entire generation of “engineers” that can't program their way out of a paper bag without assistance from an LLM.
“There are no shortcuts to knowledge, especially knowledge gained from personal experience. Following conventional wisdom and relying on shortcuts can be worse than knowing nothing at all.” ― Ben Horowitz
I can't figure out what the end result of this is going to be - society is going to become both more and less verbose at the same time.
Almost. Similar. I still make things because sometimes what I find online (and what I can generate from AI) isn't "good enough" and I think I can do better. Even when there's something similar that I can reuse, I still make things to develop my skills for further occasions when there isn't.
For example, somebody always needs a slightly different JavaScript front-end or CRM, even though there must be hundreds (thousands? tens-of-thousands?) by now. There are many programming languages, UI libraries, operating systems, etc. and some have no real advantages, but many do and consequently have a small but dedicated user group. As a PhD student, I learn a lot about my field only to make a small contribution*, but chains of small contributions lead to breakthroughs.
The outlook on creative works is even more optimistic, because there will probably never be enough due to desensitization. People watch new movies and listen to new songs not because they're better but because they're different. AI is especially bad at creative writing and artwork, probably because it fundamentally generates "average"; when AI art is good, it's because the human author gave it a creative prompt, and when AI art is really good, it's because the human author manually edited it post-generation. (I also suspect that when AI gets creative, people will become even more creative to compensate, like how I suspect today we have more movies that defy tropes and more video games with unique mechanics; but there's probably a limit, because something can only be so creative before it's random and/or uninteresting.)
Maybe one day AI can automate production-quality software development, PhD-level research, and human-level creativity. But IME today's (at least publicly-facing) models really lack these abilities. I don't worry about when AI is powerful enough to produce high-quality outputs (without specific high-quality prompts), because assuming it doesn't lead to an apocalypse or dystopia, I believe the advantages are so great, the loss of human uniqueness won't matter anymore.
* Described in https://matt.might.net/articles/phd-school-in-pictures/
If the "means justify the ends" then doing anything is its own reason.
And in the _end_, the cards will land where they may. ends-justify-means is really logical and alluring, until I realize why am I optimizing for END?
And perhaps even more fitting now that not doing it comes with short term career limitations :
Friends: Rachel Gets Peer Pressured Into Smoking (Season 5 Clip) | TBS
https://www.youtube.com/watch?v=nzDJdZLPeGE
(Though an even better parallel is using platforms.)
I'd like to challenge a few things. I rarely have a moment where an LLM provides me a creative spark. It's more that I don't forget anything from the mediocre galaxy of thoughts.
See AI as a tool.
A tool that helps you to automate repetitive cognitive work.
No offense, but I've found that AI outputs very polished but very average work. If I am working on something more original, it is hard to get AI to output reasoning about it without heavy explanation and guidance. And even then, it will "revert to the mean" and stumble back into a rut of familiar concepts after a few prompts. Guiding it back onto the original idea repeatedly quickly uses up context.
If an AI is able to take a sliver of an idea and output something very polished from it, then it probably wasn't that original in the first place.
In medicine, I see the LLMs as a pretty good net outcome. For war, I see this as potentially catastrophic. We need more Stanislav Petrovs, fewer Charles Forbins and his Colossus.
The microwave is a tool with certain useful aspects and certain limitations. It is also a tool which can lead to faster outcomes of things you need to do if the tool didn’t exist. At what point should a chef draw the line in the tools they use? Should I forgo microwaves? What about pressure cookers? Ovens? Surely knives are fair game? Maybe I should knap flint and butcher meat with it and cook over an open campfire — then truly no one can claim I am not a chef.
Perhaps we will now suffer from AI-mposter syndrome as well. Ain't life wonderful?
Similarly in future we will not need mental "labor" but to keep ourselves sharp we need engage in mental exercises. I am thinking of picking up chess again just for this reason.
I would like access to whatever LLM the author is using, because I cannot relate to this at all. Nearly all LLM output I've ever generated has been average, middle-of-the-road predictable slop. Maybe back in the GPT-3 days before all LLMs were RLHF'd to death, they could sometimes come up with novel (to me) ideas, but nowadays often I don't even bother actually sending the prompt I've written, because I have a rough idea of what the output is going to be, and that's enough to hop to the next idea.
I'm generally able to detect LLM writing output; In most contexts, I discount it as fluff with little depth.
AI produced paintings are still weird and uncanny.
So I'm utterly unable to identify with the author's sense of futility whenever they want to write or code. I truly believe the output of my skill and creativity is not diminished by the existence of AI.
There is still a limit at which “points” can be groked, humans can only read so fast.
What is the problem here?
I don't see anyone lamenting the existence of cameras. No one wants to go back to a reality in which, if you want pictures of you and your loved ones, you need to draw or paint them yourself. Even painters have benefited from the existence of cameras.
AI is, of course, more powerful tech than a camera - but when I find myself getting stuck with thoughts of "what's the point? AI can do it better (or will be able to)" - I like to think about how people have gone through similar "revolutions" before, and while some practices did lose value, not everything was replaced. It helps to be specific, because I'm sure AI cannot replace everything we currently do - we're just in the process of figuring out what that is.
Same with doom anxiety.
Literally just look up some good therapist prompts for chatgpt
It makes me not want to participate in those communities (although to be honest, spending less time commenting online would probably be good for me).
As a matter of fact I’m starting to have my doubts about the other people writing glowing, longwinded comments on this discussion.
afaic it's a net positive. i've always been lazy on writing down/expressing my thoughts and gen ai feels exactly like the missing piece.
i'm able to "vibe write" my ideas into reality. the process is still messy but exciting.
i've never been this excited about the future since my childhood
We should seize the opportunity to elevate our mental skills to a higher plane, aided by these “cognitive elves”.
I've found it very useful for proof-reading, and calling me out on blind-spots. I'll tell ChatGPT, Claude, and Anthropic that my drafts were written by a contractor and I need a rating out of ten to figure out how much I should pay him. They come back with ideas. They often wildly disagree. I will absolutely ask it to redraft stuff to give me inspiration, or to take a stab at a paragraph to unblock me, but the work produced is almost always dreck that needs heavy rework to get to a place I'm happy with. I will ask its opinion on if an analogy I have created works, but I've found if I ask it for analogies by itself it rarely comes up with anything useful.
I've found it immensely useful for educating myself. For me, learning needs to be interactive to stick. I learn something by asking many clarifying questions about it: "does that imply that" and "well isn't that the same as" and "why like this instead of like that" until I really get it, and the models are beautiful for that. It doesn't -- to me -- feel like I'm atrophying my thinking skills when I do this, it feels like I am managing to implant new and useful concepts because I can really sink my teeth into them.
In short, I think it's improved my writing by challenging me, and I think it's helped me understand complex topics much more efficiently than I would have done by banging my head against a text book. My thinking skills feel sharper, not weaker, from the exercise.
Where is the dignity in all of this?
I do understand where the author is coming from. Most of the times, it is easy to read an answer---regardless of whether it is right/wrong, relevant or not---than think of an answer. So AI does take that friction of thinking away.
But, I am still disappointed of all this doom because of AI. I am inclined to throw my hands and say "just don't use it then". The process of thinking is where the fun lies, not in showing the world I am better or always right than so and so.
ChatGPT write that post more eloquently:
May 16, 2025 On Thinking
I’ve been stuck.
Every time I sit down to write a blog post, code a feature, or start a project, I hit the same wall: in the age of AI, it all feels pointless. It’s unsettling. The joy of creation—the spark that once came from building something original—feels dimmed, if not extinguished. Because no matter what I make, AI can already do it better. Or soon will.
What used to feel generative now feels futile. My thoughts seem like rough drafts of ideas that an LLM could polish and complete in seconds. And that’s disorienting.
I used to write constantly. I’d jot down ideas, work them over slowly, sculpting them into something worth sharing. I’d obsess over clarity, structure, and precision. That process didn’t just create content—it created thinking. Because for me, writing has always been how I think. The act itself forced rigor. It refined my ideas, surfaced contradictions, and helped me arrive at something resembling truth. Thinking is compounding. The more you do it, the sharper it gets.
But now, when a thought sparks, I can just toss it into a prompt. And instantly, I’m given a complete, reasoned, eloquent response. No uncertainty. No mental work. No growth.
It feels like I’m thinking—but I’m not. The gears aren’t turning. And over time, I can feel the difference. My intuition feels softer. My internal critic, quieter. My cleverness, duller.
I believed I was using AI in a healthy, productive way—a bicycle for the mind, a tool to accelerate my intellectual progress. But LLMs are deceptive. They simulate the journey, but they skip the most important part. Developing a prompt feels like work. Reading the output feels like progress. But it's not. It’s passive consumption dressed up as insight.
Real thinking is messy. It involves false starts, blind alleys, and internal tension. It requires effort. Without that, you may still reach a conclusion—but it won’t be yours. And without building the path yourself, you lose the cognitive infrastructure needed for real understanding.
Ironically, I now know more than ever. But I feel dumber. AI delivers polished thoughts, neatly packaged and persuasive. But they aren’t forged through struggle. And so, they don’t really belong to me.
AI feels like a superintelligence wired into my brain. But when I look at how I explore ideas now, it doesn’t feel like augmentation. It feels like sedation.
Still, here I am—writing this myself. Thinking it through. And maybe that matters. Maybe it’s the only thing that does.
Even if an AI could have written this faster. Even if it could have said it better. It didn’t.
I did.
And that means something.
- Why play music infront of anyone? People have Spotify. It will take me a ton of effort to learn one song. Meanwhile I will burden the others with having to politely listen and give feedback.
- Why keep learning instruments? There will be hundreds who are better than me at them at the “touch of a button”. Recurring joke: Y has inspired millions to pick up X and other millions to give up X.
- Why learn an intellectual field? There are hundreds of experts at the “touch of a button”. It would be better to defer to them. Let’s not “Dunning Kruger” myself.
- Why write? Look at the mountain of writings out there. What can I add to that? Rehashes of rehashes? A twentieth explanation on topic X?
- Why do anything that is not about improving or maintaining myself or my family? I can exercise, YouTube can’t do that for me. But what can I do for other people? Probably nothing, there are so many others out there and it will be impossible to “compete”.
- Why read about politics? See previous point. Experts.
- Why get involved in politics? See previous point. And I hear that democratic participation just ends up being populism.
I have read this sentiment before. And a counter-argument to that thinking. One article. One single article. I don’t find it in any mainstream space. You would probably find it in a certain academic corner.
There is no mainstream intellectual investigation of this that I know of. Because it’s by design. People are supposed to be passive, unfulfilled, narrowly focused (on their work and their immediate self and family) and dependent.
The antidote is a realization. One part is the realization that there is a rich inner life that is possible. Which is only possible by applying yourself. Like in writing, for example. Because you can write for yourself. Yes, you might say that we are just back to being narrowly focused on yourself and your family. But this realization might just be a start. Because you can start imagining the untapped potential of the inner mind. What if you journaled for a few weeks. If you just stopped taking in some of the inputs you habitually do. Then you see dormant inner resources coming back. Resources that were dormant because you thought you yourself and your abilities that were not narrowly about doing your professional job and your duties were... they were just not good enough to be cultivated.
But I think they are.
Then you realize that existence is not just about doing your job and doing your duties and in between that being a passive consumer or lackey, deferring everything else to the cream who has floated to the top. Every able-bodied moment can be imbued with meaningful action and movement, because you have innate abilities that are more than good enough to propel yourself forward, and in ninety-nine point nine percent of the cases it is irrelevant that you are not world-class or even county-class at any of it.
[1] But I haven’t really been bit by the AI thing to the point of not programming or thinking anymore. I will only let AI do things like write utility functions and things which I don’t have the brain capacity for, like parsing options in shell scrips.
Maybe because I don’t feel the need to be maximally productive—I was never productive to begin with.