yannyu

> So what do you do if you are charged with providing shareholders ever-increasing returns on their investments by selling goods and services?

> Growth must be maintained. Value must be created. The futurist never took into account that part.

The futurists did take this into account, they just weren't writing utopian fiction. Weyland Yutani, Tyrell Corp, and other Megacorps were the futurists warning us what could happen if we let profit and sharedholder value becomes the most important thing in society.

Structured data doesn't have be a database. It can be a checklist, a particular working layout, or even just a defined process. Many high functioning companies spent a lot of time on those kinds of things, which became a competitive advantage.

As much as I hated schooling, I do want to say that there are parts of learning that are simply hard. There are parts that you can build enthusiasm for with project work and prioritizing for engagement. But there are many things that people should learn that will require drudgery to learn and won't excite all people.

Doing derivatives, learning the periodic table, basic language and alphabet skills, playing an instrument are foundational skills that will require deliberate practice to learn, something that isn't typically part of project based learning. At some point in education with most fields, you will have to move beyond concepts and do some rote memorization and repetition of principles in order to get to higher level concepts. You can't gamify your way out of education, despite our best attempts to do so.

> in class programming exams are basically the same thing as leetcode job interviews, and we all know what a bad proxy those are for "real" development work.

Confusing university learning for "real industry work" is a mistake and we've known it's a mistake for a while. We can have classes which teach what life in industry is like, but assuming that the role of university is to teach people how to fit directly into industry is mistaking the purpose of university and K-12 education as a whole.

Writing long-form prose and essays isn't something I've done in a long time, but I wouldn't say it was wasted effort. Long-form prose forces you to do things that you don't always do when writing emails and powerpoints, and I rely on those skills every day.

Proof isn't the goal here. Centuries of thought and experimentation have made it clear that we currently have no ability to decisively determine whether something is conscious.

However, as humans we intuitively build projections of what we believe is the internal world of other beings. We also clearly believe there is a continuum of complexity of thought among all the beings that we have observed.

The question then becomes, what behaviors of computer programs match up with what we consider conscious behaviors of other beings we have observed? This is a necessary question because we don't have access into the internal states of others, so we have to interrogate the full complexity of what we believe represents consciousness, and whether these beings match those behaviors.

That is what I meant, but my brain has been rotted by the MCU it seems.

More importantly, if your entire existence were being fed a corpus of text and then being asked to regurgitate it on demand, would you be remotely similar to the person you are now? When we take consciousness-capable beings and subject them to forms of sensory and agency deprivation, the results might also have you assume they weren't capable of consciousness to begin with.

This is where the robotics industry wants to go. Generalist robots that have an intelligence capable of learning through observation without retraining (in the ML sense). Whether and when we'll get there is another question entirely.

I agree, evaluation of consciousness is another problem entirely.

However, the point I'm making is that even assuming an agent/thing is capable of achieving consciousness, it would have to have a suitably complex environment and the capability of forming an independent feedback loop with that environment to even begin to display conscious capability.

If the agent/thing is capable of achieving consciousness but is not in a suitable environment, then we'd likely never see it doing things that resemble consciousness as we understand it. Which is something we have seen occur in the real world many times.

Okay, you're right. There is a world, and some hypotheses, and some falsifiability.

But how rich is this world?

Does this world progress without direct action from another entity? Can the agent in this case form hypotheses and test them without intervention? Can the agent form their own goals and move towards them? Does the agent have agency, or is it simply responding to inputs?

If the world doesn’t develop and change on its own, and the agent can’t act independently, is it really an inhabited world? Or just a controlled workspace?

I don't think it's brief at all. Animals do this experimentation as well, but clearly in different ways. The scientific method is a formalized version of this idea, but even the first human who cooked meat or used a stick as a weapon had a falsifiable hypothesis, even if it wasn't something they could express or explain. And the consequences of testing the hypothesis were something that affected the way they acted from there on out.

I agree, I'm sure people have put together things like this. There's a significant profit and science motive to do so. JEPA and predictive world models are also a similar implementation or thought experiment.

Coherence here isn't about legible text, it's environmental coherence where you can deduce truths about the world through hypotheses and experimentation. Coherence isn't about a consistent story narrative, it's about a persistent world with falsifiable beliefs and consequences.

Okay, so you're talking about LLMs specifically in the context of a ChatGPT, Claude, or pick-your-preferred-chatbot. Which isn't just an LLM, but also a UI, a memory manager, a prompt builder, a vectorDB, a system prompt, and everything else that goes into making it feel like a person.

Let's work with that.

In a given context window or conversation, yes, you can have a very human-like conversation and the chatbot will give the feeling of understanding your world and what it's like. But this still isn't a real world, and the chatbot isn't really forming hypotheses that can be disproven. At best, it's a D&D style tabletop roleplaying game with you as the DM. You are the human arbiter of what is true and what is not for this chatbot, and the world it inhabits is the one you provide it. You tell it what you want, you tell it what to do, and it responds purely to you. That isn't a real world, it's just a narrative based on your words.

We can form and test hypotheses and experience the consequences. And then take that knowledge to our next trial. Even dogs and cats do this on a daily basis. Without that, how would we even evaluate whether something is conscious?

Let’s make an ironman assumption: maybe consciousness could arise entirely within a textual universe. No embodiment, no sensors, no physical grounding. Just patterns, symbols, and feedback loops inside a linguistic world. If that’s possible in principle, what would it look like? What would it require?

The missing variable in most debates is environmental coherence. Any conscious agent, textual or physical, has to inhabit a world whose structure is stable, self-consistent, and rich enough to support persistent internal dynamics. Even a purely symbolic mind would still need a coherent symbolic universe. And this is precisely where LLMs fall short, through no fault of their own. The universe they operate in isn’t a world—it’s a superposition of countless incompatible snippets of text. It has no unified physics, no consistent ontology, no object permanence, no stable causal texture. It’s a fragmented, discontinuous series of words and tokens held together by probability and dataset curation rather than coherent laws.

A conscious textual agent would need something like a unified narrative environment with real feedback: symbols that maintain identity over time, a stable substrate where “being someone” is definable, the ability to form and test a hypothesis, and experience the consequences. LLMs don’t have that. They exist in a shifting cloud of possibilities with no single consistent reality to anchor self-maintaining loops. They can generate pockets of local coherence, but they can’t accumulate global coherence across time.

So even if consciousness-in-text were possible in principle, the core requirement isn’t just architecture or emergent cleverness—it’s coherence of habitat. A conscious system, physical or textual, can only be as coherent as the world it lives in. And LLMs don’t live in a world today. They’re still prisoners in the cave, predicting symbols and shadows of worlds they never inhabit.

Pretty similar to social media in a lot of ways. They've strip mined the commons and provided us a corporate controlled walled garden to compensate us for our loss.

The naming collision here is unfortunate since the two kinds of models described couldn't be any more different in purpose. Maybe JEPA-type world models should explicitly be called "predictive world models".

Social media is just a portion of the problem though. As parents, we hear from our kids and other parents about how children all over the country are desperate to get their hands on a smartphone, and we have conversations about the right time to get them a phone, what functionality that phone should have, how much parents should monitor it, etc.

When you ask the kids why they want a smartphone, typically the main two reasons are to be able to communicate with friends and to be present on social media. A child's experience is that socializing is done through texting, video calls, and social media now, rather than being in-person at a park, store, or cafe.

When you ask parents why they gave their kids a smartphone, typically parents will cite the socialization reason above, as well as safety and "not wanting their kids to fall behind". Many parents believe it's in their and their child's best interest to have the ability to monitor where their child is and what their child is doing and are willing to go along with the socialization thing to achieve that.

Social media is a part of the problem, but the underlying problem for children is that many aren't able or willing to socialize outside of devices, and the underlying problem for parents is that they aren't able or willing to let their children be out of communication for long periods of time.

It isn't just that social media is a problem, it's that we have cultivated a world where children, parents, and cities/governments see personal devices and social media as the best available solution for the intersecting problems of community, socialization, and safety.

I think the underlying issue that the article briefly mentions but is likely a much more prominent cause is that we have destroyed local, physical communities over the past 30 years.

The same community where I grew up biking, walking, and wandering around as a child now has police who say that they must respond if an adult calls in about unsupervised children anywhere in town.

A combination of rising costs, privatization, social media, and smart phones has made it so children can't spend time together without adults monitoring them. They literally are not engaging with each other in physical space the way that many of us engaged with each other as children up until the 2000s. We can see and feel that things are different as adults, but for children the real impact is socially and emotionally stunting. Children no longer have any low-stakes space where they can experiment and learn. And when we destroy the physical, in-person community where children can find support and comfort, should we be surprised when they latch onto the loudest people on the internet?

I agree. The two of us are talking past each other, and I wonder if it's because there's a certain strain of thought around LLMs that believes that epistemological questions and technology that we don't fully understand are somehow unique to computer science problems.

Questions about the nature of knowledge (epistemology and other philosophical/cognitive studies) in humans are still unsolved to this day, and frankly may never be fully understood. I'm not saying this makes LLM automatically similar to human intelligence, but there are plenty of behaviors, instincts, and knowledge across many kinds of objects that we don't fully understand the origin of. LLMs aren't qualitatively different in this way.

There are many technologies that we used that we didn't fully understand at the time, even iterating and improving on those designs without having a strong theory behind them. Only later did we develop the theoretical frameworks that explain how those things work. Much like we're now researching the underpinnings of how LLMs work to develop more robust theories around them.

I'm genuinely trying to engage in a conversation and understand where this person is coming from and what they think is so unique about this moment and this technology. I understand the technological feat and I think it's a huge step forward, but I don't understand the mysticism that has emerged around it.

I never claimed we already know everything about LLMs. Knowing "everything about" anything these days is impossible given the complexity of our technology. Even antennae, a centuries old technology, is something we're still innovating on and don't completely understand in all domains.

But that's a categorically different statement than "no one understands how an LLM works", because we absolutely do.

You're spending a lot of time describing whether we know or don't know LLMs, but you're not talking at all about what it is that you think we do or do not understand. Instead of describing what you think the state of the knowledge is about LLMs, can you talk about what it is that you think that is unknown or not understood?

It's remarkably easy to get ChatGPT to hallucinate. A go-to for me is to ask it about factual things as compared to other factual things. The description of the Bluey episode is wrong in nearly every possible way. Additionally, the response continues on analyzing in multiple ways, and again continues to get the fundamental facts and therefore the analysis wrong.

Prompt: Compare the Space episode of Bluey to Interstellar

Response: That’s a really fun and surprisingly fitting comparison! Let’s break down how Bluey’s “Space” episode parallels Christopher Nolan’s Interstellar — both emotionally and thematically — even though one is a 7-minute preschool cartoon and the other a 3-hour sci-fi epic.

1. Core Theme: Love Across Distance

Interstellar: The film centers on a father (Cooper) who leaves his daughter (Murph) to save humanity, exploring how love transcends time and space — literally. The emotional heart of Interstellar is the bond between parent and child that endures despite impossible separation.

Bluey: “Space” In this episode, Bluey, Bingo, and their dad (Bandit) play an imaginative game where they’re astronauts who have to separate to save their mission. The moment Bluey and Bandit must say goodbye echoes that same emotional tension — the pain of letting go but also the love that motivates sacrifice.

Parallel: Both stories use the setting of outer space as a metaphor for emotional distance — how people we love can feel far away, and how connection can bridge that gap.

> Then why not say "they are just computer programs"?

LLMs are probabilistic or non-deterministic computer programs, plenty of people say this. That is not much different than saying "LLMs are probabilistic next-token prediction based on current context".

> I think the reason people don't say that is because they want to say "I already understand what they are, and I'm not impressed and it's nothing new". But what the comment you are replying to is saying is that the inner workings are the important innovative stuff.

But we already know the inner workings. It's transformers, embeddings, and math at a scale that we couldn't do before 2015. We already had multi-layer perceptrons with backpropagation and recurrent neural networks and markov chains before this, but the hardware to do this kind of contextual next-token prediction simply didn't exist at those times.

I understand that it feels like there's a lot going on with these chatbots, but half of the illusion of chatbots isn't even the LLM, it's the context management that is exceptionally mundane compared to the LLM itself. These things are combined with a carefully crafted UX to deliberately convey the impression that you're talking to a human. But in the end, it is just a program and it's just doing context management and token prediction that happens to align (most of the time) with human expectations because it was designed to do so.

The two of you seem to be implying there's something spooky or mysterious happening with LLMs that goes beyond our comprehension of them, but I'm not seeing the components of your argument for this.

> Yeah, but that's their interface. That informs surprisingly little about their inner workings.

I'm not sure I follow. LLMs are probabilistic next-token prediction based on current context, that is a factual, foundational statement about the technology that runs all LLMs today.

We can ascribe other things to that, such as reasoning or knowledge or agency, but that doesn't change how they work. Their fundamental architecture is well understood, even if we allow for the idea that maybe there are some emergent behaviors that we haven't described completely.

> It's a possible outcome of the process, but it's not the only possible outcome.

Again, you can ascribe these other things to it, but to say that these external descriptions of outputs call into question the architecture that runs these LLMs is a strange thing to say.

> Looking at a LLMs I/O structure and training process is not enough to conclude much of anything. And that's the misconception.

I don't see how that's a misconception. We evaluate all pretty much everything by inputs and outputs. And we use those to infer internal state. Because that's all we're capable of in the real world.

What's the misconception? LLMs are probabilistic next-token prediction based on current context, right?

You're right, but also coding 10 years ago, 20 years ago, and 30 years ago looked very different to coding today in most cases. In every decade, we've abstracted out things that were critical and manual before. Are LLMs writing the code that much different than pulling libraries rather than rolling your own? Or automating memory management instead of manually holding and releasing? Or using if/else/for instead of building your own logic for jumping to a subroutine?

You don't think that it's easier to identify whether something is bubonic plague than it would be to identify whether two DNA samples are related? Especially when one of them is aged such that it's not even a whole sequence anymore?

Yes, our ability to manipulate and read DNA has increased significantly in the past 40 years. But you can't create data from something that isn't there or has been corrupted beyond recovery.

And as far as the efficacy of police goes, I don't think that a warrant is sufficient to prove that there's confirmed utility in getting these samples.

I agree with you in principle. And I agree that the government has been steadily eroding our privacy. And I agree that California probably shouldn't be retaining these samples forever.

However, the likelihood of being able to "sequence" the biomatter on a blood spot is quite low, and the probability of getting good signal out of it continues to go down over time. It'll still remain useful for various kinds of spot testing and genetic disease testing, but it's not going to produce a fully validated genomic sequence or even be that useful for forensic purposes.

This isn't some sort of sealed blood vial, it's literally just blood on paper.

That seems like a fair criticism. I don't know enough to quantify the benefit of retaining these samples, but I do know that the reason for keeping samples primarily relates to quality control, research, and development of tests.

There is a process for people to have the sample destroyed, I also have no idea how easy or how often that is used.