keeda

>I can play that game too. The JPEG process extracts perceptual shorthand from information and encodes that into "quantized coefficients". These "quantized coefficients" do absolutely nothing until you run them through a "reconstituter".

But the "quantized coefficients" represent only color information of pixels, where are the abstract concepts of the things depicted in the image? I mean, you can't just dismiss salient details as "high-tech jargon" and replace it with completely unrelated alternatives ¯\_(ツ)_/¯

And I do mean abstract concepts. E.g. if it's a picture of a cat, do any of the quantized coefficients map to the concept of a cat, in that they can identify or regenerate any cat regardless of the infinite variety of pixels that can depict an infinite variety of cats?

You know, like they found artificial neurons to recognize arbitrary cats in arbitrary pictures in this research from 2012, which was a precursor to what LLMs are doing today: https://research.google/pubs/building-high-level-features-us... -- that represents the abstract concept of "cat."

Note that it's not even "high-tech jargon" it's very much a plain english description of what is happening inside LLMs. The rest of your comment would be addressed once you internalize that difference ;-)

Today we have a tiny fraction (like 2%?) of the global supply needed to make enough robots to replace all humans.

This is before we get to the other logistics like power and the rest of the materials, but we have a path to scale those up (except maybe Copper, that's another tough one.) Turns out rare earth elements are actually very abundant, but extremely hard to mine, very toxic to extract (which is partially why China has a strangehold on those) and we don't have a good path to scale those up.

All of this could be moot if we figure out alternative technologies for equally capable motors that could be manufactured at scale, but AFAIK whatever promising is on the horizon would still take a very long time to ramp up.

In that context, discussing whether TS or JS are strongly typed -- especially when TS is a straightforward translation to JS -- is a diversion from the main point that LLMs have already demonstrated something much, much harder.

Nobody can say the same about LLMs, because nobody has managed to decipher them yet. This is an active area of research (Mechanistic Interpretability.)

Your post is conflating lossy discarding of information with extracting abstract concepts from information and encoding them into weights. This is why those weights do absolutely nothing until you run a prompt through them. On the other hand, obviously any media file can be decoded by itself without needing a prompt.

Those blocky compression artifacts you see are not hallucinations, they are the image viewer just filling in for missing details. You don't even need lossy compression for this; take a lossless image like a BMP and zoom in, you'll see those blocks again!

Interestingly, image viewers typically do this via interpolation of adjacent pixels, but a lot of hallucinations are actually the result of extrapolation, the opposite mechanism. Which is why LLMs can create output that was NEVER in any input data anywhere (hence the term "hallucination"!)

And if you think lossy compression is sufficient to avoid the "verbatim" requirement of copyright claims, you're welcome to explore the legality of selling transcoded versions of copyrighted content ;-)

I'm not sure what those links are in relation to?

LLMs do, and they encode so many concepts and the relationships between them into so many weights that it is a literally incomprehensible blob of floats. They are not just a storage format, and there is no way to recover the original content verbatim from these weights, except for a very small handful of extremely popular works like Harry Potter.

As another example, JPEG will discard details from the original image, and when decoded, will display missing details as blocks. But it will never hallucinate output that never existed in the input data. Like, a JPEG of a cat can never randomly be decoded into an image of a dog.

While he's never been dismissive of the technology itself, it is fascinating to see this kind of work from him even as he rails at all the hype around AI.

https://intelligence.exponentialview.co/

Another thing to consider is that most of the 3T CapEx spend is from hyperscalers free cash flow, and the debt is a smaller (but fast-growing) fraction. Napkin math suggests if all AI CapEx is written off today, hyperscalers could cover their debts in about 5-6 years using pre-AI levels of free cash flow.

Maybe Nvidia is not sounding the safety drumbeat because it stands directly to lose out if demand from training slows down ;-)

In the above description, at no point does the actual verbatim content exist anywhere in the model, and copyright law being about rights to reproducing copies (verbatim or substantial portions thereof) does not really apply and does not need any exceptions or qualifications. (If you’re thinking of regurgitation, you should look into studies about it to see how vanishingly rare it is.)

https://georgzoeller.com/blog/posts/what-reverse-engineering...

If you look for actual evidence of AI replacing authors the evidence is thin. One study found no real impact, and another found authors using AI (not AI itself!) competing with other authors and applying downward pressure on sales through increased competition. I have seen no evidence yet of AI replacing authors directly, and this is probably their biggest challenge.

But these sound bites sure sound damning though!

I think courts will consider them, but I suspect it will weigh legal doctrines like Fair Use and actual economic studies more than these statements to determine infringement. These statements will likely matter more after a finding of infringement to show willfulness, and hence the damages calculation, if any. IANAL, so a real lawyer should keep me honest here.

In case you mistake him for an “AI booster”, you should see this guy’s other posts, he is extremely critical of the AI frontier labs and Tech Bros and this whole AI bubble.

But he is also a very experienced game developer, and under his control the latest models have pulled off these crazy feats. What TFA is the analysis of a domain expert and what it means for the industry.

https://georgzoeller.com/blog/posts/what-reverse-engineering...

He also did this thing where he “spliced” different game concepts and turned WoW into an FPS: https://news.ycombinator.com/item?id=49652089

In case you mistake him for an “AI booster”, you should see this guy’s other posts, he is extremely critical of the AI frontier labs and Tech Bros and this whole AI bubble.

But he is also a very experienced game developer, and under his control the latest models have pulled off these crazy feats. What you are reading is the analysis of a domain expert and what it means for the industry.

If this VUI (Vibed User Interface) future comes about it entirely dismantles Apple's (and Android's) App Store moat. From a cynical perspective this may be why Apple may be shifting their strategy to enshittification: https://techcrunch.com/2026/09/06/phil-schillers-app-store-e...

Google has seen the future, and like their strategy with search (but unlike Apple, which continues blocking and suing "everything" app makers) are willing to disrupt themselves: https://techcrunch.com/2026/05/12/googles-create-my-widget-f...

However, if the slew of AI wearable or "companion" devices coming out are any indication -- from the failed Humane Pin, to Meta's keychain and glasses, to Microsoft's office badge (which OMG is so on-point in its enterpriseyness) to whatever Jonny Ive is cooking up at OpenAI -- the smartphone form factor may not be the eventual one we settle on for AI. Something to keep in mind when designing the OS.

Hah, shots fired! https://steve-yegge.medium.com/why-i-left-google-to-join-gra... (Which to be fair was a prescient critique of Google too.)

But 8 is not too early to learn programming as such, because my kids started around that age. Only they started creating dumb games using Scratch in a modern browser on a full-fledged modern OS. The environment you start with probably matters a lot for how much you take to it.

I think the company that cracks the mass market robotics puzzle will be the next trillion (quadrillion?) dollar company. I suspect that would require bypassing the dependency on rare earths, but there seem to be no serious contenders out there? All the big shots are focusing on using current technologies because they basically “solve” motors. Maybe the company that does crack this puzzle will come out of left field with an unconventional technology that scales with whatever raw material is abundant, like OpenAI did with ChatGPT and compute.

By some estimates, today's global supply is like 2% (or was it 0.2%?) of the total amount required to enable such widespread deployment of robots. Until we can scale that up (or figure out how to bypass that dependency) there is a very hard constraint on how fast they can be deployed.

Maybe not you personally, but many, many other people have killed many, many pedestrians, and when they exhibit a pattern of bad driving -- or other deviant behavior -- we take them off the streets. That is an example of society being robust.

Except, over here, we're rushing to make AI, which we know has many deviant behaviors and we know caused harms in many circumstances (starting with AI-assisted suicides), even more powerful AND deploy it in more and more real world systems!

As history and, literally, current events show us again and again, society has failure modes that lead to widespread harm and destruction. We've had world wars and then literally had multiple close calls with nuclear war right after. And then we have all that's going on out there. (In related news, the Pentagon threw a hissy fit because Anthropic would not let them use Claude for autonomous killing machines. Do I have to even mention what they've been up to these days?)

Most of these failures are caused by misaligned incentives and socioeconomic forces. The incentives and forces around AI have hints of many brand new failure modes that we can't even foresee because things are moving so fast.

> Then why argue for setting a pace for the frontier labs if we've already surpassed WOPR level integration / intelligence.

"We're already going down this mountain pass at 200 miles an hour, why slow down now?"

I think we should not just slow down frontier AI development, we should also slow down where and how that AI gets deployed.

A lot of my comments get responses that assume I'm an "AI booster" but what I really am is someone highly motivated to understand the capabilities and limits of this technology, because it is already very directly impacting the future of my kids.

What I tell my kids is a function of what I believe the future holds: nobody will be doing low-level cognitive work anymore, it will all be done by AI, so our job is to be better than AI. Unfortunately:

a) Low-level cognitive work is like (SWAG) 80% of all knowledge work, and so more than 80% of jobs are effectively gone, they just don't know it yet.

b) AI is getting really good really fast so that 80% is also growing quickly.

That means that the days of doing a junior-to-mid level job, kind of just coasting along -- often until retirement -- are gone. Everybody has to be good enough to be doing the top 20% of cognitive work, which involves fuzzy things we call innovation, expertise, domain knowledge, intuition, taste, handling uncertainty, and the like.

So my advice to my kids is:

1. Whatever you do, be the BEST at it, regardless of grades. Grades already meant little, now they mean even less. Being the best likely also necessitates strategic avoidance of AI, but I'm willing to be proven wrong on that.

2. Be "high agency". Annoying buzzword but perfectly captures the trait everybody will be hiring for. The 80% of the jobs were those doing the drudgework for the 20% "high agency" people calling the shots. And those are the exact jobs that are gone. Just do things, because now you can.

3. Critical thinking may soon be the only job requirement remaining. Get bullet-proof at that and learn relevant skills (maybe data literacy?)

I'm also encouraging trade school, but that seems less attractive to the kids.

I'm not saying skip college just yet, but we do intend to aggressively "value-maxx" college education (e.g. community college) and focus on building something, anything, practical -- since showing what you've built is better than degrees from most colleges.

Case in point, that hallucination (which, if it was an evil superintelligence, could have been "strategic") that almost led to US boarding a Chinese ship over suspicions of nuclear weapons: https://www.msn.com/en-gb/news/other/us-military-ai-failure-...

It doesn't have SkyNet, it just has to be WOPR. And it doesn't have to be people who are motivated to cause harm, it just has to be people making consequential decisions who are careless, distracted, paranoid or anxious -- which everybody is at some point or another.

This raises many intriguing questions:

- Does that mean we have created AI in God's image?

- Or was their "misalignment" between God and us like there seems to be between us and AI?

- Did God cast man out of Eden because he was worried we'd destroy heaven?

- Is the Bible our system prompt or a user prompt?

- Is life an eval to vet whether individual agents err people can contribute to his tasks without going rogue and destroying heaven?

- Are harmful human behaviors in the name of religion unintended "reward hacking"?

- Does this explain the rather weird relationship between God that requires getting twisted into logical pretzels to rationalize?

- Did God go through this crisis himself? ;-)

I'm sure there are many more similar lines of inquiry but I'm just not familiar enough with Christianity to make the connections. Would be fun to read an expert thesis that explores this parallel!