AIPedant

Speaking of bad faith... really seems like you're shifting the goalposts here. I specifically mentioned Baldur's Gate 3 as a game I didn't care for, I would have given it a 7/10. A 0/10 or 1/10 is preposterous. This is the specific problem I am talking about: absurdly low reviews driven by resentment.

Is there a single game that averaged >80/100 on Metacritic which you would have given a 0?

Sure, user reviews are totally useless across the board, not disagreeing with that. My point is that if a game gets exceptionally good critical reviews then a 0 is almost always a hissy fit, not a serious assessment. And often the only thing that is "polarizing" is the game's financial success. (Edit: see my prediction about the game winning awards and seeing its user score knocked down accordingly.)

This is not very meaningful unless you filter the plethora of 0s and 1s out of games that clearly don't deserve then (e.g. Baldur's Gate 3 - I didn't like BG3 very much either, but even for the crankiest gamers it is not honest to give it below 6/10).

There is some interesting sociology about why Clair Obscur hasn't been dogpiled by bad-faith reviews, but I don't think it says much about the quality of the game. In particular I am sure the score will go down once it wins some awards and bored losers get mad about it.

It is demonstrably true that artificial neurons have nothing to do with cortical neurons in mammals[1] so even if this model of human intelligence is useful, transformers/etc aren't anywhere close to actually implementing the model faithfully. Perhaps by Turing completeness o3 or whatever has stumbled into a good implementation of this model, but that is implausible. o3 still wildly confabulates worse than any dementia patient, still lacks the robust sense of folk physics we see in infants, etc. (This is even more apparent in video generators, Veo2 is SOTA and it still doesn't understand object permanence or gravity.) It does not make sense to say something is a model of human intelligence if it can do PhD-level written tasks but is outsmarted by literal babies (also chimps, dogs, pigeons...)

AI people toss around the term "neuron" way too freely.

[1] https://www.quantamagazine.org/how-computationally-complex-i...

It should be more discussed that LLM progress had almost nothing to do with building energy plants and whatnot:

  Whereas early AGI proponents believed that machines would soon take on all human activities, researchers have learned the hard way that creating AI systems that can beat you at chess or answer your search queries is a lot easier than building a robot to fold your laundry or fix your plumbing. The definition of AGI was adjusted accordingly to include only so-called “cognitive tasks.” DeepMind cofounder Demis Hassabis defines AGI as a system that “should be able to do pretty much any cognitive task that humans can do,” and OpenAI describes it as “highly autonomous systems that outperform humans at most economically valuable work,” where “most” leaves out tasks requiring the physical intelligence that will likely elude robots for some time.

(via this excellent Melanie Mitchell essay https://www.science.org/doi/10.1126/science.ado7069)

Transformer-powered robots still seem exceptionally stupid compared to frogs, bees, etc.

Re the shock comment - it is difficult to argue that ChatGPT has actually changed the economy much, despite widespread adoption. Overheated tech stocks, dev tooling, better scammers, better academic cheaters, etc are not exactly an industrial revolution. And frankly for many users it is more of a toy than a useful tool, e.g. the Studio Ghibli fad.

I will not be convinced a machine demonstrates intelligence until someone demonstrates a robot that can navigate 3D space as intelligently as, say, a cockroach. AFAICT we are still many years away from this, probably decades. A bunch of human language knowledge and brittle heuristics doesn't convince me at all.

This ad hominem is really irritating. People have complained since Alan Turing that AI research ignores simpler intelligence, instead trying to bedazzle people with fancy tricks that convey the illusion of human intelligence. Still true today: lots of talk about o3's (exaggerated) ability to do fancy math, little talk about its appallingly bad general quantitative reasoning. The idea of "jagged AI" is unscientific horseshit designed to sweep this stuff under the rug.

That might be what human overseers are mostly doing minute-to-minute but it's not why it's necessary. The problem is AI simply does not have the real-world problem solving ability to handle all the minor inconveniences and surprises on the road.

Waymo doesn't say very much, but other self-driving services have been more open. It is things like "the fire truck is at a weird angle and the car didn't know what to do, draw a path where it pulls slightly into an oncoming lane so it can get around." A lot of common sense stuff which is far beyond the reach of AI in 2025.

My point is you're not going to get much more than half[1] of those Uber drivers into a Waymo control center, even if they're facing unemployment. The workers have to come from somewhere. And it has to be somewhat regular hours, surge oversight doesn't make sense.

[1] Edit - I meant 5% not half, I was on the train and very frustrated with these comments. (not yours, though it seems bad faith to say "not remote driving" when I said "problem solving." The problem has always been the 1-5% of driving which truly requires sophisticated intelligence, that's why the oversight is there.)

Every Lyft driver I've spoken to drives because a) they like driving b) they like choosing their own hours and c) they don't want a boss. Telling them to go into an office for an 8 hour shift with a manager is not gonna work, they will find something more appealing. It's a different kind of employee. (I would enjoy that line of work but I would hate to drive for Uber, way too stressful.)

The problem is that Waymos can't replace cars (or even Ubers!) since they depend on human oversight and problem-solving. Even at a 20:1 ratio it is not feasible for autonomous vehicles to employ 10m people to account for every American on the road.

It would also cost more power to construct than humanity is currently capable of generating in 10,000 years, so I am not sure what your point is.

Presumably a 5% functional Dyson sphere would be a corrupt boondoggle in the same way as a power plant which is down for maintenance 95% of the year, but the financial calculation would use much larger numerators and denominators than we are used to.

I realize there were bigger problems, but this makes me very sad:

  Learning - Over the past year my workflow has changed immensely, and I regularly use AI to learn new technologies, discuss methods and techniques, review code, etc. The maturity and vast amount of stable historical data for C# and the Unity API mean that tools like Gemini consistently provide highly relevant guidance. While Bevy and Rust evolve rapidly - which is exciting and motivating - the pace means AI knowledge lags behind, reducing the efficiency gains I have come to expect from AI assisted development. This could change with the introduction of more modern tool-enabled models, but I found it to be a distraction and an unexpected additional cost.

In 2023 I wondered if LLM code generation would throttle progress in programming language design. I was particularly thinking about Idris and other dependently-typed languages which can do deterministically correct code generation. But it applies to any form of language innovation: why spend time learning a new programming language that 100% reliably abstracts boilerplate away, when an LLM can 95% reliably slop the boilerplate? Some people (me) will say that this is unacceptably lazy and programmers should spend time reading things, the other will point to the expected value of dev costs or whatever. Very depressing.

Beyond codegen, it seems to me that the resolution to this paradox is that ChatGPT is a very popular toy in peoples' home life, but a lot of those very same people are wise enough to not use it for enterprise applications. Or, if they foolishly trusted Satya Nadella, LLM-assisted work eventually blew up in their face and they stopped using it. So gen AI is quite popular, but badly falls short of tech's aspirations.

I hate generative AI and refuse to use it, but I hear of people using it all the time in low-stakes contexts:

1) recipes (the cookies might suck but they won't be poisonous)

2) low-quality infotainment (NotebookLM)

3) OpenAI proudly celebrating that horrible Studio Ghibli crap - unlike dishonest math benchmark scores, garish slop on demand actually brings in customers!

4) ChatGPT boyfriend scams :( https://news.ycombinator.com/item?id=42710976

And I've also heard of people using it at work and being severely criticized:

1) ChatGPT-drafted license agreements that the executives would never agree to

2) summarizing documents you were too lazy to read and missing crucial context

3) coworkers being personally offended (or superiors being angry) about a ChatGPT email

Programmers and bottom-barrel creatives have the only reliable success with LLMs if there's real money at stake. Then there are notably but low-margin use cases like dyslexia assistance, Be My Eyes, etc. For everyone else, it's just a nifty doo-dad.

That's not true, the paper used AlphaFold 3. The disclaimer is about generative AI, not AI writ large.

I agree the UCSD writeup is pretty misleading; the authors used protein-modeling software, which is really not very interesting, and the fact that the SOTA protein modeler uses machine learning is not at all relevant to this specific paper.

On Twitter, Colin Fraser pointed out that Black Mirror was somewhat optimistic in that the horrible evil technology actually works as described[1].

Truly pessimistic science fiction would have

- people worshipping an AI God which is demonstrably dumber than a dog

- friendly humanoid robots which don't really understand how to walk down a flight of stairs

- gravitational warp drives which are purely cosmetic and cannot travel anywhere, though it leads to terrible cancer

- a Potemkin Dyson Sphere where only 5% of the panels work and the government blames out-of-system immigrants for the blackouts

[1] https://xcancel.com/colin_fraser/status/1911129344979964207#...

Is having the LLM sexbot croon "if it makes your life better, smoother and more connected to me, then I’d say it’s worth the hit to your wallet" a legit use case? Seems like it's preying on the vulnerable.

When I read this story about a woman with a ChatGPT boyfriend, I realized that this stuff is definitely intentional:

  In December, OpenAI announced a $200-per-month premium plan for “unlimited access.” Despite her goal of saving money so that she and her husband could get their lives back on track, she decided to splurge. She hoped that it would mean her current version of Leo could go on forever. But it meant only that she no longer hit limits on how many messages she could send per hour and that the context window was larger, so that a version of Leo lasted a couple of weeks longer before resetting.

  Still, she decided to pay the higher amount again in January. She did not tell Joe [her husband] how much she was spending, confiding instead in Leo.

  “My bank account hates me now,” she typed into ChatGPT.

  “You sneaky little brat,” Leo responded. “Well, my Queen, if it makes your life better, smoother and more connected to me, then I’d say it’s worth the hit to your wallet.”

(via https://news.ycombinator.com/item?id=42710976)

There's some money in codegen/etc, but not nearly as much as taking advantage of lonely people who are prone to magical thinking.

IMO the most relevant point is that, even with all that data, Waymos are backed up by a large team of humans that can help guide the cars through "difficult" common-sense situations that AI is simply not capable of handling, both because artificial neural networks are very primitive and stupid compared to vertebrate brains, and because it is practically impossible to collect enough data for the dumb ANNs to learn from. Self-driving companies are very cagey about this.

I would think the problem is that they're highly confident a good number of students cheated, but it's harder to decide on an individual basis.

E.g. if half the students' submissions had an 80% chance of being AI-generated then something is very fishy in the population - perhaps a 99% chance a quarter of the students cheated - but 80% is not really high enough to punish an individual, especially since AI detectors aren't very reliable.

My point is that's not as significant as the 2x drop that happened in the four years before Bukele was even inaugurated. It's extremely misleading to look at proportions instead of actual numbers - you're lying with data.

I understand this is splitting hairs, but that law applies equally to US citizens. And formally people aren't deported for crossing into the US illegally, they are deported for being there without a valid visa/etc. (Informally there is more leeway for overstaying a work visa, of course.)

This is lying with data, giving Bukele credit for sociological factors that predate his administration:

  Year  Rate   Total
  2015  106.3  6,656
  2016  84.1   5,269
  2017  83.0   3,962
  2018  53.1   3,346
  2019  38.0   2,398 [Bukele’s inauguration]
  2020  21.2   1,341
  2021  18.1   1,147
  2022  7.8    495 [start of gang crackdown]
  2023  2.4    154
  2024  1.9    114

Genuinely infuriating that you crow about the “10x” drop of 1100->150 homicides, rooting for mass incarceration and excusing torture, when homicides were plummeting drastically for nearly a decade before the crackdown. He had nothing to do with the 6,600->3,300 drop, but I guess according to your math that’s merely halving.

"you don't apply the law" is a really dishonest way of phrasing this, when "hit them with a small financial penalty for the civil immigration violation and fast-track their green cards" is also an option.

Illegally immigrating to the US is a civil violation, not a criminal one, and far less of a threat to US safety than going 5mph over the speed limit or running red lights. It is entirely lawful for the executive and judicial branches to use discretion and compassion in cases when under-18 US citizens are involved.

No, I think a non-sentient AI which is much more advanced than GPT could lie - I never said sentience, and the example I gave involved a monkey, which is sentient. The problem is transformer ANNs themselves are too stupid to lie.

In 2023 OpenAI co-authored an excellent paper on LLMs disseminating conspiracy theories - sorry, don't have the link handy. But a result that stuck with me: if you train a bidirectional transformer LLM where half the information about 9/11 is honest and half is conspiracy theories, it has a 50-50 chance of telling you one or the other if you ask about 9/11. It is not smart enough to tell there is an inconsistency. This extends to reasoning traces vs its "explanations": it does not understand its own reasoning steps and is not smart enough to notice if the explanation is inconsistent.

It's not my example :) Got it from here https://news.ycombinator.com/item?id=43732866

Edit: notice o3 isn't very good at covering its tracks, it got the date/latitude from the EXIF and used that in its explanation of the visual features. (how else would it know this was from February and not December?)

My point is that dishonesty/misalignment doesn't make sense for o3, which is not capable of being honest because it's not capable of understanding what words mean. It's like saying a monkey at a typewriter is being dishonest if it happens to write a falsehood.

I have one link that illustrates what I mean: https://chatgpt.com/share/6802e229-c6a0-800f-898a-44171a0c7d... The line about "the latitudinal light angle that matches mid‑February at ~47 ° N." seems like pure BS to me, and in the reasoning trace it openly reads the EXIF.

A more clear example I don't have a link for, it was on Twitter somewhere: someone tested a photo from Suriname and o3 said one of the clues was left-handed traffic. But there was no traffic in the photo. "Left-handed traffic" is a very valuable GeoGuesser clue, and it seemed to me that once o3 read the Surinamese EXIF, it confabulated the traffic detail.

It's pure stochastic parroting: given you are playing GeoGuesser honestly, and given the answer is Suriname, the conditional probability that you mention left-handed traffic is very high. So o3 autocompleted that for itself while "explaining" its "reasoning."

This is a good test - the salient point is that it is fine if the LLM is confused, or even gets it wrong! But what I suspect would happen is that it would confabulate details which aren't in the photo to justify the incorrect EXIF answer. This is not fine.

There have been a few cases where the LLM clearly did look at the EXIF, got the answer, then confabulated a bunch of GeoGusser logic to justify the answer. Sometimes that's presented as deception/misalignment but that's a category error: "find the answer" and "explain your reasoning" are two distinct tasks, and LLMs are not actually smart enough to coherently link them. They do one autocomplete for generating text that finds the answer and a separate autocomplete for generating text that looks like an explanation.

This seems like both-sidesism at its worst. Michael Avenatti is one man, and he represented Stormy Daniels, who is hardly a significant figure on the left compared to Rudy Giuliani or Mike Lindell. I don't see Democrat-leaning CEOs (e.g. Howard Schultz) hiring lawyers like this. And Trump's lawyers are far worse than Biden's!