Hacker News Reader

To have a good mental model for modern AI agents you have to understand both the LLM and the other stuff that's built up around it. OP is correct about the behavior of LLMs, and that is valuable information to keep in mind. Then you layer on top of that an understanding that some implementations of agents will sometimes automatically feed search results into context, if you ask them to or are paying for an advanced tier or whatever the extra qualifications are for your particular tool.

If you skip this two-part understanding then you run the risk of missing when the agent decided not to do a search for some reason and is therefore entirely dependent on statistical probability in the training data. I've personally seen people without this mental model take an LLM at its word when it was wrong because they'd gotten used to it looking things up for them.

Defending TFA a little ... the rest of the article builds up context around the word "Completely", so that the single example of "Highway zipping" is not just what is being discussed.

"Completely" here should be expanded to include all the unique and unforseen circumstances a driver might encounter, such as a policeman directing traffic manually or any other "soft" situation that is not well represented in training.

Not to mention the somewhat extreme amount of apriori and continuous mapping that goes into operating a fleet of AVs. That is hardly to be considered "Completely autonomous".

This isn't just pedantry, the disconnect between a technical person's deep understanding and a common user's everyday experience is pretty much what the article hinges on. Try taking a Waymo from SF to NYC. This seems like something a "Completely autonomous" car should be able to do given a layperson's understanding of "Completely", without the experts' long list of caveats.

> For example, while researching new monitors today, I told Gemini...

You told an agent, not just an LLM.

> And this is already not hype, it's reality anywhere Waymo operates.

Some beg to differ; see e.g. https://www.youtube.com/watch?v=040ejWnFkj0 .

> For example, while researching new monitors today, I told Gemini to compare $NEW_MODEL_1 with $NEW_MODEL_2.

But this feature was a staple of most online shops that sell monitors and a bunch of "review" sites. You don't need a highly complex system to compare 2 monitors, you need a spreadsheet.

Waymo only works because it's geofenced — that's a massive barrier to "completely autonomous" (or level 5 automation)

waymo is pretty amazing but it's still short of complete full self driving, and it's not at all clear that it can close the gap. similary, current LLMs are really remarkable. but it's not at all clear that they will make the leap into "real" intelligence.

> what LLMs do is string together words in a statistically highly probable manner.

I guess the article fails to admit that when you have billions of connected points in a vector space, "stringing together" is not simply "stringing together". I'm not a fanboy but somehow GPT/attention based logic is capable of parsing input and data then remodeling it in depths that are surprising.

Waymo operates on highly controlled and mapped environments. Can they handle Rome or Mumbai?

Describing SF's Tenderloin at night as a "highly controlled environment" would be stretching it.

Anyway, you're moving the goalposts here. Waymo is operating at scale in actual human cities in actual rush hour traffic. Sure, it would struggle in Buffalo during a snowstorm or in Mumbai during the monsoon, but so do human drivers.

> Sure, it would struggle in Buffalo during a snowstorm or in Mumbai during the monsoon, but so do human drivers.

We don't expect technology to be on par with human capabilities but to exceed them.

This feels a lot like "moving the goalposts." First, it was complete science fiction to have technology in the car. Then, it was in the car, but it could only do navigation and music, it can't operate the car the way humans can. Then, it can prevent you from weaving out of your lane, and it can stop the car if you're about to crash into something, but it can't help you with your commute. Then, it can speed up, slow down, and steer on the highway, but it can't take you door to door. Now, it can take you door to door, but only in certain environments, it can't do it everywhere.

All of the above happened over the last ~20 year or so. The progression clearly seems to point to this being more than hype, even if it takes us longer to realize than originally anticipated.

Not so much moving the goalposts as pointing out that playing American football is not like playing soccer (football - that is, driving in Rome) or even cricket (Mumbai).

In fact, cricket doesn't even _have_ goalposts, it has wickets. Driving in cities outside North America is very different.

> Driving in cities outside North America is very different.

Waymo is testing in Japan: https://waymo.com/blog/2025/04/new-beginnings-in-japan

I'm not sure I get your analogy here. If you're suggesting that it's not "moving the goalposts" because it's pointing out that driving in Rome or Mumbai is different than driving in North America, then that is exactly what is meant by moving the goalposts.

10 years ago the claim was that "cars can't drive autonomously," Waymo quietly chips away to the point that they absolutely can drive autonomously, even in an unpredictable environment (with evidently drastically lower-than-human accident rates, for example), and the reaction of those original people is to say "yeah but it can't drive in [even more complex place]"

Sure, that's not exactly surprising. We generally don't design technology to do the most complex version of the task it's supposed to do first. We generally start with a simpler scenario it can accomplish and progressively enhance it as we learn more. Cars have been doing that for decades.

So perhaps the tech doesn't work in Mumbai or Rome yet. Maybe we'll advance the tech to do that thing, or maybe we'll come up with a different solution to autonomous driving in these places if we find out it'll be more expensive to advance this technology than it will be to do something else instead. But either way, it's already doing the thing that many, many people claimed it can't do, and those people are now claiming there's something else it can't do. That is the very definition of moving the goalposts.

> Waymo quietly chips away (..)

Perfect example of the saying: "if you have a big problem, first solve the smaller problems. Then your bigger problem may turn out to be not so big after all".

Current AI is much like that: one 'little' problem after another being solved (or at least, progressing).

It's not really moving the goalposts, though. The idea of a self driving car has always been "I can get in my car, tell it where I want to go, and then it goes there while I read a book."

Having navigation and music, and lane assist, and adaptive cruise control, and some cars that can operate autonomously in some environments is great, but it's not what we meant when we said self driving cars.

The point is not that those things were meant when we said self driving cars. It's that, at every step along the way, there were a group of people who doubted that cars could do that thing, and then they did that thing. And then the thing we said they can't do changed to something else.

Today, you absolutely can "get in a car, tell it where you want to go, and it goes there while you read a book" - it's literally what Waymo is and has been doing. And now we're saying it can't do it in Mumbai, so it's still not self-driving.

At some point, the distinction seems pointless. We are undeniably continuing to make progress on the road to autonomous driving, and it does work in certain scenarios today. To suggest things are slowing down because we haven't met the most reason interpretation of the words is neither helpful nor correct.

> It's that, at every step along the way, there were a group of people who doubted that cars could do that thing

...Can you cite that?

> And then the thing we said they can't do changed to something else.

...And they were the same people?

> We are undeniably continuing to make progress

Where did anyone deny this?

> To suggest things are slowing down

Where did anyone make this argument?

The quote from TFA:

> but the hype has promised completely autonomous cars reliably zipping about in rush hour traffic.

The author did not restrict that to SF, and is presumably referring to "hype" that "promised" this globally.

You conveniently left out the first part of the sentence you quoted:

> Currently available software may very well make human drivers both more comfortable and safe...

Which is objectively not what Waymo does, and whether intentional or not, invalidates the progress that has been made.

Also, immediately preceding that:

> Driverless vehicles in closed systems have been in use for a long time.

Which is also not what current frontier self driving technology is.

> Where did anyone make this argument?

The title of the article is quite literally "Is Winter Coming?"

Why is that the standard? I, a human, can’t handle driving in Mumbai.

And lol at anyone who thinks any urban driving environment is “highly controlled”.

> Why is that the standard? I, a human, can’t handle driving in Mumbai.

Aren’t you confusing “navigating” vs “driving”?

In Indian traffic, navigating is the least of your worries.

The Western mind simply cannot fathom what some of our Eastern brothers and sisters have managed to achieve on the roads.

I remember the first time I went to visit my father in Kathmandu what his address was, and he patiently explained to me the street he lived on simply had no name or unique identifier. Or driving for the first time in Vietnam and being inducted into a traffic system where your only responsibility is the cone in front of you of things you can see. Or the terror of realizing that your second taxi driver of the day in Bangkok is literally on speed.

All this to say: no, I assume he means he can’t handle driving in Mumbai.

I, a human who learned to drive in Mumbai, can't handle driving in Mumbai anymore.

some things in "Car-culture" that surprised me on my trips to india;

1. People do not as a matter of their daily complaints complain about bad traffic, bad drivers, dents, door dings etc. 2. There are less accidents per capita than US 3. Insurance is required, body shops work better than in US. 4. Electrification of tuk-tuk fleet is....impressive.

Waymo/Autonomos driving would drastically slow down most of transporation infrastructure in most of the world. I don't think waymo should spend billions figuirng out how to drive better than Indians.

Have you been to San Francisco?

Compared to driving in the developing world though, SF traffic is very structured and very tame.

My favorite data point here is Cairo: the sound of traffic there is horns blaring and metal-on-metal. Driving in Cario is a contact sport. And it doesn't seem to matter how nice a car is: a fancy Mercedes will have as many body dents as a rust bucket Lada.

Waymo works where it works and it's useful where it works. Can a Mumbai autorickshaw handle an American freeway? Does that make it a pointless vehicle?