Hacker News Reader

Comment by ben_w | Hacker News Reader

> Is nobody in these very rich guys' spheres pushing back on their thought process?

Yes, frequently and loudly.

When Altman was collecting the award at Cambridge the other year, protesters dropped in on the after-award public talk/Q&A session, and he actively empathised with the protestors.

> So far we are multiple years in with much investment and little return, and no obvious large-scale product-market fit, much less a superintelligence.

I just got back from an Indian restaurant in the middle of Berlin, and the table next to me I overheard a daughter talking to her mother about ChatGPT and KI (Künstliche Intelligenz, the German for AI).

The product market fit is fantastic. This isn't the first time I've heard random strangers discussing it in public.

What's not obvious is how to monetise it. Old meme parroted around was "has no moat", which IMO is like saying Microsoft has no moat for spreadsheets: sure, anyone can make the core tech, and sure we don't know who is Microsoft vs StarOffice vs ClarisWorks vs Google Docs, but there's more than zero moat. From what I've seen, if OpenAI didn't develop new products, they'd be making enough to be profitable, but it's a Red Queen race to remain worth paying for.

As for "much less a superintelligence": even the current models meet every definition of "very smart" I had while growing up, despite their errors. As an adult, I'd still call them book-smart if not abstractly smart. Students or recent graduates, but not wise enough to know their limits and be cautious.

For current standards of what intelligence means, we'd better hope we don't get ASI in the next decade or two, because if and when that happens then "humans need not apply" — and by extension, foundational assumptions of economics may just stop holding true.

View on HN

> When Altman was collecting the award at Cambridge the other year, protesters dropped in on the after-award public talk/Q&A session, and he actively empathised with the protestors.

He always does that to give himself cover, but he has clearly shown that his words mean very little in this regard. He always dodges criticism. He used to talk about the importance of him being accountable to the OpenAI board and them being able to fire him if necessary when people were questioning the dangers of having one person have this much control over something as big as bleeding edge AI. He also used to mention how he had no direct financial interests in the company since he had no equity.

Then the board did fire him. What happened next? He came back, the board is gone, he now openly has complete control over OpenAI, and they have given him a potentially huge equity package. I really don't think Sam Altman is particularly trustworthy. He will say whatever he needs to say to get what he wants.

Wasn't he fired for questionable reasons? I thought everyone wanted him back, and that's why he was able to return. It was, as I remember, just the board that wanted him out.

I imagine if he was doing something truly nefarious, opinions might have been different, but I have no idea what kind of cult of personality he has at that company, so I might be wrong here.

> I thought everyone wanted him back, and that's why he was able to return.

Everyone working at OpenAI wanted him back. Which only includes people who have a significant motivation to see OpenAI succeed financially.

Also, there are rumours he can be vindictive. For all I know, that might be a smear campaign. But if that were the case, and half the people at OpenAI wanted him back, the other half would have a motivation to follow so as not to get whatever punishment from Sam.

It sounds to me people working in AI these days have a lot more options than being fraid of a particularly vindictive man.

Between just working on ML tech and working on ML tech with US$XXX billion of Microsoft’s money—salaries aside—imaginably, having access to nuclear plants worth of energy, compute power and cutting edge GPUs, legal department for defending intellectual property violation, etc., makes for a very different value proposition compared to some other startup in the space. (The self-image of working at a company allegedly “advancing humanity” likely helps, too.)

True to some extend. But if you have OpenAI on your resume I have to imagine there’s no necessity for your next job to be much different (e.g. go to any FAANG)

I think (based on very shallow research) among FAANG behemoths Meta is the closest in terms of resources thrown at ML, but it’s an ad company that hasn’t got such a “we advance humanity” image. Equity vesting and that sort of stuff can additionally make moving companies a problematic prospect, even if the new place offered a competitive salary…

Yeah, Meta pays well, but they probably wouldn't match the $10M equity pile that some generic engineer or researcher who joined 1 year before ChatGPT dropped could be sitting on. Ofc, the equity->cash conversion rate for OpenAI pseudo stocks isn't clear, but those paper gains would be tough to abandon.

> Wasn't he fired for questionable reasons?

There were a number of concerns, including for-profit projects he was pursuing despite his public insistence on OpenAI being non-profit as well as generally deceptive behaviour on his part. The last part at least is consistent with what others have said about Altman previously, including what allegedly led to his exit from YC, although they have kept those stories pretty quiet. But it seems like PG himself no longer has a lot of trust in Altman after he basically made him his heir apparent, and he has known him for a while now.

What's more, the driving force behind the board's move to remove him from OpenAI was reportedly Ilya Sutskever who was one of their key people and one of the handful of original founders. So it wasn't just a bunch of external people who wanted Altman gone, but at least one high level insider who has been around for the entire thing.

Altman himself was even once asked by journalists whether we could trust him to which he outright answered "no". But then he pointed to the fact that the board could remove him for any reason. Clearly, he was suggesting that there were strong safe guards in place to keep him in check, but it is now apparent that those safe guards never really existed while his answer still stands.

Don’t pay much particular attention to AI but I’m not seeing the knock on him saying “no.”

What human in the world could be trusted with AI? Only delusional people could say yes to that question.

> What human in the world could be trusted with AI? Only delusional people could say yes to that question.

"AI" is too broad a topic to draw that conclusion.

Almost everyone can be trusted with Stockfish. Almost: https://github.com/RonSijm/ButtFish

Most of us can be trusted with current LLMs, despite the nefarious purposes that they can be put to by a minority. Spam and fraud are still minority actors, even though these tools increase the capabilities of those actors; and they're still "only" at the student/recent graduate level for many tasks, so using them for hazardous chemical synthesis will likely literally explode in the face of the person attempting it.

Face recognition AI (and robot control attached to it) is already more than capable enough to violate the 1980 Protocol on Blinding Laser Weapons, and we're just lucky random lone-wolf malicious actors have not yet tried to exploit this: https://en.wikipedia.org/wiki/Protocol_on_Blinding_Laser_Wea...

We don't yet, it seems, have an AI capable enough that a random person wishing for world peace will get a world with the peace of the grave, nor will someone asking for "as many paperclips as possible" accidentally von-Neumann the universe into nanoscale paperclip-shaped molecules — indeed, I would argue that if natural language understanding continues in the current path, we won't ever get that kind of "it did what I said but not what I meant" scenario; what I currently think most likely is as an analogy for the industrial revolution, where we knew CO2 was a greenhouse gas around the turn of the 1900s, and still have not yet stopped emitting it because most people prefer the immediate results of things that emit it over the distant benefits of not emitting it.

But even that kind of AI doesn't seem to be here just yet. It may turn out that GenAI videos, images, text, and voice is in fact as much of a risk as CO2, that it collapses any possibility of large-scale cooperation, but for the moment that's not clear.

a smartphone can be used to guide an ICBM.

what human in the would could be trusted with it?

A road can be used for abductions, acts of violence, and other harmful activities.

What human in the world could be trusted with it [roads]?

Most humans are frequently harming themselves (and sometimes harming others) with their smartphones, so...

> I thought everyone wanted him back,

Ilyia Sutskever who was the chief Scientist of the company and honestly irreplacable in terms of AI knowledge left after Altman returned.

Only 1 of the 6 board members are still at OAI.

He also apologized and said he was wrong.

That was only after it was apparent that a majority of employees would back Altman's return I believe. A majority of which who had spent less time with him than Ilya had in all likelihood.

Define everyone. I was delighted when they fired him. I don't believe he has humanity's best interest at heart.

745 of 770 employees responded on a weekend to call for his return, threatened mass resignations if the board did not resign; among the signatories was board member Sutskever, who defected from the board and publicly apologized for his participation in the board's previous actions.

>745 of 700 employees responded on a weekend to call for his return, threatened mass resignations if the board did not resign

I would think final count doesn’t really matter. Self serving cowards, like me, would sign it once they see the way the wind was blowing. How many signed it before Satya at Microsoft indicated support for Altman?

I followed the drama. The point I was (somewhat unsuccessfully) trying to make was that while, sure, there were groups who wanted him back (mainly the groups with vested financial interests and associated leverage), my sense was that the way it played out was not necessarily in line with wider humanity's best interest, i.e. as would have been hoped based on OpenAI's publicly stated goals.

Oh, in that case sure.

The statements in the whole popcorn drama still don't add up in my mind, so from the point of view of "humanity's best interest", I'd say it's still bad.

I thought you meant at the time, not with the benefit of hindsight.

>my sense was that the way it played out was not necessarily in line with wider humanity's best interest,

Sure. But you make the foolish assumption here that humanity even has humanity's best interests at heart. Sentiment may be negative on current LLM generative based AI, but there's still plenty of people with either potential vested interests or simply seeing missing the forest for the tree. It's pretty hard to say "everyone hates/loves AI" at this current time.

I'd do that too if I held stock, and I think the guy is a borderline-vampire.

745 of 700?

Whoops, typo for 770, edited to correct. Thanks!

* 747/770 https://www.nasdaq.com/articles/747-of-770-openai-employees-...

At the time, was it possible for people working at OpenAI to, er, "cash out"?

I don't actually know the answer to that, but I'm suggesting that perhaps people had additional motives for the organization to preserve its current hype/investment growth strategies.

He was fired because he was leading the for profit in a direction that was contrary to the mission of the non profit parent that was supposed to be controlling the for profit.

Those working for the for profit got $$$ in their eyes once MSFT started throwing billions at them and said fuck the mission and bring back Altman we’re going full Gordon Gekko now.

This guy is outright scary. He gives me the chills.

the whole group of these techbro billionaires come off as barely human ghouls when you hear them speak, imo

see also: peter thiel on piers morgan recently.

They probably know that nobody would vote for them which is why they fund Trump and other politicians who do have social skills.

Indeed. Words are very very inexpensive and fool a lot of people. Never pay attention to any words. Judge people by their actions.

> The product market fit is fantastic. This isn't the first time I've heard random strangers discussing it in public.

Hardly the evidence of PMF. There is always something new in the zeitgeist, that every one is talking about, some more so than others .

2 years before it was VR, few years before that NFTs and blockchain everything, before that it was self driving cars before that personal voice assistants like Siri and so on .

- self driving has not transformed us into minority report and despite how far it has come it cannot in next 30 years be ubiquitous, even if the l5 magic tech exists today in every new car sold it will take 15 years for current cars to lifecycle.

- Crypto has not replaced fiat currency , even in most generous reading you can see it as store of value like gold or whatever useless baubles people assign arbitrary value to, but has no traction for 3 out of other 4 key functions of money .

- VR is not transformative to every day life and is 5 fundamental breakthroughs away.

- Voice assistants are useless setting alarms and selecting music 10 years in.

There has been meaningful and measurable in each of these fields, but none of them have met the high bar of world transforming .

AI is aiming for much higher bar of singularity and consciousness. Just in every hype cycles we are in peak of inflated expectations, we will reach a plateau of productivity where it is will be useful in specific areas (as it already is) and people will move on to the next fad.

> 2 years before it was VR, few years before that NFTs and blockchain everything, before that it was self driving cars before that personal voice assistants like Siri and so on .

I never saw people talking about VR in public, nor NFTs, and the closest I got to seeing blockchain in public were adverts, not hearing random people around me chatting about it. The only people I ever saw in real life talking about self-driving cars were the ones I was talking to myself, and everyone else was dismissive of them. Voice assistants were mainly mocked from day 1, with the Alex advert being re-dubbed as a dystopian nightmare.

> AI is aiming for much higher bar of singularity and consciousness.

No, it's aiming to be economically useful.

"The singularity" is what a lot of people think is an automatic consequence of being able to solve tasks related to AI; me, I think that's how we sustained Moore's Law so long (computers designing computers, you can't place a billion transistors by hand, but even if you could the scale is now well into the zone where quantum tunnelling has to be accounted for in the design), and that "singularities" are a sign something is wrong with the model.

"Consciousness" has 40 definitions, and is therefore not even a meaningful target.

> Just in every hype cycles we are in peak of inflated expectations, we will reach a plateau of productivity where it is will be useful in specific areas (as it already is) and people will move on to the next fad.

In that at least, we agree.

> never saw people talking about VR in public, nor NFTs, and the closest I got to seeing blockchain in public were adverts, not hearing random people around me chatting about it. The only people I ever saw in real life talking about self-driving cars were the ones I was talking to myself, and everyone else was dismissive of them. Voice assistants were mainly mocked from day 1, with the Alex advert being re-dubbed as a dystopian nightmare.

Isn't that funny? It speaks exactly to the GP's point about how frothing and uninformative the "zeitgeist" can be, as my experience happens to have been the opposite of yours. The people I happened to be casually brushing around during those earlier fads happened to be hip to them, or maybe my ears were more attentive, and so I heard them mentioned more often and with more enthusiasm.

In contrast, most of what I happen to hear about generative AI, outside of HN, tends to be idly laughing at its foibles and misrepresentation if it's mentioned at all.

I don't expect you to have the same experience as me, but I'm careful not to assume too much based on either of ours.

I mean nobody needs to bother with anecdotes here. chatgpt.com hit 3.7B visits in October and was #8 in worldwide internet traffic. Open AI say they have 300M weekly active users and 1B messages per day.

The idea that it's some fad with no utility or that the general public knows little about it is at this point laughable. It's the software product with by far the fastest adoption any software product has ever seen.

And if people want to lump that in with bitcoin or VR or whatever, shrug, i just don't know what to say.

Yes, because it is free.

How many paid users? Is a paid user generating profit right now? Google had to inject advertising in its search results to start earning real money. What will OpenAI do? That's the PMF.

There are millions of free apps.

They had some estimated 10M paid users a few months ago.

>Is a paid user generating profit right now?

For themselves ? I don't know and neither does the person saying they have no PMF.

There is nothing stopping Open AI from inserting ads into GPT's responses, either explicitly with the search tool or weaved into predicted responses.

Their revenue nearly quadrupled year on year and model training and inference costs have reduced by several orders of magnitude in the last few years.

I'm not saying that Open AI are foolproof. But the money being pumped to them is more or less expected.

90% of my use of OpenAI is to avoid the crap of internet search (Google or ddg, even when I find the right page finding the specific part of a pain)

At some point chatgpt will become crap and the cycle will repeat

That actual isn't evidence at all that the general public knows a lot about it, or even uses it much. It can be a small group of very active users.

To put it frankly, not at this scale.

I don't know what you think nearly 4B visits per month is (and top ten in worldwide traffic) that "a small group of active users" can generate. 300M active users a week is not a small group and you do not get there without general public awareness.

> I mean nobody needs to bother with anecdotes here. chatgpt.com hit 3.7B visits in October and was #8 in worldwide internet traffic. Open AI say they have 300M weekly active users and 1B messages per day.

As far as the "zeitgeist" conversation goes, though, there's one story in the non-tech news about AI right now and it isn't how it's making everyone's life easier. It's, yet again, a story about how someone trusted it and got burned because the markov chain landed on something not true.

> ... and that "singularities" are a sign something is wrong with the model.

Einstein thought that way too because his model predicted singularities in space-time fabric. Turns out he was wrong. Nowadays we call them blackholes and have images of them too.

He wasn’t exactly wrong in that the model still breaks down for black holes. Even though we know they exist we don’t know what goes inside them.

Black holes have had two different singularities in general relativity:

(1) the event horizon was the first singularity, but that turned out to be a free choice of the coordinate system, i.e. a mistake in the maths — and we can't see past the event horizons anyway, so we don't know what happens on the other side of them even though they're not "real" singularities

(2) the point in the centre where the geodesics stop, which everyone in the field knows is incompatible with (a) the assumption in the maths of relativity that spacetime is smooth and can be differentiated everywhere, and (b) quantum mechanics

The image is not of the singularity itself but the accretion disc.

This is not to discount that amazing accomplishment but to point out that, in addition to misrepresenting what the singularity is in a black hole, you also misrepresented what the image of a black hole is.

Self-driving won't take over by just being available in the new cars people are buying anyway.

It'll take over when people find it cheaper to ride robotaxis than to own a vehicle at all. That's potentially a much quicker transition, requiring significantly fewer new vehicles.

Nobody is going to sit and wait for a robo uber in the suburbs for ten minutes to go to the grocery store ten miles down the road. This is the main problem, America has suburbanised itself to hell and no amount of robo taxis will create the world you are suggesting. And elsewhere in the world there’s no need because most places have adequate public transit which solves the problem way more efficiently than robo taxis ever will.

People wait for things all the time and if it can take away the hassle or driving your kids somewhere, maintaining multiple vehicles for multiple family members etc. I'd bet that it does take off. Public transport doesn't solve groceries or other types of shopping though even in highly connected cities, especially for people with health problems or just aging ones.

Imagine a parent today driving to work and dropping off their two kids at school on the way. Instead of one car making three stops, you're suggesting three robo taxis would accomplish the same thing. What do you think already clogged roads would look like in this future? The only scalable solution is public transit, walking, and cycling.

I used to live in Germany, which had lots of excellent public transit, walking, and cycling. There was still quite a bit of car traffic. If even Germany's public transport isn't enough to eliminate cars, then we should put some effort into improving car transport, in addition to whatever we do with buses and bike lanes.

To that end, I'm not convinced automation will make things worse.

If the kid's school is near the parent's route to work, there's nothing stopping the parent from saving money by taking the kid in the same taxi.

If the kid's school is in the opposite direction, then a separate robotaxi can be more efficient. What matters is total system mileage. If a robotaxi takes the kid, then picks up a commuter starting near the school, then we save the parent's trip from the school back to their starting point.

Nobody is going to sit and wait for a robo uber in the suburbs for ten minutes to go to the grocery store ten miles down the road.

I absolutely would do that. Sometimes I already do wait that kind of time for a taxi or bus. Having the robo taxi turn up in ten minutes from when I decided I wanted to go to the shop would be fantastic.

Public transport doesn't compare to robotaxis. If robotaxis become a reality I expect a significant drop in car ownership and public transport use. The price will be competitive with the price of public transport, at least for single fare prices.

I can catch a bus every five minutes a one minute walk from my house in Utrecht. Those buses go anywhere. In Oslo I could ride the T-bane to the forest or to downtown. In Tokyo it was obvious public transit is the only way to go. All the places I lived outside the USA I would never take a taxi much less a robot one in lieu of Public transit.

Because taxis are not cheap enough. Once they become cheap enough, by not having a driver, the equation changes. Travel becomes more convenient.

In a city I don’t take public transport over uber due to the cost but due to the convenience and speed.

The only exception is the US, and even then that doesn’t apply in Manhattan.

That's what I'm saying. When price goes down, robotaxis become a more viable option due to convenience and speed. You're shopping? Just put it in the trunk. Have fun with 4 grocery bags on public transport.

Public transport can be faster for going from A to B, but most people live a couple of kms from A and they're not going to exactly B but somewhere in the vicinity. The "last mile" will more likely be by robotaxi.

> When price goes down, robotaxis become a more viable option due to convenience and speed.

Lower prices don't make things more convenient or fast.

> You're shopping? Just put it in the trunk. Have fun with 4 grocery bags on public transport.

Here is "The Shops at South Town" mall in Salt Lake City: https://www.google.com/maps/@40.5617162,-111.8944801,171m/da...

Note that some parts of that car park are 300m from some of the shops in that building. (There are choices of shops and parking spaces larger than that, but it looks like I can easily justify 300m without resorting to maxima).

Here is my old apartment: https://www.google.com/maps/place/Prenzlauer+Promenade+1,+13...

Note that within 300m of that address — i.e. the bounds of the example American shopping mall car park — there are 6 [super]markets, a pharmacy, at least five cafe/bakeries, two cinemas, several takeaways, at least two restaurants, a dentist, a car repair shop, a car dealership, a pet-goods shop, a kitchenware shop, a newsagent/post office, two sets of tram stops, four sets of bus stops, and I'm not counting any corner shops ("Späti" in the local language).

Carrying a few bags of groceries around inside well-planned cities is pretty trivial. Anything connected to that area by a single mode of transport is no harder to take shopping through than the transit systems within a mall — escalator, elevator, travelator — and that means one of the main commercial areas of the city (Alexanderplatz) and the road to it (which itself is basically a 3km long half-density strip mall with a tram the whole length and a railway crossing it in the middle and another at the south end) are both trivial to reach even with shopping — I've even seen someone taking an actual, literal, kitchen sink on one of Berlin's trams.

Even in my current place, still in Berlin but close to the outskirts without the advantages of density, a mere four grocery bags is easier to get through public transport than it is from one end of an American car park to the other.

> Lower prices don't make things more convenient or fast.

You are replying to a straw man, I never said anything like that. It makes for dull discussions.

> Even in my current place, still in Berlin but close to the outskirts without the advantages of density, a mere four grocery bags is easier to get through public transport than it is from one end of an American car park to the other.

It's unclear to me why you compare it to a geographically unrelated area. You should compare the public transport with a robotaxi alternative. At similar pricing, how is convenience and speed impacted.

>> Lower prices don't make things more convenient or fast.

> You are replying to a straw man, I never said anything like that. It makes for dull discussions.

If you didn't mean that, why did you say "When price goes down, robotaxis become a more viable option due to convenience and speed"?

> You should compare the public transport with a robotaxi alternative.

The alternative being "do I put these four bags in a trunk, or on the seat next to me?"

Not the op. If you reread what he wrote I think he’s trying to say the price going down changes the viability of the option—not the convenience and speed which he treats as inherent to the car.

Said another way: When price goes down, robotaxis (which are convenient and speedy) become a more viable option.

Thanks, that reading looks plausible :)

Yes, that is what I meant.

That's because people are behind the wheel. Once car ownership drops, the computerized traffic control can get much more efficient.

Computerised public transport is even more efficient

Your public transport with robo taxis takes up far more space in the surface than there is available in a normal city, as shown in this photo.

https://danielbowen.com/2012/09/19/road-space-photo/

Not sure what you think is my public transport. My ideal computerized public transport combines vehicles of all sizes and types, and an uber-like app let's me choose what I prefer based on cost and other attributes.

Why would you wait for 10 minutes? Robo taxis would be driving all around to be optimally available to the market.

The optimal route means picking up multiple people on the same route, and dropping them off elsewhere on the route, and then making the vehicle large enough so that they can do this comfortably, i.e. a bus.

It ends up being another form of public transport — perhaps one with no predictable timetable, or perhaps timetables are themselves a useful Schelling point, I wouldn't want to guess.

But as fixed bus routes make it possible to plan things like bus shelters, I think I'd prefer a bus over the logical progression of self-driving "taxis" that constantly move rather than finding parking spots.

That's optimal for the vehicles, but less optimal for the people, who have to walk to stops. I used to live in Germany which had fantastic public transport, but walking a mile to the nearest stop was something we took in stride. That's great for health, but didn't prevent Germans from being quite fond of their personal automobiles, and there didn't seem to be noticeably less traffic than in the US.

Meanwhile I think plenty of people find Uber perfectly convenient, just a bit expensive.

As far as physical efficiency goes, Tesla's plan is more efficient than their competitors'. Their dedicated robotaxis are either small two-seaters, or decent-size buses.

So people are not willing to wait for a robotaxi, but they are willing to wait for buses and trains?

Assuming we get to your level of value prop. It will still only be a choice for new first time buyers

If I already own a car, for which am paying an emi and no hope of selling for good value (because market is dropping as newest buyers disappear) it won’t make economic sense for me till I generate LTV from the car to switch over .

Either way it till take 1-2 decades after ubiquitous L5 availability, even if that was possible at all and soon

there are already cheaper ways to ride from point A to point B without owning a car (especially in urban areas) and yet car ownership has never been higher… ideal robotaxis will take at least one full generation to actually materialize.

just as a silly personal example, I took a car to the shop and got Uber credits. my wife was heading into the city for an event (had to deal with both traffic and parking) and I was like “take a free Uber” and she was like “no way, driving my own car…” my daughter on the other hand…

> despite how far it has come it cannot in next 30 years be ubiquitous, even if the l5 magic tech exists today in every new car sold it will take 15 years for current cars to lifecycle.

We must have a different meaning of ubiquitous. Ubiquitous means "found everywhere" not "has eliminated everything else."

You don't even need to cycle the fleet once to meet the definition of ubiquitous. If you can get a self driving car in any city and many towns, you see them all over the place, and a third of trips are done with them, that'd be ubiquitous in my book.

I don't see why you couldn't get there in 3 decades. I don't think it's likely in 12 years but it seems possible in that timeframe.

I considered fleet change as the transformation that eliminates driving as a needed skill .

Getting a slightly better version of Waymo in SF is just augmenting for a poor public transit with a very American car solution .

It scales pretty poorly and cannot replace good public transit, at best can augment them as last mile connectivity .

There is value to those of course and is disruptive to a sector, it is still a specific use case and not world changing. You are only disrupting the taxi industry not driving itself .

The claims at the time was not we are going replace taxis with robotaxis , but driving itself will be a thing of the past. That is the nature of hype cycles .

>people will move on to the next fad

AI isn't really a fad. It's going to something more like electricity, say.

My favorite comparison right now is to the microwave oven.

It's a novel thing. It's not going anywhere. It makes some things easier, sometimes, in both personal and professional settings. But you're not at a big loss without it and in many cases your choice to do without is a choice that favors quality, control, nuance, and skill refinement.

AI probably has more capacity to be incorporated into many things than microwaves.

I will bite, going with your analogy comparing to electricity on the other end of the spectrum.

electricity is not ubiquitous in many parts of the global south full 130 years.

The common definition is- access to electricity source that can provide very basic lighting, and charge a phone or power a radio for 4 hours per day. Not even grid connected.

Even with this basic definition there is still full 660 million people who don’t have access to even that. It only dropped below a billion in 2015.

https://ourworldindata.org/energy-access

You need beyond the basic to be meaningfully transformative, clean cooking fuel access is one proxy used that is still 2Billion people without access for example.

Nobody is denying the transformative nature of electricity or say internet , but even these things take a long long time, more than one life time to propagate and be world changing, and not because of their choice of lifestyle to live disconnected.

If electricity, clean water, sanitation and nutrition and basic healthcare is not available to half the population, what is AI going to do in next 3- 5 years or whatever ridiculous timeframe Sam Altman keeps talking about .

And this is a not productivity problem, there is more than enough food production capacity to feed the world, yet starvation and malnutrition is not solved.

AI may rock your world, it is not going to rock the world anytime soon even if was like electricity.

Something can be transformative long (long) before its "completely available to everyone ". Clearly electricity is transformative despite 660 million people not giving it.

So I guess it all depends on your definition of "transformative".

If a billion people are affected? 2 billion? One continent? All of them?

If something is big in Australia, Asia, Europe and North America, is that enough to describe it is "rock the world"?

People have been saying that for a few years now. I've been using it as a very advanced autocomplete while coding for that time and not much else.

I use it for cooking, proofreading, learning German, skimming long documents, shopping advice (by uploading photos), and asking dumb questions about physics that get me [closed] if I were to post on physics stack exchange, in addition to being a coding assistant.

Most of those existed before llms. Also still like a microwave, not very essential.

Farming existed before the combine harvester, and like the combine harvester LLMs are very useful tools in these tasks.

You can't get a useful response from German for Dummies if you ask it to rate the email you're writing in German and let you know what needs improvement, human teachers who can are expensive, and friends may not wish to be constantly bothered by a 40 year old asking the kinds of questions a native speaker might have asked their teacher when they were 13.

As a new language learner you're, by definition, not going to be in a place to judge the quality of the "feedback" you get from it.

Course not. I can make the judgement call "this is good at German" by virtue of all the Germans around me using it.

(Conversely, I'm told it's quite bad at, say, Chinese — though mainly by people online, as there aren't many native Chinese speakers in my life right now).

Electricity probably had niche uses for the first few years but as electrical stuff got better it got used in all sorts of things. I imagine AI will go a bit like that if quicker.

Looking back at the history of electricity, there is early stuff from the Greeks and then:

Faraday's homopolar disc generator 1831

Light bulb 1870

Transistor 1947

etc. So quite leisurely.

Considering the exponential increases in invention and adoption of new technologies for the past 200 years, I'm not sure that timeline is very close to what I'd expect from a new technology in this era.

I'd argue that we've already passed peak LLM as I think, after been having burned by it a few times, it's usage as a professional or reliable technology has seceded, rather than accelerated, over the past 6 months. Gone are the days of doom sayers talking about AGI taking over the world and LLMs replacing every single job at the call center and in software teams. Now everyone's scaled back to the things you can actually let it do, which is assist with a lot of coddling/verifying.

Common, VR, NFTs and blockchain were always abysses of void looking for a usecase. Driving cars maybe, but development has been stalling for 15 years.

Not to their proponents particularly at the time.

For the rest of us in HN it was easier to judge on much mileage will get, but a long time in each cycle we were in the minority, just like the AI skeptics including me and many here are today on how much we will value get.

Don’t get are wrong, AI is more real and more useful than the things before it, but everything is on a spectrum of success - some more and some less, but none of it has been or will be world changing or truly global.

The true breakthroughs rarely get public or even tech community attention until late in the cycle although their progress is public and can be tracked, like with EUV or mRNA or GLP-1 etc . Others which are incremental changes like 5G or cheaper smartphones do not get as much attention as their impact should warrant especially compared to the fads

Hype cycles don’t correlate well with real tech cycles

I'd say VR is just another form factor for games rather than being a void looking for a use-case, but I agree about NFTs and cryptocurrency*.

* technically not all blockchains because git and BitTorrent are both also hash trees and therefore non-currency block-chains, but definitely the cryptocurrency uses of them.

The point is not that they are snakeoil and have zero value . Every one of these things have some value in some area of applications.

However the hype cycle just as now in AI claimed it will change everything, not hey here is interesting new thing and it is going be used in select applications . Nobody is investing billions and billions on that pitch

Consciousness is a stupid and unreasonable goal, it is basically impossible to confirm that a machine isn’t just faking it really well.

Singularity is at least definable…Although I think it is not really the bar required to be really impactful. If we get an AI system that can do the work of like 60% of hardcore knowledge workers, 80% of office workers, and 95% of CEOs/politicians and other pablumists, it could be really change how the economy works without actually being a singularity.

Those numbers are approximately zero, because demand will immediately 10x as soon as AI makes that affordable, as computers have always done. So, profitable (if the AI can cost less than a human for the work), but not changing how the economy works

If white collar jobs get automated away to the extent that blue collar jobs were in the industrial revolution, that will be, by definition, a massive upheaval of the economy.

Comparing the site that was #8 in Internet worldwide traffic last month, has 300M weekly active users, 1B messages a day and is basically the fastest adopted software product in history to NFTs, VR and Blockchain does not make any sense.

Did you really just compare the thing that makes me able to code and ship 5x faster to... NFTs?

> code and ship

I notice that doesn't say what happened to the throughput of "diagnose, fix, and reduce technical debt." :p

Matter of perspective, Ai tools value depends on your existing skill level and whether it only doing what you already can do and you can correct it very well.

However that kind of user in my experience is rare , vast majority are junior developers who do not understand the output well enough and now generate 5x of it that I have to review and they no longer make the effort to understand any foundational concept .

This is not new, the rise in popularity of StackOverflow created a class of developers who didn’t want to spend the time to learn properly reading the manual.

In itself reducing the skill level needed to be productive is not inherently bad, it was PhD in CS could only program in 70s and 80s. Each generation of tech and abstraction opened up software development to newer class of employees.

However the difference is the industry desperately needed expansion each time because PCs became popular or the desktop or mobile devices etc . There was enormous demand for new applications both personal and professional which couldn’t be satisfied with kind of volume CS research programs could fulfill.

There is no such transformation currently , there aren’t a new class of users or major applications coming up in a such high volume, that quality drop is compensated by quantity .

There is lot more need for higher quality products as software market is plateauing from the highs mobile adoption gave.

really depends on the kind of AI. If I said I shipped a AI-genned game 5x faster , then people would indeed compare it to NFTs.

> 2 years before it was VR, few years before that NFTs and blockchain everything

Are you so deep in the SV tech echo chamber that you can’t distinguish between tech bro / snake oil salesman hype and something that actual normal people are engaged by?

The only people talking about NFTs in a non-disparaging way were delusional in large part because they themselves bought into it wanted to egg others on. NFTs were just a stupid faddish speculative ‘commodity’ market. It’s a completely different thing.

Ditto with “blockchain”. That well was positioned by people trying to make money from cryptocurrencies. The mechanics allowed for that. Again, it’s a completely different thing. Civilians with no vested interest weren’t sitting in restaurants singing the praises of decentralised ledgers or monkey pictures on someone’s Google Drive. I assure you. I also assure you that this IS however happening with ChatGPT. Out here in the real world, we see it.

This doesn’t mean that there aren’t issues with LLMs, both philosophically and in therms of their actual output. This doesn’t mean there aren’t the same old SV snake oil salespeople and YouTube wannabes trying to push some BS products and courses. But let’s not pretend for a second like this is usefully comparable to “blockchain” and “NFTs”.

I think that underestimates public engagements in these hypes. Billions were wasted by "normal" people worldwide on VR headsets, in-game nfts, and shitcoins. Real world money, not some inflated VC capital projections. And while the current LLMs still are an "advanced Eliza", regular people are quite happy to pay for its services.

I am sorry, every A-List celebrity was shilling a NFT project, from superbowl to other public spheres of discourse seemed all crypto. The future president of the country has a crypto project to sell, I don’t see how much mainstream it can get.

Yes, There is close connection to SV hype cycle and what happens in mainstream, most mainstream ones originate in SV but the ones I mentioned specifically impacted the larger public not just SV tech sphere

Blockchain and NFT took over the Super Bowl. That's pretty mainstream. There are Bitcoin ATMs.

Can you expand on your spreadsheet analogy?

I think Joel Spolsky explained the main Office moat well here: https://www.joelonsoftware.com/2008/02/19/why-are-the-micros...

> ... it might take you weeks to change your page layout algorithm to accommodate it. If you don’t, customers will open their Word files in your clone and all the pages will be messed up.

Basically, people who use Office have extremely specific expectations. (I've seen people try a single keyboard shortcut, see that it doesn't work in a web-based application, and declare that whole thing "doesn't work".) Reimplementing all that stuff is really time consuming. There's also a strong network effect - if your company uses Office, you'll probably use it too.

On the other hand, people don't have extremely specific expectations for LLMs because 1) they're fairly new and 2) they're almost always nondeterministic anyway. They don't care so much about using the same one as everyone they know or work with, because there's no network aspect of the product.

I don't think the moats are similar.

"Basically, people who use Office have extremely specific expectations."

Interesting point, but to OP's point- This wasn't true when Office was first introduced and Office still created a domineering market share. In fact, I'd argue these moat-by-idiosyncracy features are a result of that market share. There is nothing stopping OpenAI from developing their own over time.

Does office actually have a moat? I thought the kids liked Google docs nowadays. (No opinion as to which is actually better, the actual thing people should do is write LaTeX in vim. You can even share documents! Just have everybody attach to the same tmux session and take turns).

If you're writing a spreadsheet in LaTeX, I suspect something has gone very wrong somewhere along the line.

Spreadsheets are as much a calculation environment as they are a table of figures, and if you're technical enough to be writing docs in LaTeX you should be doing the calculations in a more rigorous environment where you don't copy the formulas between cells.

Excel, the world's most popular functional programming language.

Excel, the only IDE universally allowed on corporate devices

I'd argue the web browser falls into that category as well.

In more modern times, although I'd say the barrier of entry is higher there.

Plus most analogs would require additional procurement, to which the response for most business users would be "No, you're not a developer."

> This wasn't true when Office was first introduced and Office still created a domineering market share

It was absolutely true at that time, but Microsoft was already a behemoth monopoly with deep customer connections throughout enterprise and was uniquely positioned to overcome the moats of their otherwise fairly secure competitors, whose users were just as loyal then as the GP is describing about users now.

Even if OpenAI could establish some kind of moat, office applications make for very poor analogy as to how they might or whether they need to.

It totally was. Bit different. Excel has Lotus keybindings available to this day, the spreadsheet was the home computer’s 1st killer app and Microsoft killer it and took the market share.

Sure, there's nothing stopping any business from developing a moat. The Excel example doesn't make the case of OpenAI any clearer to me.

> Can you expand on your spreadsheet analogy?

Sure.

(I've been coding long enough that what Joel writes about there just seems obvious to me: of course it happened like that, how else would it have?)

So, a spreadsheet in the general sense — not necessarily compatible with Microsoft's, but one that works — is quite simple to code. Precisely because it's easy, that's not something you can sell directly, because anyone else can compete easily.

And yet, Microsoft Office exists, and the Office suite is basically a cost of doing businesses. Microsoft got to be market-dominant enough to build all that complexity that became a moat, that made it hard to build a clone. Not the core tech of a spreadsheet, but everything else surrounding that core tech.

OpenAI has a little bit of that, but not much. It's only a little because while their API is cool, it's so easy to work with that you can (I have) asked the original 3.5 chat model to write its own web UI. As it happens, mine is already out of date, because the real one can better handle markdown etc., so the same sorts of principles apply, even on a smaller scale like this where it's more of "keeping up in real time" rather than "349 page PDF file just to get started".

OpenAI is iterating very effectively and very quickly with all the stuff around the LLM itself, the app, the ecosystem. But so is Anthropic, so is Apple, so is everyone — the buzz across the business world is "how are you going to integrate AI into your business?", which I suspect will go about the same as when it was "integrate the information superhighway" or "integrate apps", and what we have now in the business world is to the future of LLMs as Geocities was to the web: a glorious chaotic mess, upon which people cut their teeth in order to create the real value a decade later.

In the meantime, OpenAI is one of several companies that has a good chance of building up enough complexity over time to become an incumbent by a combination of inertia and years of cruft.

But also only a good chance. They may yet fail.

> On the other hand, people don't have extremely specific expectations for LLMs because 1) they're fairly new and 2) they're almost always nondeterministic anyway. They don't care so much about using the same one as everyone they know or work with, because there's no network aspect of the product.

For #1, I agree. That's why I don't want to bet if OpenAI is going to be to LLMs what Microsoft is to spreadsheets, or if they'll be as much a footnote to the future history of LLMs as Star Division was to spreadsheets.

For #2, network effects… I'm not sure I agree with you, but this is just anecdotal, so YMMV: in my experience, OpenAI has the public eye, much more so than the others. It's ChatGPT, not Claude, certainly not grok, that people talk about. I've installed and used Phi-3 locally, but it's not a name I hear in public. Even in business settings, it's ChatGPT first, with GitHub Copilot and Claude limited to "and also", and the other LLMs don't even get named.

> ...like saying Microsoft has no moat for spreadsheets

Which would be very inaccurate as network-effects are Excel's (and Word's) moat. Excel being bundled with Office and Windows helped, but it beat Lotus-123 by being a superior product at a time the computing landscape was changing. OpenAI has no such advantage yet: a text-based API is about as commoditized as a technology can get and OpenAI is furiously launching interfaces with lower interoperability (where one can't replace GPT-4o with Claude 3.5 via a drop-down)

> OpenAI has no such advantage yet: a text-based API is about as commoditized as a technology

It has branding, for most people AI is ChatGpt. Once you reach critical mass, getting people to switch becomes difficult if your product is good enough and most people are happy.

Branding is a very weak moat outside fashion industry and similar places - outside visible status symbols. It's a relatively strong moat for clothes, watches, even things like the iPhone.

But for things you do alone at home, this quickly goes away. Uber is a strong brand, everyone associated non-taxi taxis with Uber. When Bolt came around in much of Europe, and offered the same service with better wait times (they were paying drivers more, so lots of drivers switched), people moved over to Bolt in droves.

> Branding is a very weak moat outside fashion industry and similar places - outside visible status symbols.

Have you tried getting someone to switch from Chrome to Firefox?

UI is basically the same, the product is more performant (faster load, easy to disable all ads, etc). But the moat is how resistant a normal user is to switch from the thing they've used without having to think about it, to the new thing.

I can definitely see the ChatGPT app becoming as "sticky" as Chrome.

First of all, that's not branding, it's bias for a known, existing solution.

Second of all, people have switched browsers en masse before, it just requires a good enough reason to do so. People switched from IE6 to Firefox in a relatively large number, and then they all switched to Chrome. Even today, people on Windows tend to install Chrome, while people on Macs tend to stick to Safari, so some browser choice is happening.

Third of all, and perhaps most importantly really, Chrome is free, so the only reason to switch away would be finding some major features you want in another browser. ChatGPT is not free, and it's losing money fast at its current price point. But if they make it more expensive, as they need to unless they can significantly reduce their costs, people will look for alternatives. Other than fashion brands, I don't think you'll find almost any example of people in general using a more expensive service when a cheaper equivalent one exists and is being actively marketed.

In my opinion the difference is that a recent graduate knows to say “I don’t know” to questions they’re not sure on, whereas LLMs will extremely confidently and convincingly lie to your face and tell you dangerous nonsense.

My experience is that intellectual humility is a variable, not a universal.

Seen some students very willing to recognise their weaknesses, others who are hamstrung by their hubris. (And not just students, the worst code I've seen in my career generally came from those most certain they're right).

https://phys.org/news/2023-12-teens-dont-acknowledge-fact-ea...

And yes, this is a problem with some LLMs that are trained to always have an answer rather than to acknowledge their uncertainty.

> For current standards of what intelligence means, we'd better hope we don't get ASI in the next decade or two, because if and when that happens then "humans need not apply" — and by extension, foundational assumptions of economics may just stop holding true.

I'm not sure that we need superintelligence for that to be the case - it may depend on whether you include physical ability in the definition of intelligence.

At the point that we have an AI that's capable of every task that say a 110 IQ human is, including manipulating objects in the physical world, then basically everyone is unemployed unless they're cheaper than the AI.

While I would certainly expect a radical change to economics even from a middling IQ AI — or indeed a low IQ, as I have previously used the example of IQ 85 because that's 15.9% of the population that would become permanently unable to be economically useful — I don't think it's quite as you say.

Increasing IQ scores seem to allow increasingly difficult tasks to be performed competently — not just the same tasks faster, and also not just "increasingly difficult" in the big-O-notation sense, but it seems like below certain IQ thresholds (or above them but with certain pathologies), some thoughts just aren't "thinkable" even with unbounded time.

While this might simply be an illusion that breaks with computers because silicon outpaces synapses by literally the degree to which jogging outpaces continental drift, I don't see strong evidence at this time for the idea that this is an illusion. We may get that evidence in a very short window, but I don't see it yet.

Therefore, in the absence of full brain uploads, I suspect that higher IQ people may well be able to perform useful work even as lower IQ people are outclassed by AI.

If we do get full brain uploads, then it's the other way around, as a few super-geniuses will get their brains scanned but say it takes a billion dollars a year to run the sim in real time, then Moore's and Koomey's laws will take n years to lower that to $10 million dollars a year, 2n years to lower it to a $100k a year, and 3n years to lower it to $1k/year.

I think this trend of using IQ as a primary measuring stick is flawed.

Human minds and AI minds have radically different architectures, and therefore have different strengths and weaknesses. IQ is only one component of what allows a typical worker to do their job.

Even just comparing humans, the fact that one person with, say, a 120 IQ can do a particular job—say they are an excellent doctor—obviously does not mean that any other human with an equal or greater IQ can also do that job effectively.

> I think this trend of using IQ as a primary measuring stick is flawed.

I agree, but it's difficult to fully describe the impact of a tech where $5 gets you the ability to do arithmetic faster than literally every human combined even if we were all working at the speed of the current world record holder, yet simultaneously it took a billion dollar research team 14 years to figure out how to get a robot to reliably tie its own shoelaces (this September: https://deepmind.google/discover/blog/advances-in-robot-dext...)

So, IQ it is. Rough guide for a human equivalent intelligence scale, but being mapped to a wildly different architecture so that it, the different architecture, can be reasoned about.

> Even just comparing humans, the fact that one person with, say, a 120 IQ can do a particular job—say they are an excellent doctor—obviously does not mean that any other human with an equal or greater IQ can also do that job effectively.

Indeed, though in this case I would expect that such a human could learn any of those skills.

It's difficult to fully describe, so let's just give up and use a deeply flawed benchmark? Why not try to develop benchmarks that actually work and tell us something useful instead?

The key issue is that there is no result an AI can achieve on a standard IQ test which guarantees that same AI can do any task at a superhuman level, apart from taking IQ tests. Can an LLM that scores 250 replace a human driver? Who knows? Can it replace a senior software engineer? Who knows? Can it replace a manual laborer? Again, who can say? We know a human with a 250 IQ can do all those things, but with an AI we have no idea, because those tasks have many more inputs than IQ.

Rather than IQ, which tells us almost nothing concrete, I think we should focus on what tasks it can actually achieve. What's a Waymo's IQ? Who cares?! I don't care about its IQ. I care about its ability to drive me safely across the city. Similarly, I don't care whether a coding AI can drive a car or write great novels.

Of course it's interesting to measure and speculate about IQ as it relates to AGI, but I think it gives people the very mistaken impression that we are on some kind of linear path where all we need to do is keep pushing up a single all-important number.

> It's difficult to fully describe, so let's just give up and use a deeply flawed benchmark? Why not try to develop benchmarks that actually work and tell us something useful instead?

Two reasons. First: in this sub-thread I'm focusing on employment issues due to AI, so consider the quote from above:

> At the point that we have an AI that's capable of every task that say a 110 IQ human is, including manipulating objects in the physical world, then basically everyone is unemployed unless they're cheaper than the AI.

IQ doesn't capture what machines do, but it does seem to capture a rough approximation of what humans do, so when the question is "can this thing cause humans to become economically irrelevant?", that's still a close approximation of the target to beat.

You just have to remember that as AI don't match human thinking, so an AI which is wildly superhuman at arithmetic or chess isn't necessarily able to tie its own shoelaces. The AI has to beat humans at everything (at least, everything economically relevant) at that IQ level for this result.

Second: Lots of people are in fact trying to develop new benchmarks.

This is a major research topic all by itself (as in "I could do a PhD in this"), and also a fast-moving topic (as in "…but if I tried to do a PhD, I'd be out of date before I've finished"). I'm not going to go down that rabbit hole in the space of a comment about the exact performance thresholds an AI has to reach to be economically disruptive.

For a concrete example of quite how fast-moving the topic is, here's a graph of how fast AI is now beating new benchmarks: https://ourworldindata.org/grapher/test-scores-ai-capabiliti...

More importantly, basically all of the IQ tests are in the training sets, so it's hard to know how the models would perform on similar tests not in the training set.

Indeed. People do try to overcome this, for example see the difference in results between "Show Offline Test" and "Show Mensa Norway" on https://trackingai.org/IQ

Even the lower value is still only an upper-bound on human-equivalent IQ, as we can't really be sure the extent to which the training data is (despite efforts) enough to "train for the test", nor can we really be sure that these tests are merely a proxy for what we think we mean by intelligence rather than what we actually mean by intelligence (a problem which is why IQ tests have been changed over the years).

My intention in this sub-thread is more of the economic issues rather than the technical, which complicates things further, because if you have an AI architecture where spending a few tens of millions on compute — either from scratch or as fine-tuning — gets you superhuman performance (performance specifically, regardless of if you count it as "intelligent" or not), then any sector which employs merely a couple of hundred workers (in rich economies) will still have an economic incentive to train an AI on those workers to replace them within a year.

This is still humans having jobs, and still being economically relevant, but makes basically everyone into a contractor that has to be ready and able to change jobs suddenly, which is also economically disruptive because we're not set up for that.

This is it. Another primary example is that I can put a 80 IQ person to work on a farm or let them build something under supervision. There is no way for an AI to do either at this time, even if the AI is much smarter.

Ah, AI have been doing both farm work and "building things under supervision" work for a while now.

Robots are a thing, AI controlling robots is a thing.

Robots at this point are specialized for a highly specific task. None of them are as dynamically deployable as a 80 IQ person. You can let a walk around a construction site doing whatever, no such robot is available right now.

If I'm wrong please post where I can buy such a general robot.

Robots are hardware, AI is software, so in principle you can put any AI in charge of any robot.

A quick google got me this list of current uses, I'd suggest you can skim the first part as it will be clear when you get to the actual examples: https://builtin.com/robotics/humanoid-robots

(But disregard Atlas, IIRC Boston Dynamics don't sell that, only the robot dogs? But you didn't actually ask for a humanoid, so you can have the dogs as a replacement).

I've seen demos from Agility before this: https://agilityrobotics.com/

Likewise Figure being tested in BMW: https://www.figure.ai/

And are you really sure about what it means to be an IQ 80 human? Because that's where people start having problems reading plans, safety instructions, timesheets etc., or following multi-step instructions, handling unexpected situations.

A robot is not only hardware. A robot is hardware and software.

The robots in the links you posted are either not commercially available or not capable of working in an unrestricted construction environment. Such an environment do require a humanoid frame at this time because the entire world is geared to that. I agree that a few show promising demos, but that's not what I would "available". Perhaps I should have been more clear. Unless I(As in some rando corporation) can go out and buy it I don't consider it available.

I completely agree with everything that you're saying here - my use of the term "basically everyone" was lazy in that I'm implying that at the 110 IQ level the majority (approx 70%) of people are economically obsolete outside of niche areas (e.g. care/butler style work in which people desire the "human touch" for emotional reasons).

I think that far below the 70% level we've already broken economics. I can't see a society functioning in which most people actually _know_ that they can't break out of whatever box they're currently in - I think that things like UBI are a distraction in that they don't account for things like status jockeying that I think we're pretty hardwired to do.

Until a problem space is "solved", you will still need AIs that are more capable than those with 110 IQ to review the other AIs work. All evidence points to "mistakes will be made" with any AI system I have used and any human I have worked with.

> he actively empathised with the protestors.

I have significant doubt that Sam is capable of empathy, period. It seems like what he's capable of is an extremely convincing caricature of it which he has practiced for many years.

I mean, one could say the same about basically every human (and indeed, some AI systems).

> foundational assumptions of economics may just stop holding true

Those assumptions are already failing billions of people, some people might still be benefiting from those "assumptions of economics" so they don't see the magnitude of the problem. But just as the billions who suffer now have no power, so will you have no power once those assumptions fail for you too.

> I overheard a daughter talking to her mother about ChatGPT and KI (Künstliche Intelligenz, the German for AI).

Given the fact that how much ALL of our media beating that drum since GPT-3, I am hoping that my cat will start talking about it without understanding what it is. Even Draghi’s report has immense hidden lobbying dedicated to KI.

> The product market fit is fantastic.

> [...]

> What's not obvious is how to monetise it.

This is an interesting new use of 'product market fit'. I would have thought that in the absence of a path to monetisation, there is no market. Or we could talk about the 'market' for selling $10 bills priced at $5.

I phrased that badly, so that's fair.

If you follow the rest of that paragraph, my meaning may be clearer: there's a market for what they do right now — the OpenAI financial reports that have been bandied around say that if they were only offering this as a paid service, not also developing new models, they'd be making a profit — but it's not obvious how this will change going forward.

The first spreadsheets were easy to market: here's a solution to your problem, give us money and you can have this solution. But as time goes on, you need more than the original solution, because that's too easy to replicate.

ChatGPT was easy to replicate, and has been replicated.

The "needing more" to stay in the game is what I meant by the Red Queen race. What exists now is fine, it's a good fit, but the question of the long-term is still open.

> protesters dropped in on the after-award public talk

I’m going to guess that GP does not consider random protestors to be in Sam Altman’s ‘sphere’

> The product market fit is fantastic.

This is true insomuch that you define “product market fit” as “somebody mentioning it in an Indian restaurant in Berlin”

> every definition of "very smart"

Every definition you say?

https://bsky.app/profile/edzitron.com/post/3lclo77koj22y

> Every definition you say?

When I was growing up, "very smart" people had a law degree or a medical degree or understood quantum physics and/or relativity (but all three would be silly), or spoke perhaps three or four languages (but thirty would be a joke). They might perhaps also be above average at games of skill like chess, or perhaps could improvise poetry in some specific style (but every style would be a bit of a Mary Sue).

I may not trust ChatGPT's medical diagnosis or legal analysis, but it does pass those exams; I may not expect much novel insights about unifying QM and GR, but it seems to "know" the topics well enough for people to worry it may lead to cheating in those subjects and most others besides.

And the chess score may not be the absolute best, but it's way above the average human player.

And it seems entirely adequate in quite a lot of languages (though not native), and most styles of poetry (again, not what anyone would call the Nobel committee for, but way above what most people can dream of).

> https://bsky.app/profile/edzitron.com/post/3lclo77koj22y

We also didn't have spellcheck running all the time when I was growing up, and when we did there was quite a lot of red, but even that misses things like "I'm applying for collage" and "what lovely wether*" which were also real mistakes made by actual humans.

* spelled like that it means "castrated ram"

> I may not trust ChatGPT's medical diagnosis or legal analysis, but it does pass those exams;

It's really, really, really important to note that this could easily just be memorisation (and indeed, the GSM Symbolic paper from Apple suggests that it most likely is).

We don't have a good way of assessing the ability of models literally trained on all free text in existence, but comparing their performance based on text they've seen in training is definitely not it.

What I've always disliked about calling it "memorisation" is that we already have machines with much better memorisation than what LLMs do: the hard drives with the source documents.

To the extent that it's able to respond to and with natural language, this is already an improvement over, say, grep or PageRank.

To the extent that it is memorisation in the way we mean when we say a human has memorised something, it's an example of another aspect of what was meant by "intelligence" when I was growing up.

It's definitely weird and alien as an intelligence, but it still ticks the boxes we used to have even a decade ago, 2014, let alone the late 80s to 1999 when I turned 16.

That’s an odd way to define “smart”. The current crop of LLMs have absolutely no ability to query the raw input or output. I can look at the words I’m typing and think about them. Transformers don’t even get the raw characters, and they also don’t have any feedback from later layers into the queries generated at earlier layers. So the fact that they have any ability at all to reflect on their own input is IMO both impressive and somewhat surprising.

I occasionally wonder whether one could get better results by allowing later layers to query against the K/V data from earlier layers. This might make inference, and possibly even training, of models that don’t fit in a single node’s memory rather more complicated.

> When Altman was collecting the award at Cambridge the other year, protesters dropped in on the after-award public talk/Q&A session, and he actively empathised with the protestors.

I've known many CEO who will stare you right in the eye and say they hear your concerns then immediately disregard you and continue with business as usual. Being a psychopath is part of the job description.

> The product market fit is fantastic. This isn't the first time I've heard random strangers discussing it in public.

People were constantly discussing crypto in public, it even regularly made it into Tagesschau. Yet it remains an (influential) niche product.

People discussing a topic in public is not a good proxy for product market fit. It does not imply those people are using said product, nor does it imply people would be willing to pay for said product.

> even the current models meet every definition of "very smart" I had while growing up, despite their errors.

I recently tried to read a grocery store receipt into GPT4o and asked it what the KG price for the steak position was (it was listed as €/kg right next to the position name). It came up with some elaborate math, but not with the correct answer.

So yeah. These models might be able to confidentially answer questions, but I found them to be (partially) incorrect more often than not.

> and he actively empathised with the protestors.

https://www.simplypsychology.org/narcissistic-mirroring.html

Even if so, that's still showing awareness of what his critics are critical of.

Now, can you make a falsifiable prediction: what would a narcissist do that a normal person would not, such that you can tell if he's engaging in a narcissist process rather that what your own link also says is a perfectly normal and healthy behaviour?

> Even if so, that's still showing awareness of what his critics are critical of.

Mere awareness is really that meaningful.

>>> and he actively empathised with the protestors.

>> https://www.simplypsychology.org/narcissistic-mirroring.html

> Now, can you make a falsifiable prediction: what would a narcissist do that a normal person would not, such that you can tell if he's engaging in a narcissist process rather that what your own link also says is a perfectly normal and healthy behaviour?

He doesn't have to, because I think the point was to raise doubts about your interpretation.

But if you're looking for more evidence, there are all the stories (from many different people) about Altman being dishonest and manipulative and being very effective at it. That's the context that a lot of people are going use to interpret your claim that Altman "actively empathised with the protestors."

nit: isn't really that meaningful

Reply on news.ycombinator.com