Hacker News Reader

Comment by lolinder | Hacker News Reader

To recap OpenAI's decisions over the past year:

* They burned up the hype for GPT-5 on 4o and o1, which are great step changes but nothing the competition can't quickly replicate.

* They dissolved the safety team.

* They switched to for profit and are poised to give Altman equity.

* All while hyping AGI more than ever.

All of this suggests to me that Altman is in short-term exit preparation mode, not planning for AGI or even GPT-5. If he had another next generation model on the way he wouldn't have let the media call his "discount GPT-4" and "tree of thought" models GPT-5. If he sincerely thought AGI was on the horizon he wouldn't be eyeing the exit, and he likely wouldn't have gotten rid of the superalignment team. His actions are best explained as those of a startup CEO who sees the hype cycle he's been riding coming to an end and is looking to exit before we hit the trough of disillusionment.

None of this is to say that AI hasn't already changed a lot about the world we live in and won't continue to change things more. We will eventually hit the slope of enlightenment, but my bet is that Altman will have exited by then.

View on HN

Except the article makes none of these points. The article is saying:

a) "the technology is overhyped", based on some meaningless subjective criteria, if you think a technology is overhyped, don't invest your money or time in it. No one's forcing you.

b) "child abuse problems are more important", with a link to an article that clearly specifies that the child abuse problems have nothing to do with OpenAI.

c) "it uses too much energy and water". OpenAI is paying fair market price for that energy and what's more the infrastructure companies are using those profits to start making massive investments in alternative energy [1]. So if everything about this AI boom fails what we'll be left with is a massive amount of abundant renewable energy (the horror!)

Probably the laziest conjecture I have endured from The Atlantic.

[1]: https://www.cbc.ca/news/canada/calgary/artificial-intelligen...

>So if everything about this AI boom fails what we'll be left with is a massive amount of abundant renewable energy

Except that someone has to pay for it. AI companies are only willing to pay for power purchase agreements, not capital expenses. Same with the $7T of chip fab. Invest your money in huge capital expenditures and our investors will pay you for it on an annual basis until they get tired of losing money.

Power purchase agreements are what renewables developers use to obtain financing for construction. The contract is collateral. Worst case, the developer might need to be prepared to find a substitute offtaker for the power if a PPA with a gen AI offtaker goes bust (if brought up by whomever is underwriting the financing, if there are debt covenants, etc).

I absolutely support AI companies signing as many PPAs for low carbon energy even if they implode in the future. The PV panels, wind turbines, and potentially stationary storage will already be deployed at that point.

https://betterbuildingssolutioncenter.energy.gov/financing-n...

What if it is a 5 gigawatt data center in the middle of nowhere or if the power source is a nuclear reactor or three? Presumably, data centers are paying a premium that other customers aren't willing to pay.

Hard to say with random "what ifs." If we're starting merchant nuclear reactors back up, because they have some life left in them and can be operated safely, I support such an operating model as it pushes high carbon energy out of the generation mix until there is more clean energy on the grid.

Illinois provides subsidies to their nuclear reactors because they are low carbon [1], the federal government is subsidizing Diablo Canyon in California [2] and Palisades in Michigan [3]. Every coal fired generator in the US is more expensive to run than to replace with low carbon renewables except the one in Dry Fork, WY, [4] so subsidies are just arguments, not real economic signals.

TLDR We're kicking the can until we can get more clean energy online, and that ramp rate continues to accelerate [5].

[1] https://www.cnbc.com/2021/11/20/illinois-nuclear-power-subsi... ("Why Illinois paid $694 million to keep nuclear plants open")

[2] https://www.reuters.com/world/us/us-finalizes-11-billion-cre... ("US finalizes $1.1 billion in credits for California nuclear plant")

[3] https://news.ycombinator.com/item?id=41696884 (HN: MI nuclear plant finalizes fed loan for first reactor restart in US history")

[4] https://news.ycombinator.com/item?id=37601970 (citations)

[5] https://news.ycombinator.com/item?id=41602799 (citations)

I think you are missing my point. Microsoft signed a 20-year agreement with Constellation to restart TMI Unit 1. Do you think Constellation would make the same agreement with Open AI? The risk is completely different.

No, the point is taken, I think you’re underestimating FOMO and the hype cycle.

If a counterparty is just a bit too rational, move on, lots of other potentials. Markets are sentiment driven. When there is potentially irrational exuberance, leverage it.

Hopefully we get a nuclear reactor or three

Paying fair market price for energy and water is only fair if you don't need extra infrastructure put in place to support you. If you're expecting the local government to build you electricity generating capacity... No. That's not right.

At best, you're forcing old generation capacity that would have been retired to stay online. At worst you're forcing the government to take loans to invest in new capacity you may not be around to pay for in a few years, leaving the public finances holding the ball.

Barring societal collapse, I don't see a scenario in which extra energy isn't used up and as long as it's sold at market prices the government investments should be fine.

It does look like an exit. Employees were given the chance to cash in some of their shares at $86 billion valuation. Altman is getting shares.

New "investors" are Microsoft and Nvidia. Nvidia will get the money back as revenue and fuel the hype for other customers. Microsoft will probably pay in Azure credits.

If OpenAI does not make profit within two years, the "investment" will turn into a loan, which probably means bankruptcy. But at that stage all parties have already got what they wanted.

> If OpenAI does not make profit within two years, the "investment" will turn into a loan, which probably means bankruptcy.

I don’t believe this is accurate. I think this is what you’re referring to?:

Under the terms of the new investment round, OpenAI has two years to transform into a for-profit business or its funding will convert into debt, according to documents reviewed by The Times.

That just means investors want the business to be converted from a nonprofit entity into a regular for-profit entity. Not that they need to make a profit in 2 years, which is not typically an expectation for a company still trying to grow and capture market share.

Source: https://www.nytimes.com/2024/10/02/technology/openai-valuati...

I think for anyone who has been intimately involved with this last generation of "unicorns", it really does not look like an "exit" if that means bankruptcy, end of the company, betrayal of goals, or what have you: for a high-growth startup of approximately ten years, finding liquidity for employees is perhaps counter-intuitively a sign of deep maturity and that the company is concerned for the long haul (and that investors/the market believe this is likely).

At about the ten year mark, there has to be a changing of the guards from the foot soldiers who give their all that an unlikely institution could come to exist in the world at scale to people concerned more with stabilizing that institution and ensuring its continuity. In almost every company that has reached such scale in the last decade, this has often meant a transition from an executive team formed of early employees to a more senior C-team from elsewhere with a different skillset. In a world context where the largest companies are more likely to stay private than IPO, it's a profoundly important move to allow some liquidity for longterm employees, who otherwise might be forced to stay working at the company long past physical burnout.

> In a world context where the largest companies are more likely to stay private than IPO

Which world is this?

From a Scottish Mortgage report: "companies are staying private for longer and until higher valuations".

    Amazon founded in 1994
    1997 listed at $400m
    Google founded in 1998 
    2004 listed at $23bn
    Spotify founded in 2006
    2018 listed at $27bn
    Airbnb founded in 2008
    2020 listed at $47bn
    Epic games founded in 1991
    2022 unlisted value $32bn
    Space Exploration founded in 2002
    2022 unlisted value $125bn
    ByteDance founded in 2012
    2022 unlisted value $360bn

The only unlisted business there that could be considered among “the largest” is ByteDance. But they have a whole China thing going on, so not very representative of US/European markets.

I doubt any competitor to the largest businesses in US//Europe that is actually putting up good audited numbers is staying private. Even Stripe has been trying to go public, but it doesn’t have the numbers for the owners to want to yet.

SpaceX hasn’t tried to go public , arguably being private has helped them ability to raise a lot of money from investors is key to their successes.

Their kind of product development [1] needs a long term thinking which public markets will not not support well

[1] ignore all the mars noise, just consider reusable i.e. cheap rockets and starlink

It’s never just about the company’s numbers, but the financial environment. When interest rates come down enough to tempt investors out of the money market, you’re likely to see a new wave of IPOs.

Agreed. OP is crossing some wires. More companies are staying private right now (not by choice). Not necessarily or only the “largest companies”

Agreed. These liquidity events are a way to retain and reward early employees while allowing the founders to recover their personal finances and buckle down for the growth phase.

His recent statement that AGI may "only be a few thousand days away" is clearly an attempt to find a less quotable way of extending estimates while barely reducing hype. History generally shows that estimates of "more than five years away" are meaningless.

4000 days is just a few thousand days, but it's [checks notes] over 10 years away.

He knows as well as anyone about the 5 year hueristic (length too long to be meaningful), so he's trying to say "over 5 years", but still hold onto the sense of "meaningful estimate". My sense about Sam is that he trusts his PR instinct to be usefully manipulative enough that he doesn't think he has to plan it carefully in order for his statements to be maximally opaquely manipulative. (He speaks cleverly and manipulatively without a lot of effort.)

"a few thousand days away" feels like such a duplicitous way to phrase something when "a few years" would be more natural to everyone on the planet. It just seems intentionally manipulative and not even trying to hide it. I've never been an anti-ceo type person but something about Altman sketches me out

I don't know anything, but isn't Sam already rich? And, if OpenAI is in a position to capture profits in an expanding AI industry, isn't this the best possible time to lock-in equity for long term gains and trillionaireship?

For many people, sadly, one can never be rich enough. My point is, planning for both short term exit, and long term gains, is essentially the same in this particular situation. What a boon! Nice problem to have!

For some people the question "how much?" has only one answer – "more."

And this seems to be mostly positive feedback.

Sam was already a billionaire years ago. He is one of SV most prolific investors, with equity in hundreds of startups. He writes big checks as well from time to time, Helion for instance he gave 375 million.

He also got weird facial plastic surgery and looks like a wanted mobster now.

To be clear, Altman invested $375 million in Helion Energy's 2021 funding round. Helion Energy a for-profit fusion energy company of which he is the chairman.

He is the Executive Chairman at Helion, which means he keeps a tight grip on that particular purse string.

How? He did not have a large exit to give him a bankroll to invest. Unless you mean from whatever YC shares he received as President, which seems unlikely. Even if he was paid ~25 million in equity per year, did YC shares increase 10x in that time?

Didn't he own like 10% of Reddit. Here what pg had to say about the guy. --- Sam Altman has it. You could parachute him into an island full of cannibals and come back in 5 years and he'd be the king. If you're Sam Altman, you don't have to be profitable to convey to investors that you'll succeed with or without them. (He wasn't, and he did.) Not everyone has Sam's deal-making ability. I myself don't. But if you don't, you can let the numbers speak for you. --- https://paulgraham.com/fundraising.html

One of those quotes where I can't tell if it's a compliment or not.

I hear "He will find a way to not only survive but to thrive in difficult situations"

But I also hear "He will eat people to get ahead".

My understanding is Altman joined the Three Comma Club only this year not years ago.

Yes, he has made lots of investments over the years as head of YC but not every investment was successful. This was discussed on BBC's podcast 'Good Billionaire Bad Billionaire' recently.

He is tied for last place on the most recent Forbes Billionaire list.

https://www.forbes.com/profile/sam-altman/

For people that don't get the references, this graph is so helpful for understanding the world.

https://en.wikipedia.org/wiki/Gartner_hype_cycle

It just keeps happening over and over. I'd say we are at "Negative press begins".

I don't really follow Altman's behavior much, but just in general:

> If he sincerely thought AGI was on the horizon he wouldn't be eyeing the exit

If such a thing could exist and was right around the corner, why would you need a company for it? Couldn't the AGI manage itself better than you could? Job's done, time to get a different hobby.

If such a thing was right around the corner, the person who controlled it would be the only person left who had any kind of control over their own future.

Why would you sell that?

I'm not a believer in general intelligence myself, all we have are a small pile of specific intelligences. But if it does exists then it would be godlike to us. I can't guess at the motivations of somebody who would want to bootstrap a god, but I doubt that Altman is so strapped for cash that his primary motivator is coming up with something to sell.

Let’s say you got AGI, and it approximated a not so bright impulsive 12 year old boy. That would be an insane technological leap, but hardly one you’d want running the show.

AGI doesn’t mean smarter than the best humans.

For an intelligence to be "General" there would have to be no type of intelligence that it did not have access to (even if its capabilities in that domain were limited). The idea that that's what humans have strikes me as the same kind of thinking that led us to believe that earth was in the center of the universe. Surely there are ways of thinking that we have no concept of.

General intelligence would be like an impulsive 12 year old boy who could see 6 spatial dimensions and regarded us as cartoons for only sticking to 3.

Humans can survive in space and on the moon because our intelligence is robust to environments we never encountered (or evolved to encounter). That's "all" general intelligence is meant to refer to. General just means robust to the unknown.

I've seen some use "super" (as in superhuman) intelligence lately to describe what you're getting at.

Super feels to me like it's a difference in degree rather than in kind. Something with intelligences {A, b, c} might be super intelligent compared with {a, b, c}. i.e. more intelligent in domain A.

But if one has {a, b, c} and the other has {b, c, d} neither is more or less intelligent than the other, they just have different capabilities. "Super" is a bit too one-dimensional for the job.

I don't think using set notation is really helping your case here but (I think?) I agree.

No. AGI is talking about the "general" human intelligence rather than specialism. An AGI would be as good as a human at both composing poetry and playing chess, both identifying bird calls and proving mathematical theorems. We don't know what a categorically superior intelligence would be so we can't rate some machine on that basis.

The Lem story "Golem XIV" concerns a machine which claims it possesses categorically superior intelligence, and further that another machine humans have built (which runs but seems unwilling to communicate with them at all) is even more intelligent still.

Golem tries to explain using analogies, but it's apparent that it finds communicating with humans frustrating, the way it might be frustrating to try to explain things to a golden retriever. Lem wrote elsewhere that the single commonality between Golem's intelligence and our own is curiosity, unlike Annie, Golem is curious about the humans which is why it's bothering to communicate with them.

Humans (of course) plot to destroy both machines. Annie eliminates the conspirators, betraying a hitherto un-imagined capability to act at great distance, and the story remarks that it seems she does so the way a human would swat a buzzing insect. She doesn't fear the humans, but they're annoying so she destroyed them without a thought.

All it would want to do is talk about Minecraft and the funny memes they saw.

This is great insight. Science fiction does a great job inspiring technology research, but we forget that real-world accomplishments can be much humbler and still astonishing all the same.

I'm a bit tired of the hype surrounding LLMs, but all the same for very mundane and humbler tasks that require some intelligence modern LLMs manage to surprise me on a daily basis.

But it rarely accomplishes more than what a small collection of humans with some level of expertise can achieve, when asked.

Or accomplishments can genuinely be astounding.

Surely, the LLM models we have today are astounding by any measure, relative to just a few years ago.

But pronouncements of how this will lead to utopia, without introducing a major revision of economic arrangements, are completely, and surely intentionally/conveniently (Sam isn't an idiot) misleading.

Is OpenAI creating a class of stock so everyone can share in their gains? If not, then AGI owned by OpenAI will make OpenAI shareholders rich, very much to the degree its AGI eliminates human jobs for itself and other corporations.

How does that, as an economic situation, result in the general population being able to do anything beyond be a customer, assuming they can still make money in some way not taken over by AGI?

Utopia needs an actual plan. Not a concept of a plan.

The latter just keeps people snowed and calm before an historic level rug pull.

If it's stuck at 12 year old level intelligence then it's not generally intelligent. 12 years can learn to think like 13 year olds and so on.

A set of weights siting on your hard disk don’t evolve to get smarter on their own. AGI does however require the ability to learn new skills in new domains; if you kept exposing it I don’t see why it wouldn’t be similar. But fundamental capacities in how good it is a reasoning / planning don’t have to exceed any given human to be considered AGI.

A 12 year old that doesn’t sleep, eat, or forget, learns anything within seconds, can replicate and work together with fast as light communication and no data loss when doing so. Arrives out of the womb knowing everything the internet knows. Can work on one problem until it gets it right. Doesn’t get stale or old. Only had emotion if it’s useful. Can do math and simulate. Doesn’t get bored. Can theoretically improve its own design and run experiments with aforementioned focus, simulation ability, etc.

What could you do at 12, with half of these advantages? Choose any of them, then give yourself infinite time to use them.

And then with advancement, reaches teenage maturity and tells everyone to fuck off.

As someone who has teenagers right now, I can confirm that this is accurate.

Why do we assume an AGI would not be forgetful or would never get bored?

Because we can change the hardware or software to be what we want. It can write to storage and read it, like we can but faster.

A 12 year old could determine that "AI" is boring and counterproductive for humanity and switch off a computer or data center. Greta Thunberg did similar for the climate, perhaps we need a new child saint who fights "AI".

A 12 year old that doesn't interact with the world.

Many believe that AGI will happen in robots, and not in online services, simply because interacting with the environment might be a prerequisite for developing consciousness.

You mentioned boredom, which is interesting, as boredom may also be a trait of intelligence. An interesting question is if it will want to live at all. Humans have all these pleasure sensors and programming for staying alive and reproducing. The unburdened AGI in your description might not have good reasons to live. Marvin, the depressed robot, might become real.

>simply because interacting with the environment might be a prerequisite for developing consciousness.

We can't even define what consciousness is yet, let alone whats required to develop it.

We may not be able to fully define Consciousness, but we can observe it in humans and develop falsifiable hypotheses, of which there are several.

Take a look at this paper: https://osf.io/preprints/osf/mtgn7

I’m not sure, but it’s possible Stephen Hawking would have been fine with becoming digital, assuming he could keep all the traits he valued. He had a pretty low data communication rate, interacted digitally and did more than most humans can. Give him access to anything digital at an high speed and he’d have had a field day. If he could stay off Twitter.

> AGI doesn’t mean smarter than the best humans.

Technically no, but practically...

12 year old limitations are: A. gets tired, needs sleep B. I/O limited by muscles

Probably there are more, but if 12 year old could talk directly to electric circuits and would not need sleep or even a break, then that 12 year old would be leaps and bounds above the best human in his field of interest.

(Well motivation to finish the task is needed though)

What stops it getting more intelegent? That's litterally its primary aim. Its only limit as to how quickly it does that is hardware capacity, and I'd be shocked if it didn't somehow theorise to itself that it can and should expand its knowledge in any way possible.

Why would the only limit be hardware capacity? Can’t there be an innate limit due to the model size/architecture? Maybe LLMs can’t be superintelligent because they are missing a critical ability which can’t be overcome with any amount of training? It’s not obvious to me that it’s going to get smarter infinitely.

The limit could also be in the training data. If we could train LLMs with a corpus generated by some super intelligent entity, maybe we'd consider them super intelligent even with the same model size/architecture we have today.

"why would you need a company for it? Couldn't the AGI manage itself better than you could?"

Well, you still have to have the baby, and raise it a little. And wouldn't you still want to be known as the parent of such a bright kid as AGI? Leaving early seems to be cutting down on his legacy, if a legacy was coming.

As other comments have mentioned, AGI doesn't mean god-like or super-human intelligence.

For these models today, if we measure the amount of energy expended for training and inference how do humans compare?

> For these models today, if we measure the amount of energy expended for training and inference how do humans compare?

My best guess is 120,000 times more for training GPT-4 (based on claim it cost $63 million and that was all electricity at $0.15/kWh and looking only at the human brain and not the whole body).

But also, 4o mini would then be a killowatt hour for a million tokens at inference time, by the same assumptions that's 50 hours or just over one working week of brain energy consumption. A million tokens over 50 hours is 5.5 tokens per second, which sounds about what I expect a human brain to do, but caveat that with me not being a cognitive scientist and what we think we're thinking isn't necessarily what we're actually thinking.

I did a similar calculation a few weeks ago:

Humans consume about 100W average power (2000 kcal to watt hours/24 hours). So 8 billion people consume ~800 GW. Call it 1 TW. Average world electric power generation is 28000 TWh / (24*365 hours) ~3 TW.

Altman's AGI is Musk's full self-driving car.

If we figure out AGI, that still doesn't mean a singularity. I'm going to speak as though we're on the brink of AI outthinking every human on earth (we are not) but bear with me, I want to make it clear we're not going jobless any time soon.

For starters, we still need the AI (LLMs for now) to be more efficient, i.e. not require a datacenter to train and deploy. Yes, I know there are tiny models you can run on your home pc, but that's comparing a bycicle to a jet.

Second, for an AGI it meaningfully improve itself, it has to be smarter than not just any one person, but the sum total of all people it took to invent it. Until then no single AI can replace our human tech sphere of activity.

As long as there are limits to how smart an AI can get, there are places where humans can contribute economically. If there is ever to be a singularity, it's going to be a slow one, and large human AI vompanies will be part of the process for many decades still.

Let's go with the no next gen models, then we'd be looking at an operationalized service. I might be up for paying $20/month for one. Certainly more value than I get from Netflix. All I would want from the service is a simple low-friction UI and continued improvements to keep up with or surpass competitors. They could manage that.

The long-term problem may be access to quality/human-created training data. Especially if the ones that control that data have AI plans of their own. Even then I could see OpenAI providing service to many of them rather than each of them creating their own models.

I doubt $20 a month is going to cover it. How much would you.pay before you weren't getting a good deal?

If we’re talking about inference costs alone (minimal new training) I think it would cover it. I started doing usage-based pricing through openrouter instead of paying monthly subscriptions, and my modest LLM use costs me about $3/month. Unless you think that the API rates are also priced below cost.

According the New York Times [1], OpenAI told investors that they're costs are increasing as more people use their product, and they are planning to increase the cost of ChatGPT from $20 to $44 over the next five years. That certainly suggests that they're selling inference at a loss right now.

[1] https://www.nytimes.com/2024/09/27/technology/openai-chatgpt...

My bigger concern isn't the financial cost, it's that the amount represents a degree of energy expenditure. A flat rate might not be advisable and having usage-based pricing may limit energy waste. OTOH I don't know how much energy some of the other recurring things I consume monthly are either.

Maybe we can have a service that's an LLM with shared/cached responses as well as training on its own questions/answers for the easy stuff. Currently my usage is largely as a better search engine as google has gotten so much worse than it ever was, though I imagine I'll find custom things it's useful for as well.

The thing is, you can be economical. You don't need a GPT-4 quality model for everything. Some things are just low value where a 3.5 model would do just fine.

I never use the $20 plan but I access everything via API and i spend a couple of dollars per month.

Although lately I have a home server that can do llama 3.1 8b uncensored and that actually works amazingly well.

What’s your home server specs? I might want to host something like that too but I think I’d need to upgrade.

An old Ryzen CPU (2600 IIRC) and Radeon Pro VII 16GB. Got it new at a really good price.

It works ok but with a large context it can still run out of memory and also gets a lot slower. With small context it's super snappy and surprisingly good. What it is bad at are facts/knowledge but this is not something a LLM is meant to do anyway. OpenWebUI has really good search engine integration which makes it work like perplexity does. That's a better option for knowledge usecases.

I use Llama 3.1 8B Instruct 128k at home and that pretty much covers all my LLM needs. Don't see a reason to pay for GPT-4.

Yeah it's good, right?? Amazingly good. The first-gen small models were a bit iffy but Llama 3.1 is so good <3

The only thing I see is that it hallucinates a lot when you ask it for knowledge. Which makes sense because 8B is just not a lot to keep detailed information around. But the ability to recite training knowledge is really a misuse of LLMs and only a peculiar side-effect. I combine it with google searches (though OpenWebUI and SearXNG) and it works amazingly well then.

Oh this is great! Currently I don't incorporate web search into the UI I use, will give OpenWebUI a try.

Yeah, and realistically once we can get hardware powerful but cheap/energy efficient enough to run llm + TTS + ASR without any noticeable delay during a conversation then who needs cloud services for most stuff. The really big models will still be useful, but really only for specific things.

Theres never going to be quality data at tge scale they need it.

At best, theres a slow march to incremental improvements that look exactly like how human culture developed knowledge.

And all the downsides will remain, the same way people, despite hundreds of good sourxes of info still prefer garbage.

Video data is still very much untapped and likely to unlock a step function worth of data. Current image-language models are trained mostly on {image, caption} pairs with a bit of extra fine tuning

Do you think it matters that there's orders of magnitude less video (and audio) data than text data?

I’m not sure I agree, text is full of compressed information but lacks all of the visual cues we all use to navigate and understand our world. Video data also has temporal components which text is really bad at.

OpenAI has been training on YouTube for years.

Nobody close the the edge of this tech seems to believe that this is true, and the 4o release suggests synthetic approaches work well.

Beliefs and outcomes are different things.

I see it differently. ChatGPT holds a unique position because it has the most important asset in tech: user attention. Building a brand and capturing a significant share of eyeballs is one of the hardest challenges for any company, startup or not. From a business standpoint, the discussion around GPT-5 or AGI seems secondary. What truly matters is the growing number of users paying for ChatGPT, with potential for monetization through ads or other services in the future.

> What truly matters is the growing number of users paying for ChatGPT, with potential for monetization through ads or other services in the future.

If the end goal is monetization of ChatGPT with ads, it will be enshittified to the same degree as Google searches. If you get to that, what is the benefit of using ChatGPT if it just gives you the same ads and bullshit as Google?

I mentioned ads as just one potential avenue for monetization. My main point is that OpenAI's current market position and strong brand awareness are the real differentiators. They're in a unique spot where they have the user's attention, which offers a range of monetization options beyond just ads. The challenge for them will be to balance monetization with maintaining the user experience that made them successful in the first place.

Also, don't forget the recent Apple partnership [1], a very strong signal of their strategic positioning. Aligning with Apple reinforces their credibility and opens up even more opportunities for innovation and expansion, beyond just monetizing through ads. I just searched through this thread, and it seems the Apple partnership isn't being recognized as a significant achievement under Sam Altman's tenure as CEO, which is surprising given its importance.

[1] https://news.ycombinator.com/item?id=40328927

The internet has destroyed the value of user attention and brands; everyone has the attention span of a gnat and the loyalty of an addict. They will very quickly move on to the next shiny thing.

My comment doesn't negate your point. I'm emphasizing the business side beyond the moral considerations. If anything, we've seen that brands which contributed to unhealthy habits, like the rise of fast food and sugary drinks, still hold immense value in the market. User loyalty may be fleeting, but businesses can still thrive by leveraging brand strength and market presence, regardless of the moral implications.

The only reason Sam would leave OpenAI is if he thought AGI could only be achieved elsewhere, or that AGI was impossible without some other breakthrough in another industry (energy, hardware, etc).

High-intelligence AGI is the last human invention — the holy grail of technology. Nothing could be more ambitious, and if we know anything about Altman, it is that his ambition has no ceiling.

Having said all of that, OpenAI appears to be all in on brute-force AGI and swallowing the bitter lesson that vast and efficient compute is all you need. But they’ve overlooking a massive dataset that all known biological intelligences rely upon: qualia. By definition, qualia exist only within conscious minds. Until we train models on qualia, we’ll be stuck with LLMs that are philosophical zombies — incapable of understanding our world — a world that consists only of qualia.

Building software capable of utilizing qualia requires us to put aside the hard problem of consciousness in favor of mechanical/deterministic theories of consciousness like Attention-Schema Theory (AST). Sure, we don’t understand qualia. We might never understand. But that doesn’t mean we can’t replicate.

> High-intelligence AGI is the last human invention

Citation?

...or are you just assuming that AGI will be able to solve all of our problems, appropos of nothing but Sam Altman's word? I haven't seen a single credible study suggest that AGI is anything more than a marketing term for vaporware.

Their marketing hyperbole has cheapened much of the language around AI, so naturally it excites someone who writes like the disciple of the techno-prophets

" High-intelligence AGI is the last human invention" What? I could certainly see all kinds of entertaining arguments for this, but to write it so matter of fact was cringe inducing.

It’s true by definition. If we invent a better-than-all-humans inventor, then human invention will give way. It’s a fairly simple idea, and not one I made up.

> then human invention will give way

What? Would you mind explaining this?

It’s analogous to the automobile. People do still walk, bike, and ride horses, but the vast majority of productive land transportation is done by automobile. Same thing with electronic communication vs. written correspondence. New tech largely supplants old tech. In this case, the old tech is human ingenuity and inventiveness.

I don’t think this is a controversial take. Many people take issue with the premise that artificial intelligence will surpass human intelligence. I’m just pointing out the logical conclusion of that scenario.

Arguably cars have so many externalities they will bankrupt the earth of cheap energy sources. Walking is at least hundreds of millions of years old, and will endure after the last car stops.

Likewise (silicon based) AGI may be so costly that it exists only for a few years before it's unsustainable no matter the demand for it. Much like Bitcoin, at least in its original incarnation.

Cars never got us anywhere a human couldn't reach by foot. It just commoditized travel and changed our physical environment to better suit cars.

I really don't see any reason to believe "AGI" won't just be retreading the same thoughts humans have already had and documented. There is simply no evidence suggesting it will produce truly novel thought.

I don't know whether it's a controversial take or not, but I can't see how if one day the machine magically wakes up and somehow develops sentience that it follows logically that human intelligence would somehow "give way". I was hoping for a clear explanation of mechanically how such a thing might happen.

> Sure, we don’t understand qualia. We might never understand. But that doesn’t mean we can’t replicate.

I’m pretty sure it means exactly that. Without actually understanding subjective experience, there’s a fundamental doubt akin to the Chinese room. Sweeping that under the carpet and declaring victory doesn’t in fact victory make.

If the universe is material, then we already know with 10-billion percent certainty that some arrangement of matter causes qualia. All we have to do is figure out what arrangements do that.

Ironically, we understand consciousness perfectly. It is literally the only thing we know — conscious experience. We just don’t know, yet, how to replicate it outside of biological reproduction.

> All we have to do is figure out what arrangements do that.

That’s the hard part!

> It is literally the only thing we know — conscious experience. We just don’t know, yet, how to replicate it outside of biological reproduction.

This conflates subjective experience as necessary (an uncontroversial observation) with actually understanding what subjective experience is.

Or put another way: we all know what it’s like to breathe, but this doesn’t imply knowledge of the pulmonary system.

Agree on hard. But at least possible.

I think a better analogy would be vision. Even with a full understanding of the eye and visual cortex, one can only truly understand vision by experiencing sight. If we had to reconstruct sight from scratch, it would be more important to experience sight than to understand the neural structure of sight. It gives us something to aim for.

We basically did that with language and LLMs. Transformers aren’t based on neural structures for language processing. But they do build upon the intuition that the meaning of a sentence consists of the meaning that each word in a sentence has in relation to every other word in a sentence — the attention mechanism. We used our experience of language to construct an architecture.

I think the same is true of qualia and consciousness. We don’t need to know how the hardware works. We just need to know how the software works, and then we can build whatever hardware is necessary to run it. Luckily there’s theories of consciousness out there we can try out, with AST being the best fit I’ve seen so far.

Future NeuroLink collab? Grab the experience of qualia right from the brains of those who do the experiencing.

> All while hyping AGI more than ever.

Maybe not, since Altman pretty much said they no longer want to think it terms of "how close to AGI?". Iirc, he said they're moving away from that and instead want to move towards describing the process as hitting new specific capabilities incrementally.

> They dissolved the safety team.

I still don't get the safety team. Yes, I understand the need for a business to moderate the content they provide and rightly so. But elevating the safety to the level of the survival of humanity over a generative model, I'm not so sure. And even for so-called preventing harmful content, how can an LLM be more dangerous than the access of the books like The Anarchist Cookbook, the pamphlets on how to conduct guerrilla warfare, the training materials of how to do terrorisms, and etc? They are easily accessible on the internet, no?

Having a dedicated safety team (or as they called it "superalignment") makes sense if and only if you believe that your company is pursuing an actual sci-fi style artificial superintelligence that could put humanity itself at risk [0]:

> Superintelligence will be the most impactful technology humanity has ever invented, and could help us solve many of the world’s most important problems. But the vast power of superintelligence could also be very dangerous, and could lead to the disempowerment of humanity or even human extinction.

> How do we ensure AI systems much smarter than humans follow human intent?

This is a question that naturally arises if you are pursuing something that's superhuman, and a question that's pointless if you believe you're likely to get a really nice algorithm for solving certain kinds of problems that were hard to solve before.

Getting rid of the superalignment team showed which version Altman believes is likely.

[0] https://openai.com/index/introducing-superalignment/

And importantly it's also a question that would need an answer if the alignment should be with "individual groups" rather than "humanity".

It won't do Sam Altman and friends any good if they are the richest corpses after an unaligned AI goes rogue.

So it would be in their were egoistical self interest to make sure it doesn't.

Don't forget the board coup, the coup-reversal, complete takeover, then co-founders leaving, and Mira leaving

Yes, all of that definitely plays into the sense that not all is well at OpenAI.

Slight nit: a board can't start a coup because a coup is an illegitimate attempt to take power and the board's main job is to oversee the CEO and replace them if necessary. That's an expected exercise of power.

The coup was when the CEO reversed the board's decision to oust him and then ousted them.

Yes thats a better way to say it

i disagree that it looks like an exit, i think altman is here for the long haul. he's got a great platform and a lot of AI competitors are also preparing for the long term. no one cares about who has the best model for the next year or two, they care who has it for the next 20.

I think you're projecting a short-term cash-out mentality here.

Altman's actions are even more consistent with total confidence & dedication to a vision where OpenAI is the 1st and faraway leader in the production of the most valuable 'technology' ever. Plus, a desire to retain more personal control over that outcome – & not just conventional wealth! – than was typical with prior breakthroughs.

> you're projecting a short-term cash-out mentality

I'm a software engineer comfortably drawing a decent-but-not-FAANG paycheck at an established company with every intention of taking the slow-and-steady path to retirement. I'm not projecting, I promise.

> to a vision where OpenAI is the 1st and faraway leader in the production of the most valuable 'technology' ever

Except that OpenAI isn't a faraway leader. Their big news this week was them finally making an effort to catch up to Anthropic's Artifacts. Their best models do only marginally better in the LLM Arena than Claude, Gemini, and even the freely-released Llama 3.1 405B!

Part of why I believe Altman is looking to cash out is that I think he's smart enough to recognize that he has no moat and a very small lead. His efforts to get governments to pull up the ladder have largely failed, so the next logical choice is to exit at peak valuation rather than waiting for investors to recognize that OpenAI is increasingly just one in a crowd.

[0] https://lmarena.ai/

Even if you were right in his intentions, can you outline the scenario where Sam is able to make an exit?

I don't see how it could happen at current valuation. Who would want to pay $150 bil. for an OpenAI, where Sam is no longer part of it?

> where Sam is no longer part of it?

It can be an "exit" in that it makes Sam extremely wealthy while he still hangs on as CEO.

You're thinking of an IPO?

Sure, that's one option. But it's pretty rare for a founder of a $10+ billion startup to immediately liquidate their shares right after an IPO.

It's also a risky bet. If OpenAI goes public, they'll have a hard time justifying the same or a higher valuation than they got Microsoft and Nvidia to buy in at.

I think your biographical testimony more confirms than refutes my suspicions.

You think "smart" implies a low chance of world-epochal AI breakthroughs that would ruin your "slow-and-steady path to retirement". You think OpenAI has "no moat". You think it's wise to satisfice – taking a "decent" paycheck – rather than shoot-for-the-moon (and beyond it, control of the 'lightcone').

That's closer to the "short-term exit" mindset – get a good-enough payout, in mundane dollars, while you can – that you're alleging in Altman than what I'm seeing in his statements & behaviors. Yes, someone with your beliefs would in fact be trying to pump OpenAI's short-term hype for a timely exit and an even-more-cushy retirement.

But Altman's already a billionaire with any creature comfort he'd want, for the rest of his life, plus at least one "doomsday bunker" (per a 2016 'New Yorker' profile).

For at least 10 years, he's been consistently expressing a desire to advance, and control, breakthroughs in AI, AGI, and ASI that could – in a true believers' worldview – wipe away as obsolete all his prior billions – as well as the "slow-and-steady path[s] to retirement" pursued by others.

He knows as an insider and contributor OpenAI's actual lead over others – who have tended to approximately-reach OpenAI's capabilities a year or more later.

That you rate as OpenAI's "big news this week" their "Artifacts" end-user feature, rather than everything else announced & rolled out, plus OpenAI's gigantic leads in funds raised and revenue run-rate, suggests a particular world-view.

I would suggest that you are over-focused on the kind of small-stakes products/markets/ambitions that are highly-legible to a "software engineer comfortably drawing a decent-but-not-FAANG paycheck at an established company" – but such considerations are barely afterthoughts in the game of power & legacy that Altman is now playing.

Strong agree and strong upvote.

An intuition pump is to think of a religious zealot that has a drastically different worldview and aims and drives from you that seem completely nonsensical (e.g. perhaps not caring about wealth accumulation at all - even for stability - and only about perpetuating God's glory).

Sam is no religious zealot, but the difference in the life game he is playing and the life game most of us are playing is as vast as that between us and the zealot.

Every single word he says Public is his product.

And he doesn't even use the shift key!

Actually, his blog is correctly capitalized. Although, you would think he could hire a proper editor, since the em-dashes are not formatted correctly. Baby steps.

> ... 4o and o1, which are great step changes but nothing the competition can't quickly replicate.

Where did that assertion come from? Has anyone come close to replicating either of these yet (other than possibly Google, who hasn't fully released their thing yet either), let alone "quickly"? I wouldn't be surprised if it's these "sideways" architectural changes actually give OpenAI a deeper moat than just working on larger models.

Claude 3.5 Sonnet is at least as good as 4o.

Much better than 4o, in my experience. I've stopped using 4o almost completely, except for cases where I need good tooling more than I need good performance (PoCs and the like).

it's better at coding for me, and it looks like because it was trained on more data. i have this problem with a library that i use, and it was trained on python2 version but python3 version is completely different. Python3 versions is relatively new, but out from at leaset 2020 and you would mostly find examples with Python2 version if you googled.

They both produce garbage code in this solution. Claude versions is just 20% less garbage, but still useless. The code mixes those two, even if i specify i want the python3 version or directly specifying a version.

Oh, sorry I wasn't clear, I was referring to the advanced (speech-to-speech) voice mode, which for me is the highlight of this "omni" thing, rather than to the "strength" of the model.

I thought that might be the case. On that: even though other big players haven't copied that particular feature, I doubt (though I'm not by any means an expert) that it's anywhere near as hard to replicate as the fundamental model. I do agree that at least product-wise, they're trying to differentiate through these sort of model-adjacent features, but I think these are classic product problems requiring product solutions rather than deep tech innovation. Claude Artifacts, GPT Canvas, speech, editor integration, GitHub integration, live video integration, that kind of stuff.

It's funny and somehow disappointing how a year ago HN jerked off to Altman and down-voted any critical opinions of him. Just shows no social platform is free of strong group-think and mass hysteria.

It’s almost like people changed their opinions based on observed behaviors.

Yeah, everything from OpenAI in the last year suggests they have nothing left up their sleeve, they know the competition is going to catch up very soon, and they're trying to cash out as fast as possible before the market notices.

(In the Gell-Mann amnesia sense, make sure you take careful note of who was going "OAI has AGI internally!!!" and other such nonsense so you can not pay them any mind in the future)

Dissolve the safety team. You just made everyone stop reading the rest of your post by falsely claiming that

I don’t even know how this fake news started

Maybe the fact that they actually did dissolve the safety team formerly led by Ilya Sutskever in the aftermath of Altman's coup [0]? I'm genuinely unsure what part of this you're questioning.

[0] https://www.bloomberg.com/news/articles/2024-05-17/openai-di...

Like literally the third sentence makes it clear it was restructuring team dynamics but not eliminating safety goals or even losing the employees that are safety focused. There are many safety focused people at OpenAI and they never fired or laid off people in a safety team which is what people infer from “dissolve the safety team” when it’s used as a bullet point of criticism

It's pretty obvious to most of us that "integrating the group more deeply across its research efforts to help the company achieve its safety goals" is the corporate spin for what was actually "disperse the group that thought its job was to keep Altman himself accountable to safety goals and replace them with a group that Altman personally leads".

Eliminating Ilya's team was part of the post-coup restructuring to consolidate Altman's power, and every tribute to "safety" paid since then is spin.

https://www.merriam-webster.com/dictionary/dissolve

Definitions of dissolve:

to cause to disperse or disappear

to separate into component parts

to become dissipated (see DISSIPATE sense 1) or decompose

BREAK UP, DISPERSE

Those seem like pretty accurate descriptions of what happened. Yes, dissolve can also mean something stronger, so perhaps it is fair to call the statement ambiguous. But it isn’t incorrect.

Reply on news.ycombinator.com