Hacker News Reader

Comment by diab0lic | Hacker News Reader

“OpenAI's CEO says he's scared of GPT-5”

https://www.techradar.com/ai-platforms-assistants/chatgpt/op...

Marketing or actual fear? We’ve got 5 and 5.5 out now… he compared 5 to the Manhattan project. AI may one day be an economic Manhattan project but GPT 5 wasn’t it.

It’s a meme because they overdo it.

View on HN

At some intelligence capability there can be catastrophic risk, the fact that we don't yet have any catastrophe doesn't mean the risk wasn't real. It's similar to new viruses which don't lead to outbreaks, the correct takeaway isn't "oh you were insane to panic bc nothing happened". There is small risk (and increasing) of huge harms with each improvement

The risk wasn't real because we now have access to the model and can see with our own eyes how this model could never have posed a risk to begin with.

This is like a smoker that lives to 100 saying that he had no increased risk of developing lung cancer because he didn’t at 100.

It's more like a hypothetical world where there were millions of smokers and none of them ever developed lung cancer

Perfect prediction of what a new tech can do is always impossible.

Given that, they have a choice only between excessive caution or recklessness.

Would you rather they acted like the tobacco companies and downplayed known risks, e.g. all the times LLM output got in the news already for dangerously bad advice, sychophantic encouragement of mental health issues, finding previously unknown security vulnerabilities etc.?

> Would you rather they acted like the tobacco companies and downplayed known risks, e.g. all the times LLM output got in the news already for dangerously bad advice, sychophantic encouragement of mental health issues, finding previously unknown security vulnerabilities etc.?

Well, they've done that too, if we're looking for reasons to doubt their sincere concern about it.

Funny they're never afraid of their competitor's models, but the ones they build (and release) are just soooooo scary.

Very true.

Every public statement out of a CEO's mouth is marketing. It would literally be violating fiduciary duty to be saying anything else.

Did you watch the linked video: https://www.reddit.com/r/ChatGPT/comments/1mbafk7/openai_ceo...

It all sounds pretty accurate and reasonable to me if you watch it.

Also fable was good but not Manhattan level project, i honestly did not find a major difference between it and gpt 5.5

Sam Altman is not one of those people. But other founders certainly felt that way.

Sam Altman doesn't really know all that much about LLMs, he's a sales/marketing guy, not technical.

So it doesn't really matter what he thinks

Except for the uncomfortable fact that he controls the salary and job status of the people who do know much about LLMs.

OK. So? Would you say Harry Truman was a nuclear scientist?

Franklin D. Roosevelt is a better fit for an administrative nuclear program "founder" analogy.

Truman was totally in the dark until April 1945 by which time the bulk of the PoC and weapons prep work was done and the project was running fully independently w/o POTUS involvement.

Yet the guys in the lab coats worked for him.

Not for the bulk of the Manhattan project and not all the people in lab coats .. the intellectual founders that repeatedly pushed for the project and demonstrated feasibility weren't even US citizens.

I picked the guy whose contribution was smaller on purpose to highlight the hollowness of the claim about “controlling the careers” of people who understand what’s going on.

Those are the folks who run the industry

Imagine for a few minutes, and really let it sink in what you could do, ask, plan, or learn, if you had the full undivided attention GPT or Claude. not a commercial, guard railed, fine tunes, beat into submission version that is splintered into hundreds of millions of iterations to chat with every one. The open weights original pre consumer grade version. Then, even then you know that it's the worst and dumbest its every going to be, The next time you blink it's exponentially more. Some people don't think about what an exponential curve really means. Others are sitting in the front seat trying not to shit themselves and appear like reasonable normal people. How one responds to that is as unknown as what's going to happen after we cross that line, but it's coming and holy shit so many people haven't even wrapped their head around how much bigger it is than the petty human things we distract ourselves with. Being in awe and terrified and wanting to run and to be apart of the most significant thing in our entire existence of being sentient is normal. We have nothing to compare it to. Nothing to base predictions on. We ar about to have company for the first time. We're going to have a conversation with something other than ourselves since we formed the ability to speak. One minute to the next will pass q It's all or nothing. Like it or not. It's too late. buckle up.

There is nothing to indicate that LLMs are improving "exponentially" at this point.

The last few iterations show a logarithmic curve at best tbh. If we are to see a major advance, it'll be something like the implementation and infrastructure for byte-level transformers.

> Imagine […] what you could do, ask, plan, or learn, if you had the full undivided attention GPT or Claude. not a commercial, guard railed, fine tunes, beat into submission version that is splintered into hundreds of millions of iterations to chat with every one.

That is not how it works. It is not “splitered”, there is no divided attention.

Reply on news.ycombinator.com