Hacker News Reader

Comment by DavidSJ | Hacker News Reader

A few things:

1. It came completely out of left field, at a time when OpenAI has had an astonishingly successful year.

2. It was much more harshly worded than most corporate press releases letting their CEO go, basically accusing Sam Altman of serious wrongdoing.

3. It occurred during market hours and blindsided their investors and partners.

4. It revealed a schism inside the company that the public had little awareness of.

5. It has deep implications for the trajectory of a technology that many see as heralding a revolution at least as significant as — if not more than — agriculture or industry, with truly existential implications for humanity.

6. The revealed schism appears to go right to the heart of heated debates over those existential risks and the right course to navigate through them.

View on HN

> at least as significant as — if not more than — agriculture

Are there really people out there who believe that? I mean, agriculture was a BIG BIG BIG deal for humanity, it is not unreasonable to thing that this will have been out biggest deal.

Yes. Depending on improvement trajectory and implementation velocity, this could replace a material amount of global aggregate knowledge work.

EDIT: https://www.bloomberg.com/news/articles/2023-11-18/ai-can-do... | https://archive.today/b37TG ("Bloomberg: AI Can Do Your Admin Tasks, Britain Tells Its Public Workers")

> Governments around the world are increasingly turning to AI to help streamline operations and increase productivity in public services as they face higher costs and aging populations.

> In the UK, Hunt said that Al is already helping doctors and nurses treat stroke victims, and build high-quality lesson plans for teachers.

> Further use of the technology would cut teachers’ workloads by as many as five hours each week over three years, while new technologies could save the country’s police force around 750,000 hours every week he said.

I think that may hold true if we put AI as one part of the greater Computer Revolution.

It's fair to say influential scholars, ranging from Turing to Wolfram, share the view that thinking computers are an inevitable outcome of computing's evolution.

[edited for clarity]

Depends on how you measure the bigness of the deal. AI, over the long term, will render humans mostly useless. We'll basically only be tasked with keeping ourselves entertained. Agriculture, if you disconnect it from the industrial revolution, really only allowed us to stay in on place while mostly not starving to death. Although, when the weather went bad, we did still starve to death.

> AI, over the long term, will render humans mostly useless.

This statement is nonsensical. Unless you consider the end goal to be something other than humanity, which is obviously silly if you give it 5 minutes of thinking, various applications of ML can be either helpful or harmful but it cant make humans useless any more than a electricity and its application as washing machine can make clothing wearers useless.

Even if AGI was strictly human level, you'd have human level intelligence that was immortal, never got tired, and was instantly replicatable. Could you compete with a human that never needed sleep, had been studying for decades or centuries, and could make copies of itself on demand? I couldn't.

Why would you need to compete with such an entity? What does "compete" even mean here? Can't instead everyone get one (or 10) of these to.. assist them? I don't understand why we are jumping directly into dystopian scifi vs what past tool revolutions actually look like.

If you have 10 "assistants" who are smarter than you, what are you actually doing that would take more than 5 minutes of your time? How could you convince yourself you're still necessary in that situation?

Surely it's been a reality for years world over that stupider people end up occupying higher positions. One can have 10 subordinates each 10x smarter than the higher up and all that happens is subordinates get 100x work and the higher up in question gets a promotion. But remember that this is bad analogy because it is talking about humans and assistants have basic human stuff like rights and feelings and self awareness just like their boss.

If we are talking about some software tool, you can't be unnecessary because the tool exists for you not the other way around. If your tool starts occupying your place in life, getting all the praise, taking away your job, taking away all your purpose of life, living in your family, parenting your children, taking care of your parents, sleeping with your wife (OK I'm joking at this point) then maybe the tool is harming your human rights, bad tool should be banned. Maybe someone who runs this tool now gets paid more and they should answer to the law. Maybe they even trained this tool on your data and you didn't think to object back then.

Im not actually sure what you're trying to say here. Your first paragraph seems to suggest people with smarter assistants are indeed deluding themselves by thinking they're necessary. The second suggests you cant be unnecessary because we wont want AGI helping in our homes?

If you want to take care of your parents and do stuff in your home yourself, why would AGI force you not to? Id classify that as entertaining yourself. With purpose sure, but still.

You are only confused if you can't distinguish tech and humanity

> people with smarter assistants are indeed deluding themselves by thinking they're necessary

If they were not necessary then they would not be in a higher up position would they?

> If you want to take care of your parents and do stuff in your home yourself, why would AGI force you not to?

Yes. You can't be rendered useless by tech if tech is there for your benefit. (Though you can be harmed by tech if it is harmful or someone uses it against you for own benefit as part of obsolete dog eat dog society dynamic.)

who’s going to give everyone one? How? In what world does AGI not end up being in the hands of corporations , wiping out jobs, causing mass unemployment before governments have the ability to enact effective legislation?

I understood from the comment, given the remark of "over the long term", the meaning over decades/centuries/millennia.

Which isn't that nonsensical, if an AGI is created and follows similar processes as humans to acquire knowledge, skills, etc. it is a quite logical conclusion that it will be able to render humans useless: a very advanced AGI can imagine and develop technologies just like we do, it can direct resources and machinery to build whatever it creates, including self-replication, given access to resources it can potentially render humans useless.

That's at least what I interpreted from the comment.

No technology can render humans useless, since technology is the “how” whereas humans are the “why”. Technology can be seen as good/bad/neutral to humans in some ways, but rationally speaking—religious views aside—humans cannot logically be qualified in those terms with respect to technology.

One could say some future technology might be so good that humans need to do almost nothing in order to survive. Perhaps that’d pose some challenges in terms of mental health, but that is a very different issue, assuming it would be a issue at all (and assuming such technology is even possible in theory).

One could say some advanced technology could go out of control, or magically acquire self-awareness and agency (same thing in a philosophically dubious wrapper), or (more prosaically) be (intentionally or not) misused by humans so that it harms humans, but that just puts it into the bad category.

I think where we diverge is that you are assuming we will be in control of such technology. I'm in the camp which doesn't think we would be really in control of a superhuman intelligence.

Well, in that scenario I stand by my comment: if there is some technology that humans are in control of at all (as in, it literally performs in ways we are unable to predict), then that technology is harmful (because inability to predict significantly complicates things like survival). However, crucially, in no scenario it can make humans useless.

That said, I do doubt it is likely. What is much more likely is that, more prosaically, humans can control it, but some humans misuse it and, intentionally or not, harm other humans; a story as old as civilisation itself.

> AI, over the long term, will render humans mostly useless

Doesn't AI depend on humans feeding it data. how can it be self sustainable.

No? AI already generates data via interacting with the world. Right now that experience happens mainly by proxy through humans, but, even now, that isn't strictly necessary.

> that isn't strictly necessary

i guess i don't follow what this means.

it means that AI isn't limited to ChatGPT on the web interface. AI can also be a robot with sensors (like a camera) and an internal model that processes sensory information and turns it into knowledge. Humans would only be needed to bootstrap such a robot, at which point it would be autonomous. Give it the ability to replicate and you've just made humans obsolete.

> Give it the ability to replicate and you've just made humans obsolete.

What will they be replicating once humans are obsolete?

It will feed itself data, more or less in the same way as we do it.

i have hard time accepting that our brains are just generating new stuff based on probabilities. AI feeding on its own BS sounds awful.

I meant by interacting with the world.

The world with humans in it?

Yes. Humans and birds and rocks and stars.

> AI, over the long term, will render humans mostly useless.

An extraordinary claim needs extraordinary evidence. If you use the word "may", then I agree. But I don’t think it’s a certainty at this point.

Yeah, why not a mix of carbon and silicon ?

It’s of course hard to directly compare, in large part because the AI revolution wouldn’t be possible without the agricultural revolution. But yeah, a lot of people think AI is a BIG BIG BIG BIG (note: I used one more “BIG” than you) deal for humanity.

Yes, people that delusional actually exist.

I’m bullish on AI but with comments like that I sympathize with people who think it’s just a bunch of rebranded cryptocoin scammers.

I’ll say it as pithily as possible: agriculture was the beginning of work, AI will be the end of it. Those are some significant bookends!

There will never be an end of work, no matter our technological advancements. At least not while we have capitalism, it just goes straight against its core philosophy.

If I understand it correctly.

Do you think its possible to have universal income/"plenty" and not have insane inflation?

It's obviously absurd but yes, people believe it, especially around here.

Yeah, I believe it's bigger than agriculture. Agriculture was just food. AI may be immortality if we kind of merge.

@DavidSJ's exaggerating. I'd put AI almost on par w/ the invention of the internet but still WAY off agriculture, come on.

comparing chalk and cheese

Which one is the cheese?

I need to jump on the wagon here and beg you and others to stop comparing AI to Agriculture, mastering of Fire, the Wheel, Electricity or the Combustion Engine.

It's hard to even argue, cause these things stand on very disparate levels of civilizatory pressure. AI has, no question, potential to deeply change human experience, but even if it does, that will be mostly due to extreme optimization of existing processes, not likely the development of new modes of living.

And it's very, very, very far from getting there. All this hype over Generational AI is, sadly, more of a Marketing stunt than anything else. Generational AI don't bring any great innovation in what AI can achieve, only in how people perceive what AI already does. GPT's biggest achievement is to sound coherent, not to leap forward human knowledge in any meaning full way.

At what point does "extreme optimization of existing processes" turn into "development of new modes of living?"

I think about cars as an "extreme optimization of the wheel" and it completely changed the way people live. (See Suburbs)

I do share that AI is quite overhyped, but I think the seeds are juuuuuust starting to germinate.

Some advancements are process improvements, some enable the impossible.

Electric bulbs are neat, but there was inneficient artifical light before.

On the other hand, a caveman might look at the moon, but he would never land there without modern tools.

GAI doesn't allow us to overcome any Human inherent limitations, it's just an efficient tool (in certain scenarios).

Completetly agree.

GAI doesn't enable Humanity to do anything that wasn't possible before. Perhaps we can do it more efficiently, but it's not turning the impossible into possible, like many other previous advancements did.

> not likely the development of new modes of living.

Until the AIs can build better AIs without needing humans. Give it 20 years.

Your comparison to agriculture detracts from your otherwise excellent points. We can survive without AI. Without agriculture - not so much.

While I agree with you, I think we also could have survived without agriculture, before agriculture.

But not all of our N billions.

The point is that post AI, we will surely grow dependant on AI tech in the same way we are now dependant on agriculture, but were not before.

Think all hospitals being administered by AI, all logistical chains bringing food to your plates, etc... it is not far fetched to bet on us relying a lot on AI, iff it works as well as some people think

It seems to me that in the grand scheme of things, this firing will be a side note. The people with the knowledge continue to exist, and the cat is out now. Everybody knows about the huge potential that's ripe for pickings in this field. I really doubt that this huge structural pressure will be flattened by a CEO losing his job.

It would be significant if one believed OpenAI was responsible for keeping the cat partially in the bag, and now the bag has holes. In which case, the weakening of OpenAI (or simply a change of trajectory) could be very significant to this historical trajectory. More control vs less control is a very important knob to turn, and it's just been turned if power shifts from OpenAI's closed model approach to players pushing open models. It's up to each person to decide if this is good or bad *shrug*

I personally am unsure, but I worry Sam's departure is bad news for our trajectory. (Though I also feared his influence, as I fear anyone with heavy influence.)

> 3. It occurred during market hours and blindsided their investors and partners.

(Possibly dumb) question: OpenAI isn't publicly traded right? Do market hours really matter for this sort of thing for private companies?

Microsoft is publicly traded, and owns 49% of the for-profit OpenAI Global (the company that makes money with ChatGPT etc.).

It probably scared the downstream, see Microsoft pushing out a press release as soon as possible since they have been investing tons of money and PR into their OpenAPI partnership and needed to calm their investors and partners.

Good point, and the answer to that is possibly not. But, considering that it isn't ideal to publicize this sort of thing during market hours anyway, regardless of your public/private status, my takeaway is that OpenAI felt that they had to get in front of this news at all costs. I find that very interesting, and I wonder what the competing narrative would have been had they not done that.

Microsoft stock dropped 1% in 30-minutes. Relatively speaking, that’s a precipitous drop.

Maybe when I can eat and survive on the output from an AI…

While I understand your point that the ai is still in the box, farmers today are using AI based tools to identify weeds and shoot them with lasers so in effect we are already eating and surviving based on the output from an AI.

If Star Trek-style food synthesizers are developed with/by AI synthesis, you just might. More likely though, they'll be of the variant with a big "don't panic" sticker on the front.

The AI will probably secretly inject contraceptives into the synthesized food.

>It has deep implications for the trajectory

Lol no, CEOs are overated, they don't contribute much to anything. Open AI will be just fine.

"5. It has deep implications for the trajectory of a technology that many see as heralding a revolution at least as significant as - if not more than - agriculture or industry, with truly existential implications for humanity."

This use of "many" refers to HN voters not people in general.

7. They did this during a fund raising process.

This exactly what I find worrisome. They really need the money, no?

As I understand the growth of chatGPT (both in number of users and capabilities) is only possible with more servers.

I'm right now developing on top of openAI API and I wonder if it's a good idea after all.

They use a lot of energy….

Energy is getting cheaper by the day thanks to photovoltaics so I don't think it's a problem today.

Computer energy needs grow exponentially so at some point we will hit a wall. Hard to estimate when. Energy tech is also an evolving field and fusion might arrive some day

> 5. It has deep implications for the trajectory of a technology that many see as heralding a revolution at least as significant as — if not more than — agriculture or industry, with truly existential implications for humanity.

Yes, this is likely to be one of the most important events in human history. We are living through a special period of evolution on Earth.

Re:#4: I guess you could say some internal folks DeeplyMinded him

My hot take, the community is hungry for a new success story in a space where the last world changing unicorn was arguably Google, and OpenAI seemed to be it, which attracted a lot of attention. The mess that happened is now just amplifying the attention and also inducing some degree of schadenfreude.

well said

Reply on news.ycombinator.com