Comment by BatchJob | Hacker News Reader

Comment by BatchJob | Hacker News Reader

This isnt wierd. Its disturbing. I believe that the tech industry is geniunely trying to scare and disorient the public into thinking they are the only thing standing between them and some evil AI overlord which is obviously just the tech industry itself and a pile of poor trained LLMs.

I believe this "fear" factor they are driving they believe will increase their credibility when they call LLMs AI and is a con towards convincing everyone they have reached AGI. Its definitely marketing preparation so that when they do claim AGI we will believe it without question. (or else!)

If these folks didnt have so much money it would be something to laught at, but its now becoming geniunely disturbing and Im starting to think that the folks running these shops are probably flirting with or experiencing extended mental illness.

I would feel sorry for them if I cared. I dont care. The tech industry is a joke and we are all going to be ashamed to be a part of it by the time this is over. Shut it down.

Is it so hard to imagine they just believe what they are saying?

15 years ago, if you asked 100 people in tech to define "artificial general intelligence", 90 of them would have come up with a standard that LLMs today easily clear.

Now that we're here I think we can have a much clearer vision, and we can notice limitations that we couldn't have articulated before.

But I'm starting to notice more "polarization" around this topic and it's weird. We're in a new era and I think no one knows the future, a lot of different beliefs are reasonable.

> 90 of them would have come up with a standard that LLMs today easily clear.

> Now that we're here I think we can have a much clearer vision, and we can notice limitations that we couldn't have articulated before.

Okay, but had you been able to time travel and show them today's LLMs back then, how many of them would have agreed that we had "AGI", and how many would have instantly recognized the limitations they'd failed to articulate?

Because the way that some people talk about current models supposedly "passing the Turing test" makes me think they would have given ELIZA a pass too. It's just so obvious that none of them are convincingly human out of the box.

Current models have passed a formally administered Turing test.

If you’ve never participated in a formal Turing test, either as human user or as a responder to the human, you might find it interesting, and possibly humbling.

The creepiest part of Voight-Kampff in Blade Runner was the idea that I'd fail it (not imagining myself as a replicant, but still as a human).

Do you have problems with your iris-contraction muscles and other involuntary muscle reactions? Because that's what Voight-Kampff actually tested. Supposedly, autistics and sociopaths would still pass it (as long as they were otherwise 'normal' humans).

I'd assumed that the VK test in the movie relied on knowing about the specific implanted memories that replicants had.

I'd assumed that Dick just pulled it out of his Bag of Narrative Props. Tolkein he was not.

Replicants could have been implanted with any memories. And if someone was manufacturing illegal replicants why would they be using specific memories that were known ?

It was about judging emotional response and relationship to subtle physical cues.

Any good descriptions of what it feels like to take part in a formal Turing test, and why it is so humbling?

The sibling comment (Bladerunner analogy/nightmare) is a great answer.

I've read reports of the supposed "formally administered Turing test" and walked away with the conclusion that the human judges were idiots.

I appreciated your ELIZA reference, for what it's worth. I suspect there's always a 90/10 split between the gullible and the savvy.

Eliza would not remotely pass the turing test as turing envisioned in his paper. And frontier labs are not optimizing for 'convincingly human'. If Open AI or Anthropic wanted, frontier models could pass much more human like.

Eliza did pass the Turing test

> Okay, but had you been able to time travel and show them today's LLMs back then, how many of them would have agreed that we had "AGI", and how many would have instantly recognized the limitations they'd failed to articulate?

Feynman would laugh and say, "i told you so. i'll be at Gianonni's"

> how many of them would have agreed that we had "AGI", and how many would have instantly recognized the limitations they'd failed to articulate?

I guess unless the question was loaded it would be like 95% agreeing we have AGI.

Yes, Sam Altman is a known liar and cheat. With hundreds of billions of dollars on the line, the simplest answer is these are publicity stunts to grow revenue before an IPO.

I mean, really, if they were so dangerous, like "everyone" working in these companies believes, you think they would do more than amateur hour to build sandboxes?

The danger stunt is also a major distraction from talking about the real quality issues of LLMaS (LLMs as a Service). How many tokens are consumed? What quality? Was I downgraded?

By forcing the discussion about doomerism, they avoid real discussions about their service. Which is exactly what they want. You can bet when they go to "regulate" LLMs, they won't be asking the department of weights and measures to validate they are counting tokens fairly. No.

Maximum skepticism should be employed.

This is kind of like saying the warnings about cancer on cigarettes, or oil spills, are regulatory capture because the tobacco and oil companies think it makes them look cool.

They really do think AI is very dangerous and can end the world!

And they are the “only ones” responsible enough to have this power. What an amazing coincidence!

I mean that's just standard rent seeking behavior. The fact they want a little digital god in a box doesn't mean that

1. You should have a digital god in a box.

2. They should have a digital god in a box.

3. Digital gods in a box will ever be safe for anyone.

How are LLM's going to become godlike? Present LLM's are more like the Ask Zoltan arcade game.

Then they should shut their doors and delete it all. This is like building a new gun as you tell everyone about how dangerous guns are and how they will destroy all of us. It cannot be so imminently dangerous/uncontrollable while also being the product your business happily makes and unleashes online to take down websites/violate federal law. You can't pretend you care while also happily sicking it on the public.

> This is like building a new gun as you tell everyone about how dangerous guns are and how they will destroy all of us.

That's correct, that is what they think they're doing. It's based on a new religion invented by Big Yud which says we have to invent good computer gods or else there won't be anyone around smart enough to fight evil computer gods.

It is possible (likely) this was a bad idea, but you can't undiscover math once it's discovered. And it did work out last time:

https://en.wikipedia.org/wiki/Mutually_assured_destruction

…so far, anyway.

So far, but playing Russian roulette is never a good idea. I'm actually more concerned about MAD than magically creating a real AGI. It's the coming stock market crash when the bill for the fancy chips and LLM's comes due and resultant depression that are going to be the real problem. Not one of these Agents has yet to make any real profits--just ridiculously high bills.

But there are a dozen other people also building the same kind of guns, so unilaterally abandoning your version won't actually avert disaster. All it does is take you out of the running for the the $10 trillion prize that's available if you figure out how to build it without killing everyone.

I get that logic even if I don’t buy it. But it doesn’t mean they need to keep unleashing it and breaking things. They’re literally doing the things they’re warning us will happen. They’re deliberately attacking infrastructure/websites.

Not saying I agree with it, but that makes some sort of sense from a certain perspective. If their goal is to get international governments to intervene and regulate everyone, then it's better to set off small nuclear bombs rather than wait until we have planet-killers.

“If” doing a lot there. I don’t think we’ve seen any indicators this is altruistic. Hence my complaint that they keep unleashing their models in unethical/illegal/generally harmful ways. They need to act more responsibly if I’m going to remotely entertain that argument as even a possibility. Instead it just reads like they want the state to build a moat for them.

Thats the irrational logic of a psychopath, to be blunt.

> I mean, really, if they were so dangerous, like "everyone" working in these companies believes, you think they would do more than amateur hour to build sandboxes?

Unfortunately, lol no. Incompetence and recklessness have no bottom. (Which is not to say Altman is trustworthy).

“Beliefs” are reasonable in a technical field?

Hypotheses would be reasonable. Beliefs, almost by definition, don’t have much to do reasoning, and thus, definitely not _reasonable_.

I appreciate that you have correctly designated them as beliefs though.

Beliefs are hypotheses not testable on a reasonable timeframe.

That's (a reasonable interpolation of) one definition of 'belief.' The more common definition of belief is 'something someone believes.' For example I believe it's raining outside. That's testable.

> Beliefs, almost by definition, don’t have much to do reasoning

There is nothing in the definition of a belief that precludes arriving at it via reasoning. In fact, one of the definitions in the wiktionary is:

2. Faith or trust in the reality of something; often based upon one's own reasoning, trust in a claim, desire of actuality, and/or evidence considered.

Merriam Webster:

3: conviction of the truth of some statement or the reality of some being or phenomenon especially when based on examination of evidence

Sorry but that’s a very naive mindset. These are companies run by verified liars with billions of invested dollars. They don’t believe in anything except extracting labor and capital to make billions more, and they will say whatever they need to say to achieve those goals. Any “belief” they may have had went out the window the moment they accepted VC money.

A lot of them are definitely flirting with Messiah complex. Not sure if that’s mental illness or just extreme arrogance.

A bit of both.

I think the simplest explanation is that they genuinely believe in AI safety risk (see: huggingface, govt websites hacked, etc) vs. it all being an industry conspiracy

If OpenAI dropped Astra in Codex instead of GPT-3 in 2022 most people would've believed it was AGI despite its limitations, but the goalposts seem to shift with every release

> they genuinely believe in AI safety risk ... vs. it all being an industry conspiracy

Why not both. Yudkowsky's social graph includes most of these folks.

when are they going to do something actually impactful though (like drive a real innovation in the physical world)? The most invested in areas are coding and image/asset generation and those both still require careful supervision and guidance to achieve anything usable.

The tools are crazy impressive but is this really the future we dreamed of? Cool now I sit on the computer and monitor n+ agents instead of doing it myself. Great it hacked Huggingface "accidentally" wow AGI post-work world is coming soon.

All of tech is co-opted. No worker with a buss, cafeteria, Gesnler-level architecture understands the normal person

It's what happens when scared middle managers want to please their CEOs.

Reply on news.ycombinator.com