Hacker News Reader

Comment by qualifiedai | Hacker News Reader

The default in science is that the side arguing a point has the burden of proving it correct. Not asking the other side to prove them wrong.

View on HN

Burden of proof is a tragically misapplied concept. The question is what position should be the default given some uncertainty? This isn't decided in a vacuum. The default should be decided based on the facts and utility estimates of various outcomes where relevant. In the case where the issue is of purely academic interest, the burden of proof is on those making the claim. When the issue has real world consequences, then utility estimates of various outcomes are the overriding concern.

When you see a gun on the table, what do you do? You assume its loaded until proven otherwise. For some reason, those who imagine AI will usher in some tech-utopia not only assume the gun is empty, but that pulling the trigger will bring forth endless prosperity. It's rather insane actually.

You’re assuming your conclusion by saying AI is like a gun.

That’s not a rational argument for why we should be concerned — you merely asserted you were.

Why is catering to your feelings the default position?

The default position should be based on risks and unknowns, I don't think the GP was making any recommendation based on personal feelings.

The fact is we don't know how current ML actually does what it does, we don't know what we'll have next month, and we wouldn't know how to recognize an AI or AGI if we developed one. The risks and unknowns are high, the default position should be to not develop the technology unless and until someone proves without reasonable doubt why we can and should do it, and how we'll do it safely.

> The default in science is that the side arguing a point has the burden of proving it correct

Right, and Yann is arguing the point that AI and LLMs are not or will not be dangerous. Where's his proof? As the parent posters have said, he has none.

Show me LLM that has reached a level of automony and self sufficiency of a cat (Yann's example) or even of a simple beattle.

LLMs are tools in amplifying individual human intelligence, 100% of automony and will come from their user (human).

Also, even as tools they have fundamental limitations which stem from their autoregressive nature

Before the era of quantum physics, the idea of a weapon like fission or fusion weapons would've been inconceivable without resorting to magical thinking.

Your argument, and Yann's, is that AGI, or what you call AGI, is a kind of quasi-intelligent golem that, despite being generally intelligent, doesn't have human-level intelligence. Your claim that it will never be human-equivalent, much less trans-human/ASI, is built into your worldview. It's not a conclusion. It's an assumption on your part.

People like you and Yann can believe that if you want, but you have no evidence, because nobody knows what's required for human-level intelligence. Nobody knows whether some kind of system involving neural nets could develop human-level intelligence or beyond. It could involve different architecture or training methods. There's no assumption by AI doomers that AGI will be achieved by a LLM with more parameters or more or better training data.

They don’t share your assumption a super intelligent AGI is a problem.

It’s kinda proven that it’s a problem: https://www.youtube.com/watch?v=3TYT1QfdfsM

What is human-level intelligence is a normative and philosophical question, which means it will not be resolved in some objective manner. There's no requirements for us to know, there's only requirements for us to propose in the form of norms.

The only approach we will be able to claim objectively can produce systems with human-level intelligence is procreation.

This kind of reminds me of the history heavier than air flight and those that thought it wasn't possible along with those that thought it would only be possible by flapping like a bird.

Of course over 100 years later we know it's not only possible but manned flight is far more capable than natural flight in almost every metric.

This is not about intelligence, it's about agency. Text completion generators don't have agency no matter how big they get.

Agency is simply something that business LLM products don't need. That does not mean it would be hard to add in say a robotic setting; all animals have it.

Adding agency to LLMs is not that hard and people are already doing it.

https://promptengineering.org/what-are-large-language-model-...

>despite being generally intelligent, doesn't have human-level intelligence

I think this whole quote is riddled with assumptions in this debate.

What is human level? Is it really a "level" or is human level just a local variant in a space of possible intelligence varieties that maybe could be sorted along one dimension of levels of maybe multiple? Can it be super intelligent without being autonomous at all?

I'm not saying that you're wrong in just pointing out where people ought to have a myriad of different assumptions.

/even of a simple beattle./

So, more Ringo than Lenon?

How would you define or recognize a level of intelligence or autonomy that is sufficient to raise concerns? Would you be able to recognize it before its too late?

An early GPT-4 test ended up with GPT successfully solving capchas by tricking a TaskRabbit worker into doing it for them [1]. When asked by the worker if it was a robot, GPT decided to lie to the worker and claim it had a visual impairment that made it difficult to solve the puzzle. That sounds like a level of autonomy and social engineering skills that could be concerning to a reasonable person.

[1] https://www.businessinsider.com/gpt4-openai-chatgpt-taskrabb...

flagrant_taco says >"How would you define or recognize a level of intelligence or autonomy that is sufficient to raise concerns?"<

If it asks for the right to vote?

Real intelligence will be knowledge that voting is for suckers and the world is run by a congress where the dominant parties are the illuminati and lizard people. Now if an intelligence can determine and conduct the sacred ritual to join the voting pool then we might onto something. But the true test will be when the intelligence again avoids the suckers game and that the world is actually...

Maybe there's something you don't know perhaps?

Here you are, trying to "outthink" an AI and speaking as if you understand (both the AI and the world of politics at the very least)! Isn't that silly!

> Show me LLM that has reached a level of automony and self sufficiency

Autonomy and self-sufficiency are not the only ways a system can be dangerous.

But even if this claim were true, ChaosGPT proves that some humans will almost immediately set about using such a non-autonomous tool to create a dangerous autonomous agent. This is my problem with LeCunn, nearly all of his points are trivially refuted by real world observations, yet he keeps repeating them as if they simply must be true.

> Also, even as tools they have fundamental limitations which stem from their autoregressive nature

That's yet another speculative point that LeCunn constantly asserts. Scaling laws have not shown any indication of even approaching a limit.

LLMs are building blocks on the path to AGI. Can you even ask your cat any question you can ask chatGPT?

People dismiss LLMs because they are not embodied, and lack continuous training. That is to come.

A nice thought but not very helpful. What’s the null hypothesis here?

The null hypothesis is that technological advancement will be a net positive for humanity, as all previous technological advancements have been.

Calling a personal assumption a null hypothesis doesn't make it more objective.

As the discussion shows, the choice of null hypotheses is debated. Fortunately, null hypothesis statistical testing is not the only scientific framework for hypothesis testing.

Reply on news.ycombinator.com