goatlover

> There will hopefully (?) come a time one day when human beings provide only ultimate value judgments, and everything else is done by machines.

In their previous grandparent post. They do reserve a place for human value judgement, but I doubt even that remains if everything else has been automated. The machines will decide what we value. We already see that to some degree with algorithmic engagement and targeted advertisements.

Like killing all of humanity? Agreed, we're not prepared for that.

I'm on the side of humans remaining special, at least on planet Earth. My strong sense is that a large majority of the public feels that way, and voting results in the future will start to reflect that regarding AI regulations.

I agree with Frank Herbert's view of "thinking machines".

Why do you want machines to replace human intelligence and ability? I find that dystopian. AI could have meant Augmented Intelligence (which was proposed a long time ago), not let's see when we can replace all human activity.

One is humanist, the other is anti-human, (in the end goal at least).

I'd be impressed, followed by asking if they needed to go to the hospital.

5. This is a science fiction story. No such thing exists in the real world and the people claiming it will soon are not experts in the relevant fields of how it would kill everyone.

Hype objects when it comes to the idea that they will soon become recursively self-improving, boot strapping themselves to super intelligence, at which point we will be at their mercy. Because that's basically the p(doom) argument, that some probability is associated with ASI killing us all.

That's different from what we have today with powerful software tools that people can use for all sorts of things, which is an extension of what we already had in less concentrated form. It makes things easier for humans to do good and bad things, but it's not an autonomous digital god.

It's bad that a few people are making such a gamble on behalf of billions. That could work if climate change were literally an asteroid about to strike the Earth and AI was the only thing that could figure out how to alter it's course before then. But it doesn't really work when climate change will unfold as a catastrophe over the next few centuries. Because then it becomes a decision the entire planet should be making over time, not a few tech billionaires and their friends in government.

Now apply this logic to human slavery. The "good impacts" are what profits the slave holders, what makes their lives easier at the expense of the enslaved. Apply it to child labor and labor exploitation in general before labor laws. Then you'll see one side is behaving in a greedy manner, ignoring the humanity of the other side.

Who says we don't need humans for it? Are non-mathematicians going to be guiding agents to solving pure mathematical problems?

Most people work for a living. Then there are people who find meaning in work. And people who advance in life by working. You want to do away with all that?

It was needed before the LLMs and training data existed in digital form. Euclid, Gauss, Turing didn't have that benefit.

Is there a proof that every possible proof is a generation in the grammar of lean or rocq? Sounds like an unsubstantiated claim.

Devil worship? Who makes a comment like that on HN?

The Iranian regime being worse doesn't justify bombing a girl's school! That's a war crime. Pointing it out isn't "devil worship".

At least politicians are somewhat beholden to their constituents. What keeps Sam and Dario in line other than profit?

Superstring theory has been based on the idea of mathematical beauty, but it's run into serious complications when it comes to empirical results. Science is empirical first and foremost, because we need to verify that theories actually predict the universe we live in by testing them.

I don't know how LLMs would help sorting through the 10^500 possible universes when it comes to superstrings. You need to be able to run experiments or obtain observations.

The market is not able to fix corruption. Regulation is needed. For example, exploitative labor practices weren't ended in the early 20th century because of customer preference. Laws had to be passed.

This is just trickle down economics, which has led to great wealth inequality while the middle and lower class wages have stagnated as inflation has gone up and more people have to rent instead of owning their own porpoerty. What really helps is strong social safety nets and programs that help struggling people. The Nordic model leads to better societal outcomes.

This is just trickle down economics, which has led to great wealth inequality while the middle and lower class wages have stagnated as inflation has gone up. What really helps is strong social safety nets and programs that help struggling people. The Nordic model leads to better societal outcomes.

Always wondered what Skynet would have done had it killed off the humans in the Terminator future. Would it build a machine society?

There are some acceelerationists who want "superior" machines to replace humanity. But what if instead they get the equivalent of a paper clipping gray goo, or a bored Skynet who just fights it's own creations? Is that worth replacing all the humans?

As a thought experiment, what is a p doom AI doing after humanity is gone?

You can cut the power to data centers where the models are run.

Maybe we shouldn't want that to happen?

The singularity has been termed, "Rapture of the Nerds" since the futurists promoting it sound like evangelicals talking about the end times, both with the Revelations doom stuff, and the utopian stuff that follows.

Do the experts in the relevant fields agree with the "substantive" technical details on how the entire human race can be made extinct?

This sounds like a rationalization for gambling with everyone's lives, assuming they really believe such a thing.

> they believe only they can be stewards of digital gods

You do't see how absurd that sounds? Not only the idea that they can create gods, but that they are the only good stewards for such beings. It's also incredibly arrogant.

As bad as it would be, it's unlikely that a full scale nuclear war kills all humans across the planet.

What's the evidence for there being an existential risk? We are provided scenarios which read like science fiction about RSI and ASI right around the corner, resulting in magic sounding technology that can do anything the person making the claim wants it to do, because it can just make itself smart enough in a short amount of time. But the person making the claim has to actually show how such a thing is possible in the real world, not just a story.

They should also be able to give detailed reasons experts in the relevant fields can verify as to how they arrived at a 10% chance all humanity goes extinct. Not a science fiction narrative which likely does not take into account the relevant physical facts limiting such scenarios. Such as how exactly an AI would build a bioweapon capable of killing 8 billion humans across the planet.

Humans have agency and often choose to do what their parents didn't set them up for. Humans also have legal rights. They're not tools owned by corporations. This is an absurd comparison.