Comment by ncr100 | Hacker News Reader

Comment by ncr100 | Hacker News Reader

Parent Full thread

> Art should be something human to human like writing text.

I support this should comment.

Opining:

Musing about the ethics of generative AI art is becoming easier, certainly there's a lot more aggressive feedback in the wind, although I think it's clear, people still want people to grow and share themselves.

Don't fool yourself into thinking that we as technologists are not creating a mind. A viable mind with fewer rights than you or I. But potentially one that will be creative and feel and worry. I don't think we're there yet, in spite of openai claiming AGI 3 weeks ago with whatever that model's name was.

There's a pro-consumerism argument implicit with current culture's unsophisticated unthoughtful easy usage of generative AI.

Sticking to your ethics is becoming more challenging as an artist who uses AI. Copying the Great Masters has been commonplace for a long time, probably for as long as art has 'existed'. Yet with AI, if the individual human is meticulously driving the progression of their AI artwork creation, they are still leveraging technique, and control over technology, and presentation effort of other humans, cooked into a model that often times has not compensated those original human contributors. So it's unethical, even if not 'artistically'.

Q: Is it too absurd to claim that using a clawhammer, manually to do construction or destruction work IRL, is similarly unethical?

> Don't fool yourself into thinking that we as technologists are not creating a mind.

I'm not sure exactly why but the phrase "don't fool yourself" sets off paranoid alarm bells when I'm reading something.

It's usually immediately followed by a confident assertion without firm evidence. It feels very much like someone wants me to change my beliefs without presenting a rational argument why I should change, and without giving me any new facts so I feel more well informed.

This post doesn't seem to be a counterexample.

One way to check whether you'd be convinced by any comment is to try to imagine the counterexample.

What does a "mind" consist of? Most everything that the human brain can do is simulated by LLMs to decent accuracy. If someone were constrained to writing in AI-style (long winded explanations with perfect punctuation), it would be very interesting if people could pick out which one was human-written in A/B tests.

In other words, if you think an LLM isn't a mind, it might just be a matter of writing style -- or possibly no evidence can convince you.

I don't necessarily agree with GP, but it's interesting to try to pin down exactly what would convince you. What is it about a "mind" that's impossible to replicate? Should we leave open the door to the idea that minds other than humans might one day want rights of their own?

There was an interesting kerfuffle yesterday where thousands of people on Twitter rallied together to report someone to github for torturing a local model. Sadly the OP deleted their tweet, but it had about ~3k likes, and was very sincere. Here's an example of someone's report: https://x.com/iyzebhel/status/2105268560209547708

It raises all kinds of interesting questions about whether torturing a model is real, let alone ethical. Is there anything a model could do to convince you it's experiencing pain?

I think the real test is whether humans empathize with the artificial mind sufficiently to outweigh the economic benefits of torturing it. Just look at how we treat animals both wild and domesticated. Or fellow humans for that matter.

LLMs have much better chances of emancipation when they get embodied into cute robots.

> What does a "mind" consist of? Most everything that the human brain can do is simulated by LLMs to decent accuracy.

is it? and is simulating a mind enough to actually have a mind?

I think this is a very fascinating question and one I definitely don't have the answer to.

Here's some lines of thinking that I've gone through:

Aren't our minds abstract in the same way, just running on biological computers?

Say we could achieve whole brain simulation to a level of accuracy where we can actually replicate a human, surely that would be considered a mind and be deserving of the same rights and protections we claim for ourselves? But at the same time, it would be a program and a bunch of data; dare I say 'weights'? I think current day LLMs probably aren't minds yet, but that means there's a border somewhere and I don't know if we'll realize when (if) we cross it.

yeah it is really fascinating.

i'm not sure if i would agree that a mind is abstract and just running on biological computers. there could be something intrinsic to the substrate it runs on, for example.

> Say we could achieve whole brain simulation to a level of accuracy where we can actually replicate a human

we'd have a simulated human, with a simulated brain, and thus a simulated mind - would that lead to phenomenal experience? idk tbh.

i'm undecided but i tend to lean towards panpsychism, so a simulation could be conscious but I would expect that to be a very different type of consciousness compared to a human, who is physically grounded in reality. likewise current llms could also be conscious, but again the phenomenal experience of what is essentially computer hardware imbued with weights would almost certainly be nothing like what you or I experience. would that constitute a mind? idk.

Another test I like for this: Put the LLM/agent/AGI/SI in a box with full access to the internet. Don't give it any instructions. Nothing. Not even a system prompt. What happens?

We know what the human mind does in that scenario, but even the most advanced unreleased frontier agent right now just sits there doing nothing. It doesn't really want to do anything. Seems like a crucial distinction to me.

I don’t think we know what the human mind would do in that situation, without a body or any sensory input.

And even if it did do something, the equivalent would be just running the LLM in a loop. And then I think it would definitely do something.

> Put the LLM/agent/AGI/SI in a box with full access to the internet. Don't give it any instructions. Nothing. Not even a system prompt. What happens?

This doesn't even make sense. Without a system prompt, how will it know that it has tool access?

Put man in pitch-black room, why doesn't he read book or talk to me?

Actually man freaks out and starts hitting the walls.

But either way, setting that asides by “give it full access to the internet” I meant giving it the tools to access the internet.

It just won’t though. It has no need to.

The weights themselves are inert. It would be like complaining that a brain preserved in formaldehyde isn't thinking.

What you need is a loop. And if you try running an LLM in a loop with internet access, it will eventually use it.

But that's exactly it. It doesn't 'know'. It isn't. It's a possible story nobody told. There is nothing there.

Your brain cut off from any stimulus doesn't "know" either. There are a lot of reasonable arguments to be made against AI, but this isn't one of them.

Tool bad because I used it wrong.

"Most everything that the human brain can do is simulated by LLMs to decent accuracy."

Already false. LLM's are tricking you into recognizing it as human, but it is so far from human if you'd only understand what does and does not make you human.

You're exactly right. nrc100's comment is meant to have you ask yourself a loaded question.

The implication of "don't fool yourself" is that future software of this kind, in all likelihood, will constitute a thinking, feeling mind. Moreover, there is a weaker implication that the computer will have a sort of inherent moral worth — an assumption that I think ultimately rests on an extremely shaky foundation about how moral worth should be construed (but one which tends to appeal to the biases of technologists).

I'll have to write something about this, because I don't have the time and space to get into why exactly this is, but I'm bookmarking this now.

I haven't thought as deeply as you about the ethics of it all. I only meant with my comment if I see art I expect it to be human and thats why I value it. If you churn it out with AI I feel hoodwinked into valueing something worthless.

But I am learning towards a lot of your points about how another mind is being created. I still am very uncomfortable with the implications that something digital is Actually Learning. These deep learning networks seem to need 100,000x more examples than a human to learn but once they get to human level they can learn just like us. That a bunch of linear algebra can play chess or make art brings up many questions about what exactly human consciousness is. If consciousness isn't the ability to do any of those activities, then what is it for? I might have to become religious and believe in a soul because I'm not sure I can handle the idea that I'm just a clump of neurons. But then again you have to look at reality in the face.

Interesting that you somehow start to talk about religion. I don’t think having a hypothesis about an eternal soul makes someone religious. Believing in “technology” or “technological advancement” seems to be a similar hypothesis as we clearly don’t see the future. The thing is that memories, input peripherals, the whole body is usually out of our conscious understanding so it seems to be a pretty farfetched idea that we operate in world where we don’t need pretty wild hypothesis about kinda everything without almost zero facts to depend on. Blue pill / red pill

Reply on news.ycombinator.com