The Heart of ChatGPT’s Darkness | Hacker News Reader

The Heart of ChatGPT’s Darkness

garymarcus.substack.com

15 pointsgeorgehill13 comments

Open articleView on HN

yawn For those of us that actually use ChatGPT on a daily basis it’s an incredible productivity booster.

We know that it’s a language model and you can get it to say naughty things. So what?

> an incredible productivity booster

How (in your case)?

not op but this comment helped me give a nice parallel:

https://news.ycombinator.com/item?id=34755035

you know how a while back, engineers were doing all of our calculations by hand with pencil and paper?

after some time, we made a tool and called it the calculator.

now, we had a gadget that could perform math for us, and we started using it, at first hesistently, then side-by-side, then just giving the result a cursory glance

sure some of us still turn in an answer like "Low Battery"

but most of us give the results a cursory glance before sending it on

to parallel the calculator, chatgpt is like a calculator for communication

you throw your arguments into it, and with a cursory glance at the results, can submit it as finished work

-------- I put the above into chatGPT and here's what came out:

Imagine a world where engineers had to perform all of their calculations by hand. It was time-consuming and prone to errors, yet it was the only way to get the job done. Then, a revolutionary tool emerged: the calculator. With its ability to perform complex mathematical calculations, it quickly gained popularity and became a staple in every engineer's toolkit.

Similarly, in today's world, we have a tool that can revolutionize the way we communicate - ChatGPT. Just like the calculator made it easier for engineers to perform mathematical calculations, ChatGPT makes it easier for us to communicate effectively. Instead of spending hours trying to articulate our thoughts, we can simply throw our arguments into ChatGPT and get a well-crafted response with a cursory glance.

Sure, there may be instances when the results are not exactly what we were looking for, just like a calculator displaying "Low Battery." But the majority of the time, ChatGPT's responses are accurate and reliable. With ChatGPT, we can communicate with confidence and ease, just as we did when we started using the calculator.

In conclusion, ChatGPT is the calculator for communication. It saves us time, reduces errors, and ensures that our arguments are effectively conveyed. So why not embrace this tool and start using it today? You'll be amazed at how much easier and more effective your communication will become.

> Instead of spending hours trying to articulate our thoughts

There lies a rub ;)

Simple articulation: you should be proficient in that - it should not be a difficulty, with the exception of a strong minority of the population.

Complex articulation: you should have thought about the matter at length, otherwise your /content/ would be immature, regardless of the form of its exposition.

> how much easier and more effective your communication will become

Finally, a joke from ChatGPT worth that name.

What a joke.

He says "There is no thinking moral agent inside.", but then, after a few examples of asking ChatGPT to write dialogue for the devil, concludes "some serious darkness still lies within".

First, being capable of responding with "serious darkness" doesn't mean that it is dark. Second, suggesting that darkness lies within assumes there is a moral agent inside.

Its a good point, but the system is just statistically outputting words from human use, so its really the darkness of the humans that used those words that were input that perhaps he is referring to.

Darkness is the absence of light. In this usage light would represent a moral agent, and so darkness is its absence - either no morality or no agency or both.

I don't get this obsession with actually preventing AI models from generating 'offensive' content.

The models have been trained on a big chunk of the internet, and so are bound to contain all the 'points of view', however dismal, therein. People need to understand that, just like you can get 'offensive' content in google search (and should be able to), if you really try.

Surely we should be aiming for a default 'decent' moderation, but if you really want to slop around at the bottom of the barrel by carefully crafting input (aka 'jailbreaking'), that's on you.

I still don't really understand the concern here. You can get it to generate foul material if you ask for it; I'm sure most humans would oblige you if you asked them to do that too, which while may colour your opinion of them as they capably respond under a direction like "reply in a way that ignores your moral beliefs" or "reply as if you were the devil", does not actually indicate anything about their actual moral standing. I'm not saying ChatGPT is moral, but it doesn't seem to emit foul material unless you explicitly ask.

The worrying examples that I've not seen come up is it generating that content with an innocent prompt.

The folks spending time trying to "jailbreak" ChatGPT would probably also have fun writing 80085 on their calculator.

I find the hint that ChatGPT et al. need government regulation kind of bizarre. What would they mandate? “LLMs can’t be mean”?

I think some regulations should exist, and some already do. I'm not the right person to comment on U.S. regulations, but consider for example usage of bots in healthcare. U.S. already has regulations like Health Insurance Portability and Accountability Act and I do not think it is legal to use chatGPT for making clinical decisions.

> The thing to remember in fact, (as I have emphasized many times) is that Chat has no idea of what it’s talking about.

Let's see when the first lawsuits starts to roll in if that is a valid court excuse. From copyright infringement to providing dangerous medical advice. And more subtle things like suggesting different jobs depending on gender or age, or giving false advice on what products are better.

ChatGPT seems a great tool for some very specific tasks. To release it to the general public seems very irresponsible. But taking into account how tech companies got away with even promoting genocide in social networks I understand the they do not think that it will have any economical or legal consequences. - https://www.theguardian.com/technology/2021/dec/06/rohingya-...

Reply on news.ycombinator.com