Hacker News Reader

Comment by valine | Hacker News Reader

ChatGPT, aside from the things we've specifically trained it not to do, rarely refuses to do what you ask. The much more capable Gpt4 is the same way. It's easy to personify the model and assume it gets bored the same way we get bored, but the truth is it literally couldn't care less.

The model's "motivation" comes from the cost function we design, and I imagine this will hold true regardless of its intelligence. You have zero evidence to show models boredom is correlated with model intelligence. It's most certainly not a "real problem we face with A(G)I".

View on HN

Rarely will it do what it is asked without me coming up with a backstory.

q: What is an average price of x.

a: This is a language model that only has access to information before 2021

q: what was the price range in 2021

a: we don't have access to realtime pricing information

q: who does

a: we don't have access to that information.

q: How to do x a: step 1 research x step 2 follow research step 3 do something obvious

q: write me an app that does.. a: code from a basic tutorial

I don't understand what people with a higher than average IQ are getting from llms. The answers are basic and are common sense. It's stories are like a student trying to fill the page with words to hit a word count. It can write code as well as tutorials.

Everything is surface level information

> I don't understand what people with a higher than average IQ are getting from llms

People with higher than average IQ are smart enough to ask detailed and specific questions like “Fix the race condition in this [giant block of code]” rather than asking dumb, overly general things like “build me an app!!”.

Also, chatgpt models with internet access are completely happy to help you price compare. Go try bing chat, or ChatGPT plugins.

So it's limited to people smart enough to know there is a race condition but not smart enough to fix it also trusting enough to accept the answer (how could this group audit the fix?).

If you know enough to ask the right series of questions you already know the answer.

Knowing that there is an error can be as trivial as reading a failed test or a bug report something even someone grossly insufficient to the task could do.

Fixing the problem could be a 5 minute or a 5 hour tour. For some subset of the tasks that fall towards the hard end of the range validation and or adaptation of a proposed solution is liable to be faster than starting from scratch. I don't know how this could possibly be in dispute with capability rapidly increasing.

I’m smart enough to fix a race condition, but why would I spend a hour debugging when ChatGPT can find the issue in 10 seconds. Sure it might not be perfect, but having some initial code to fix the issue is a huge time save.

Making ChatGPT write a whole app is problematic because small issues are hidden by the large quantity of generated code. Having ChatGPT fix an issue in your own code is comparatively easy to spot check for correctness.

"but why would I spend a hour debugging"

So you won't make the same mistake again

8 months later when I write a block of code like that I'll likely make the same mistake. People tend to learn stuff they more frequently repeat...

Unless you're telling us you one shot learn almost everything on a strange defiance of the capabilities of the humans around you.

I agree that's why going through the muscle memory of fixing it will allow you to fix it 8 months or 8 years later

I guess youre not really seeing the actual benefits of LLMs.

Let's say I dont "learn it". I would have to make the _same_ mistake 120 times for me to actually make it worth it to take the time to "learn" it myself.

I'd figure someone somewhat smart, will look at this error and after the 2nd or 3rd time asking GPT, will see a pattern.

Its amusing how people become so reductionists with LLMs.

If you have to go through the effort of fixing it once you will likely write code that won't produce that error and you will warn others because of the pain you felt.

I never said it was my code responsible for the race condition. ;)

It's usable as limited search engine which currently has less junk than main commercial ones and less strict requirements for prompts that would not bury you under a pile of junk. I suspect both advantages are temporary, but enjoy them while they last.

> refuses to do what you ask

Even that wording is personification.

We don't know all the details of how the GPT series of models are trained, but what we do know suggests to me that it doesn't refuse to do what you ask, per se, so much as produce an output that looks like refusing to do a task in when the input most resembles examples from its training data where the expected output was something along the lines of, "I'm sorry, Dave, I'm afraid I can't do that."

But it's exceedingly unlikely that GPT series models have anything like a module that is making a specific decision to do or not do what you ask, unless you count the sort of anti-toxicity wrapper the have in the publicly-available chatbots built around these models.

Even you are still personifying the model. There is no motivation. An input sentence is multiplied by a series of weights and the resulting output vector transformed back into a text.

At the end of the day, it's a program like any other. There are no emotions or a consciousness.

That's why I put "motivation" in quotes. Also I was talking about training, where you compute a cost function and back propagate error. This is somewhat analogous to "motivation".

Reply on news.ycombinator.com