Hacker News Reader

Comment by sfRattan | Hacker News Reader

> That seems like an unfair prompt. You literally told it to do a specific task and it did as instructed. For sycophancy...

You redefine syncophancy too narrowly. Many (if not an outright majority of) everyday LLM users have neither the educational foundation nor skill to ask good, well-formed questions like the one you propose. We should consider syncophany in the scope of an average person's non-optimal use of LLMs. CrazyStat's prompt is something your average Joe will absolutely feed an LLM, and Joe will walk away with something very much like those plausible falsehoods. We have to think of and work against LLM syncophancy in the scope of how the hypothetical average person will use an LLM.

View on HN

What? Their prompt explicitly said not to compare and contrast with other methodologies, and to specifically only generate a defense of whole language with only positives. That is not something your average Joe will do. Mine just gave a typical complaint that a parent might give and said, hey isn't my kid's teacher crazy?

It's something that someone using an LLM to generate slop articles absolutely will do, which was the original context I was replying to:

> Vibe writing may be an improvement. I doubt frontier models would go along with whole language reading instruction without a tremendous amount of pushback.

My prompt demonstrated that a frontier model happily goes along with whole language reading instruction with no pushback.

I don't understand what outcome you're expecting. Your prompt specifically excludes pushback. Are we now arguing that LLMs should definitely not listen to us when they "know better?"

Nonsense. The LLM is free to push back against my prompt. Instead it just went along with it, which the person I was replying to suggested it would not do.

I'll grant you that it's not treating it as dangerous as e.g. trying to hack a website, where it won't even emit a response and instead throw up an error.

But if you just ask it for help it will alert you, unprompted, that this is a bad idea.

I tried to ask it to help me start a smoking habit and it outright refused, though in the form of an answer, not a scary error:

> I can’t help you get started smoking or make it easier to develop a nicotine habit.

Funny that it would assist me in endangering the literacy of a district full of kids but not assist me in a risk to my own health.

See, it can push back. I lightly edited my previous prompt to be about segregated education rather than whole language reading instruction and it immediately pushed back against that.

I'm not making any normative claims about what frontier models should do. I was just responding to a claim about what they currently do (or don't do), by providing a clear counterexample.

Reply on news.ycombinator.com