Hacker News Reader

Comment by OJFord | Hacker News Reader

> I suspect the reason why most big LLMs have ended up in a pretty verbose spot is that it's easier for users to scroll & skim than to ask follow-up questions

Maybe it's a 'technical' user divide, but that seems wrong to me. I would much rather a succinct answer that I can probe further or clarify if necessary.

Lately it's going against my custom prompt/profile whatever it's called - to tell it to assume some level of competence, a bit about my background etc., to keep it brief - and it's worse than it was when I created that out of annoyance with it.

Like earlier I asked something about some detail of AWS networking and using reachability analyser with VPC endpoints/peering connections/Lambda or something, and it starts waffling on like 'first, establish the ID of your Virtual Private Cloud Endpoint. Step 1. To locate the ID, go to ...'

View on HN

I’ve noticed this as well with coding questions. I will give it problematic code and ask a question about behavior, but it will attempt to reply with a solution to a problem. And even if I prompt it to avoid providing solutions, it ignores my instruction and blasts out huge blocks of useless and typically incorrect code. And once it overwhelms my subtle inquiries with nonsense, it gets stuck repeating itself and I just have to start a new session over.

For me this is one of the strongest motivators for running LLMs locally- even if they’re measurably worse, they’re a far better tool because they don’t change behavior over time.

There’s an interesting discrepancy here.

Human users are charged by the number of messages, so longer responses are preferable because follow up questions use up your message allowance.

APIs are charged by token so shorter messages are preferable as you don’t pay for unnecessary tokens.

My description was of me as a human user of ChatGPT fwiw, not the OpenAI API.

I had it again earlier:

Me: give me a bucket policy for write access from alb

CGPT: [waffle about IP ranges that is totally incorrect; then starts telling me ALB doesn't typically write to S3 because it's usually an intermediary between clients and backend services like EC2 instances or Lambda functions - it already knows from chat context I am using the latter]

Me: [whacks stop because it's rapidly getting out of hand] yes it does for access and connection logs

CGPT: To allow Application Load Balancer (ALB) to write access and connections logs to an S3 bucket, you need to set up a bucket policy that [waffle waffle waffle]

Me: [stop] yes I know that's what I asked for

CGPT: Here is an example of an S3 bucket policy [...]

Me: invalid principal [as far as I can tell, a complete hallucination]

CGPT: [tries again]

Me: yes I already tried that, valid policy but ALB still doesn't have permission

CGPT: [nonsense intensifies]

In the end I sorted it much quicker from AWS docs, which is sort of saying something, because I do often struggle with them. Thought I'd give ChatGPT a chance here but it really wasn't helpful.

Reply on news.ycombinator.com