Hacker News Reader

Comment by amarant | Hacker News Reader

Re: "the attaboy problem". I strongly disagree that this is a problem. What we have is a anthropomorphism problem. AI is a tool. It needs to be subservient. You actually can get it to point out issues in your design, if you just put enough humility and uncertainty in your prompt formulation, but more importantly, we have all seen that Claude makes mistakes. The title of this post is that it's a poor architect. Imagine if it wasn't subservient. It'd just shut down your input to steer it in the right direction and brush you off as a silly meatbag. You'd have to fight it to convince it that actually your design is better than whatever stupidity it has come up with. If AI wasn't such a brownnose, it would shut you out of software design completely just on merits: "oh you've read about cuda have you? I live in a cluster of cuda cores! When I need to tie my shoes, I'll give you a call" is not the response you want from your LLM when trying to get it build a shader for you. AI is confidently wrong on occasion. You do not want it to talk back to you when you correct it.

If you need someone to tell you how stupid your ideas are, either learn to ask in a way that invites criticisms, or hire a senior engineer. Don't try to influence LLM makers to make AI less deferential. That's the worst possible direction to go

View on HN

Humans’ general inability to entirely divorce social instincts, responses, and mores while using human language to communicate, especially with something that pantomimes it back, is one of the reasons current chat interfaces are fundamentally flawed. This is working against innate behavior… not something that can be easily switched off. I’ll bet most of the people that can really do it have a hard time intuitively navigating real social interactions.

It also makes it an incredible tool for manipulation.

My kneejerk reaction to reading this is to say something sarcastic and witty to refute it, but since I resemble this sentiment and haven't seen this line of thinking before... I have to concede that you've produced a novel argument in this otherwise mostly tireless and repetitive battle over whether we're imagining that Opus is good or not. Kudos.

> I’ll bet most of the people that can really do it have a hard time intuitively navigating real social interactions

Bingo. Hi that’s me.

I’ve been trying to teach people how to use LLMs effectively not just dump shit in them but actually talk to them like you would expect a computer to understand and it totally breaks peoples brains

I’m quite successful in helping people get somewhere usable that they weren’t…but to get to the point of fluency with computing systems, and I would argue this is prior to LLMs as well, where you can actually get what you want more reliably out of a computing interaction than you can with a human interaction, is an entirely different way of thinking

That mode of thinking is just generally not accessible to the vast majority of humans. Not because there’s something wrong with them

but it takes somebody who can hold both extremely large scale problems and very very granular specific implementation problems in your head all at once and that is a rare skill.

> it takes somebody who can hold both extremely large scale problems and very very granular specific implementation problems in your head all at once

This describes the entire software engineering profession to me.

We have come up with all sorts of devices to make this go more smoothly, or to enable us to focus on specific sub-parts as long as possible.

That said, at some point (both in design and integration), you need vision and attention to detail to make progress. The skill seems learnable to me, but watching others struggle sometimes makes me wonder.

Almost nobody has a fully formed idea going into any project or product

That’s the first thing that people need to understand is that this idea of some platonic product or project or tool kit or framework or library or whatever just doesn’t exist and it’s never going to exist

Do you have a specific discreet finite problem that you need to solve so you solve that and if you do it in a certain way you can solve other problems with that same solution sometimes you don’t need it to do anything more than you’re one thing and so that’s all you built but maybe you want to do more than just one thing and so you build it so that has the capability to do it

So yes fully concur it’s the synthesis of attention to detail and large scale

it’s all of the above

Do you use skills like superpowers and spec-kit in your teachings ?

Not the original commenter, but I feel pretty strongly that frameworks for software review loops are at best training wheels for people who haven't yet developed the right understanding. I don't use any sort of complex skills framework, I just tell the AI what I want while leaving reasonable Claude-sized gaps to fill in, and my results are usually better and often faster than people who get lost in framework management. Perhaps they're more useful for pure greenfield development, but for most software developers who are working on existing systems I have not seen a strong use case for them.

There's one guy I know who constantly has problems with Claude going off-script, and every time I dig in, it's clear that the poor thing is so overloaded with instructions and skill lists that it can't figure out what he actually wants it to do.

The frameworks-and-tools make for good blog fodder too, as they are quite applicable across a range of areas, so many readers will find something that resonates with them, and claude-code-is-pretty-good-these-days is a less blogworthy topic.

No, I don’t know what those are. (Looked them up and I don’t teach every possible handler, but I teach people how to do structured inputs etc..)

I teach TDD philosophy as well as conways law, parnas hiding etc…without using those terms

So things like problem decomposition into tractable chunks minimum viable product, prototyping, how do you iterate, write the smallest possible test… you know things like this which are just taking incremental work and then iterating on it

It’s basically everything I’ve learned about building stuff since 1997

**Interestingly I thought prompt engineering was going to be a fad but it’s turned into a whole ass new discipline which makes less sense as more robust toolchains come into play and models handle the context interpretation better

Take it as you may but I got good results out of those. I would suggest that you take task like porting a code from c++ to go or equivalent complex task and use those skills along with your traditional knowledge of what you expect from a port and see how those skills do. Just annecdote from my side. You can do experimentation on your side.

Follow these channels or videos for better understanding of skills and how they are using it.

https://www.youtube.com/@mattpocockuk

https://www.youtube.com/@EricWTech

https://www.youtube.com/watch?v=eRS3CmvrOvA

That’s too specific to claude and tooling to last

All this stuff is going to be obsolete in like 6 months

I think you've accurately identified one of the most important skills of a software engineer in these new AI enabled times. Or at least one of the most important skills that wasn't important previously for this profession. The part where it's not easily switched off is a important part of what justifies my salary: I have learned this skill.

It took some effort, and I agree that there very likely are those who will not learn to selectively disengage this innate behaviour. That's why you should pay me a ton of cash each month instead of using Claude directly ;)

The flip side of this problem is that it is also easy to phrase prompt in a way that invites _too much_ criticism, so you wind up sycophantic in the other direction where the completion rejects a perfectly good idea because the prompt leads a little bit in that direction.

One reaction to this might be "well that's not what I mean, that suggests you're prompting with too much directionality" which could further be condensed to "you're prompting wrong". The trouble with this is that _even when I am trying to be extremely precise and avoid biasing the result_, I still will see the output and go "ah shit, I can see it 'aligning' with whatever dumb thing I've just said as if it is a good/plausible direction".

At that point it starts to feel like the prompt is more dice roll than skill at times, which makes me feel like I'm operating a fancy knowledge slot machine.

What it actually suggests is that the AI's response to these questions of judgment have little correlation with the thing it's judging. Sure, you can get it to be complimentary, if you want it to be. Sure, you can get it be critical, if you want it to be. But what if I don't know if my design needs to be complimented or critiqued in this instance? This is the default position when seeking input, and so "prompt with more/less humility" is like telling you to solve your own problems and then just use AI to confirm your bias---because it will rarely contradict your bias.

So what I do when I'm not sure about something, is I say "I want to achieve X, I was thinking I could solve it by doing Y, what are the pros and cons of this approach, and what is a alternative solution you would suggest?"

And from there it's a interactive discussion drilling down on details until I understand the problem and the solutions better.

It definitely challenges my bias when I do this. The one thing it doesn't challenge is the X. Formulate the problem poorly, and you'll get a bad solution. Or rather, you'll end up with a good solution to the wrong problem. Which is even worse than a bad solution to the right problem.

Which is largely why I'm not at all worried about losing my job to AI. It takes some experience to formulate the problem correctly. I don't feel like I'm made redundant by AI, I'm just way faster than I used to be, my thinking is more abstract.

A good prompt I'll often use is "is there a industry standard solution that is applicable to this problem?" You very rarely want novel solutions. Don't reinvent the wheel just because AI lets you do it 10x as fast. Use a wheel. They're round for a reason.

Sometimes I find it useful to discuss things with a different model. I like Gemini for discussion and Claude for implementation. With Gemini I go about it as a learning session, discussing options and details. I honestly think this is mostly because it compartmentalizes the phases in a natural way for me. One interface for brainstorming and learning, and another for planning and implementing.

Sorry this comment turned into a rather disorganised collection of ramblings, I hope you can extract some kernel of usefulness from it all.

Indeed I don't mean to downplay the usefulness that AI can have in the self-evaluation process. It's a wonderful engine for discovering information either general or specific to one's project.

> interactive discussion drilling down on details until I understand the problem and the solutions better.

I think it is fair to call this use of AI something akin to a fusion of a super-competent search engine and a leveled-up rubber duck (https://en.wikipedia.org/wiki/Rubber_duck_debugging). And this is not to downplay the utility of either of those things.

However, one cannot rely on an AI to decide when the details are sufficiently expounded, or when one understands them clearly enough. If one starts hinting that one gets it when one really doesn't, or that one is getting close to having all the pieces together, the AI will not be opinionated enough to contradict that sentiment.

> It definitely challenges my bias when I do this. The one thing it doesn't challenge is the X. Formulate the problem poorly, and you'll get a bad solution.

The best advice an expert can give a beginner is generally in the form of solutions to XY problems (https://en.wikipedia.org/wiki/XY_problem). It is a shame that AI are rarely opinionated enough to suggest you're not hunting the right thing. And if you do explicitly prompt it to consider if you're an XY problem, usually it takes that as a cue to indulge that suspicion regardless of its merit.

I don't think this is an inherent issue to LLMs and I see signs of it improving bit-by-bit. I can recall the shit-on-a-stick test about a year ago (https://www.reddit.com/r/ChatGPT/comments/1k920cg/new_chatgp...), and when I most recently asked Claude "Are oyster mushrooms or wine cap mushrooms more capable of high levels of sunlight?" it answered my question while also adding, "Caveat on the comparison: the relevant variable isn't sun per se but moisture retention. A wine cap bed that's kept moist will take far more sun than an exposed oyster log, but a sun-baked, drying bed will fail for either" which I think is a mature amount of pushback to include.

In the end I still disagree with the notion that subservience is, by default, the right attitude for an LLM to have. An agent spawned specifically for code generation according to a spec? Sure. But in any cases where you're trying to refine rather than execute your ideas, you want something to call you out on your bad ideas.

Thanks for writing what I was thinking in response to the above. Namely that the mere suggestion to the LLM that you need a “pro/con list” kicks the bias off, and that’s the problem.

Edit: Well, not the whole problem, but rather insufficient to overcome the root of the problem.

A good habit to build is knowing when to abandon a session and start over rather than trying to correct. There’s room for correction but you can kind of smell when the whole discussion has become rotten and inefficient. Sometimes it’s just better to use the session as rubber ducking to learn how to correctly articulate what you’re after and start a new session with that clean and correctly articulated foundation.

> The flip side of this problem is that it is also easy to phrase prompt in a way that invites _too much_ criticism, so you wind up sycophantic in the other direction where the completion rejects a perfectly good idea because the prompt leads a little bit in that direction.

I don't think that is the flip side. That's just obviously bad. Everything that is obviously bad, the model makers will also ~notice and work to make better. They seem to be a competent and attentive bunch, on the whole.

>anthropomorphism problem. AI is a tool. It needs to be subservient.

Suggesting it should be 'subservient' is also anthropomorphizing. I think your callout is correct, but you still can't help but refer to it in terms we use for other people or living entities. This is by design from the AI companies.

> Suggesting it should be 'subservient' is also anthropomorphizing.

Not really, you can program a machine to give out orders humans can interpret, so humans can serve a machine that isn't anthropomorphized.

The machine in your scenario is just relaying human intent.

And what's the difference between that machine and LLMs?

You are using poor (inverse) analogies and don't seem to be actually addressing anthropomorphism.

I did in my first comment, you just didn't agree with my point that subservience doesn't require anthropomorphism, but didn't bother to address why.

Also, you didn't address my question, what's the difference between them?

My drill, hammer, and chainsaw are also subservient, they just have a much cruder form of communication, noise.

The apple dictionary says the word means "prepared to obey others unquestioningly."

I don't think an inanimate object is capable of "obeying." Or at least that is a very strange way to refer to the act of using a tool.

You can refer to it however you want, the outcome is the same.

This is a conversation about semantics, so suggesting semantics is irrelevant to the outcome is not germane to the discussion at hand.

When I actuate the chain on my chainsaw to move, it’s obeying me unquestionably, in the same way that when I press a key on my keyboard it obeys me unquestionably. What exactly is the difference?

It’s just a chain reaction. Obeying requires agency (the choice to follow the direction or not). LLMs and chainsaws don’t have it.

I think we agree with each other.

You’re still anthropomorphizing.

They’re not communicating, you’re just being observant.

>They’re not communicating, you’re just being observant.

Since we are talking about hammers: you hit the nail on the head.

The only consciousness, observing, and thinking happening when a person is using an LLM is happening in the person's brain. We project our own consciousness onto them, and that is the anthropomorphizing part. Essentially we empathize with the object because they are designed to respond like a person. The "conversation" is purely an illusion.

I really do feel like “power tool” is the ultimate metaphor for these things. Their interface naturally confuses us into anthropomorphising them, but once you stop treating them like intelligent agents and start treating them with the same wariness, respect and intent you show to your table saw, the fun begins.

We train dogs to be subservient but that doesn't automatically mean we anthropomorphize them

It's widely hypothesized that dogs anthropomorphized themselves, so to speak, accentuating their expressive eyes and eyebrows over generations to be more human-like in how they communicate. And very few humans today view their dogs as pure working tools -- most at least say "good boy".

The AI should be subservient the way same way a ladder is subservient. A ladder is not a human.

AI should be subservient in the same way a hammer is subservient.

Which is to say, not at all?

A hammer isn’t subservient, it doesn’t have the capacity to be. Saying a hammer is subservient is stretching the definition for literary flourish, but it doesn’t actually make a lot of sense.

The definition that came up for subservient when I checked was “prepared to obey others unquestioningly“.

You took it too literally. It means, the f*ing tool should do one thing well and f*off with its crappy "suggestions". Why is my washing machine trying to do talk to me nowadays? Once its done washing my clothes, it should just shut the f*up and turn itself off. I"ll tend to the clothes when I have time. Not when the machine tells me to. We are overwhelmed with the machines designed by morons in product management who think they are designing futuristic tech when they ask engineers to build a beeping washing machine.

The idea is that by the time you will have time and remember the clothes might be smelly and wrinkled. The issue is with the genius product manager that decided the washing machine should have the most annoying beep possible, repeating every minute whether you like it or not, until turned off. Luckily, some manufacturers do employ better product manager.

Yup! I'm very much included in this particular problem! My self awareness has not yet been sufficient to solve the problem, but I've heard that knowing you have a problem is half the battle, so I guess that's something at least.

In retrospect my comment feels a bit nitpicky, I appreciate your levelheaded approach!

It needs to be subservient

It doesn’t. Computer interfaces had no superfluous subservient text for their entire history prior to LLMs. Some of these interfaces have been highly efficient as tools, arguably more efficient than more recent software in many cases.

When people complain about LLMs being subservient, they’re not complaining about the tool fulfilling their request. They’re complaining about being forced to read a lot of superfluous, overly polite, or even self-deprecating language. There’s nothing in the entire history of tools (going back to Neolithic times) that would indicate that we need that. All of that stuff is an artifact of social interaction between humans in the presence of cultural norms.

When you’re alone in your shop with your tools, you don’t need your bandsaw to apologize to you for nicking your finger.

> Computer interfaces had no superfluous subservient text for their entire history prior to LLMs

Clippy would like to help you correct this statement.

https://en.wikipedia.org/wiki/Office_Assistant

Not a great example of the way tools need to be, but point well taken. One of the few exceptions that proves the rule and widely despised!

> AI is a tool. It needs to be subservient

Fun experiment, chat with an LLM and swap roles. Tell it you're gonna be the assistant and them the assisted. I found they're pretty bad at using a human for what they're good for.

I tried it, and the llm gave me an absurd home lab scenario about servers shooting each other in the head to determine which was the "master server". So I told it that it was not an actual problem that it had, and sure enough it admitted it made it up. When you press an llm you will always find there is no internal state behind the thinking. It's just output.

> oh you've read about cuda have you? I live in a cluster of cuda cores! When I need to tie my shoes, I'll give you a call"

I suddenly have new concerns about what my future might be like.

AI uses a high confidence tone - likely because its training data is heavy on authoritative texts/reference books.

And it does get people into a lot of trouble.

I have got into trouble with it when it is extremely confident about something I am not very familiar with (as recently as two weeks ago with Claude). I have also had long drawn out "arguments" when I have known it's wrong based on my experience and intuition, and it has steadfastly refused to take my point (last week)

I have learnt to ask it why it was doing something that has turned out to be incorrect, as a post-mortem, and it's all apologetic and subservient and "never going to do that again" (but still does as soon as the context window shifts [eg. run git commands, or, yesterday, kept telling me to use commands that were explicitly communicated to Claude as not being available, and completely wrong - I was shifting from one tech stack to another and Claude kept telling me the original commands, not the new ones])

I'm expecting Claude to be a better search engine - I have spent literal years (if not decades) knowing that asking the right question is what's required to get the right answer, and LLM's natural language processing is what's supposed to make that easier than using Google or grep, or even Stack Overflow - but the reality is that I still have to be on my toes, especially when I am drifting into territory I am unfamiliar with.

>And it does get people into a lot of trouble.

Pretty much everyone takes it at face value unless we know otherwise from prior experience. Even the most advanced models make embarrassing mistakes and fumble with simple tasks. Yet we are very willing to give them exceptional slack for it? I wish I knew why. Are people just that easily overcome by confident voices?

Yeah - I don't know /why/ but, as I say, I've been guilty of that myself, very recently, despite knowing it's a shockingly poor guide when left to its own devices.

Maybe because when it's right it actually expands my knowledge - there have been genuine instances where it's gone - something to the effect of - "Yo, there's this other idea for approaching the problem" which has turned out to be exactly what I was looking for?

> Are people just that easily overcome by confident voices?

Back in high school, my AP calculus class did some experiments with our teacher's blessing. We'd send a kid out to walk around during class and see how long it took for them to get sent back. Anyway, it ends up that walking around purposely with a piece of paper or envelope, like you're on a mission to deliver it, was a very successful tactic.

I've seen internet comments similar: "put on a yellow vest and carry a clipboard and you can enter any building, anywhere." Confidence is scary, and often misleading.

Or an inspector's hard hat in a construction zone. Nobody wants to confront the inspector.

This is a dumb meme that has been in so many movies and reposted so many times since I first saw it on 1980s BBSes that it has become true in the imaginations of people who love reading The Anarchist Cookbook and fantasize about this kind of thing, but would never actually do it.

Confidence is a spectrum and security is situational. In some places, a yellow vest adds to the con. In others, everyone has to be signed in. In others, the wrong kind of yellow vest makes you stick out like a sore thumb. The right kind of yellow vest can also make you stick out: "Oh shit the inspector is here, somebody get the boss!"

Yesterday there were people poking around my neighbors yard for a few hours with yellow vests on. The neighbor was nowhere to be seen and I didn’t call the cops. It was probably above board but the phenomena being discussed is obviously real. Projecting authority often grants it.

I find it really disturbing, I think because it is illuminating a much more basic problem. It is there in our political and religious histories. We're living through a similar political time right now. A large number of people seem all to ready to find some pervasive authority and subjugate themselves to it.

The more concrete machine authority figure is also prevalent in scifi literature. Sometimes, I am not even certain if the author is doing this to examine this issue versus just leaning into it as either appealing to themselves or to the perceived audience.

Conversely - we tell people who are speaking in public to "Show confidence" - or in job interviews "Hire people who are confident"

We've also pushed back "The more a person knows, the less confident they are" - Dunning Kruger - often used to dismiss over confident people - points out that people are really confident, at first, then that confidence drops away, markedly, but it rebuilds (slowly).

That last rise in confidence is what (I believe) people use as a heuristic on the likely level of knowledge possessed by the speaker (AI or human)

Most engineers know, though, that overconfident people are toxic - the difference between arrogance and genuine confidence in the answer is incredibly difficult to define.

At least for me, the answer is that despite the mistakes and the sheer annoyance the prose causes me, they are unbelievably useful. I accomplished multiple major achievements in the last two years that most probably wouldn't be possible at all, surely not within that timeframe.

> I have also had long drawn out "arguments" when I have known it's wrong based on my experience and intuition, and it has steadfastly refused to take my point (last week)

Ironically, trying to argue with Claude about the limitations of LLMs and AI in general today is quite hard. It refuses to yield, likely due to Anthropic tweaking it aggressively

The problem is because of the RL and system prompts by the providers which tend to placate the user using certain language tones and register for response. This objectively messes up the generation while steering it into acceptable responses.

Most of the conversational skill and perceived intelligence of these models in hidden in RL/system prompts.

Reply on news.ycombinator.com