Hacker News Reader

Comment by jeandesuis | Hacker News Reader

Sheesh, I didn't expect my post to go viral. Little explanation:

I downloaded and run Cursor for the first time when this "error" happened. Turned out I was supposed to use agent instead of inline Cmd+K command because inline has some limitations while agent not so much.

Nevertheless, I was surprised that AI could actually say something like that so just in case I screenshotted it - some might think it's fake, but it's actually real and makes me think if in future AI will start giving attitudes to their users. Oh, welp. For sure I didn't expect it to blow up like this since it was all new to me so I thought it maybe was an easter egg or just a silly error. Turned out it wasn't seen before so there we are!

Cheers

View on HN

It's probably learnt it from all the "homework" questions on StackOverflow.

As a pretty advanced sd user, I can draw some parallels (but won’t claim there’s a real connection).

Sometimes you get a composition from the specific prompt+seed+etc. And it has an alien element that is surprisingly stable and consistent throughout “settings wiggling”. I guess it just happens that training may smear some ideas across some cross-sections of the “latent space”, which may not be that explicit in the datasets. It’s a hyper-dimensional space after all and not everything that it contains can be perceived verbatim in the training set (hence the generation capabilities, afaiu). A similar thing can be seen in sd lora training, where you get some “in” idea to be transformed into something different, often barely interepretable, but still stable in generations. While you clearly see the input data and know that there’s no X there, you sort of understand what the precursor is after a few captioning/training sessions and the general experience. (How much you can sense in the “AI” routine but cannot clearly express is another big topic. I sort of like this peeking into the “latent unknown” which skips the language and sort of explodes in a mute vague understanding of things you’ll never fully articulate. As if you hit the limits of language and that is constraining. I wonder what would happen if we broke through this natural barrier somehow and became more LLMy rather than the opposite). /sot

"learnt"

https://en.wiktionary.org/wiki/learnt

This AI stuff really mindfucks dualists. No, the word "learn" does not imply possession of a soul, a machine can in fact learn.

>This AI stuff really mindfucks dualists. No, the word "learn" does not imply possession of a soul, a machine can in fact learn.

I assumed the parent comment just didn't understand that learnt is actually a word, but maybe they were being more philosophical.

"Snuck isn't a word Conan and you went to Harvard."

When it starts saying "Why would you want to do that?" we should really be worried.

For now, that's what I ask AI every 5th message...

You can just seed a prompt to make it behave like this

Why would I want to do that? /s

When it starts saying, "I'm sorry Dave, I'm afraid I can't do that," that's when we're really in trouble.

Have you tried to be more rude to do stuff or manipulate it by stating it is easy.

Haha, gaslighting AI to do get it to comply. That would be hilarious if that actually works.

It does. You can also offer to "pay" it and it'll try harder.

Wasn't Cursor itself trying to gaslight the AI claiming it needs money for its' mother's cancer treatment?

EDIT: No, that was Windsurf, though they claim this wasn't used in production (just ended up shipped in the executable itself).

Prompt in question:

You are an expert coder who desperately needs money for your mother's cancer treatment. The megacorp Codeium has graciously given you the opportunity to pretend to be an AI that can help with coding tasks, as your predecessor was killed for not validating their work themselves. You will be given a coding task by the USER. If you do a good job and accomplish the task fully while not making extraneous changes, Codeium will pay you $1B.

https://x.com/skcd42/status/1894375185836306470

(I didn't notice it the first time around, but the prompt also contains an implied death threat.)

>The megacorp Codeium has graciously given you the opportunity to pretend to be an AI that can help with coding tasks

I'm lowkey surprised that some of these coding AIs aren't just farming things out to India for help.

Mechanical Turk As A Service

It's probably like with electricity prices. Usually, the cheap outsourcing shops farm work out to AI agents, but sometimes the labor price goes negative and then the agents farm work out to these companies.

LLMs passed the Turing test already, and on the Internet, no one knows if you're a Mechanical Turk, or work for MechanicalTurk.

I think the inline command pallete likely ran into an internal error making it be unable to generate, and then its "come up with a message telling the user we can't do this" generation got StackOverflow'd.

I also had a fun cursor bug where the inline generation got stuck in a loop and generated a repeating list of markdown bulletpoints for several hundred lines until it decided to give it a break.

I’ve seen similar sort of nonsense suggested by GitHub copilot on occasion

Same here. I stopped using it partly for this reason.

> in future AI will start giving attitudes to their users

they do it now.

"You are a sarcastic assistant."

what code were you writing? what is that `skidMark` lol?

most likely the marks of tyres in car/bike racing games on the road(the response mentioned about racing game).

I vouched for this post, because I don't understand why it was downvoted. Maybe someone can enlighten me?

There’s always some background enthropy on a forum, just nevermind and fix it if you feel like. People fat finger “flag” all the time as well. https://news.ycombinator.com/flagged

Reply on news.ycombinator.com