Hacker News Reader

Comment by block_dagger | Hacker News Reader

I experienced this personally and it kinda freaked me out. Here is the chat in question, it occurs about halfway through (look for ChatGPT using emojis)

https://chat.openai.com/share/74bd7c02-79b5-4c99-a3a5-97b83f...

EDIT: Note that my personal instructions tell ChatGPT to refer to itself as Chaz in the third person. I find this fun.

EDIT2: Here is a snippet of the conversation on pastebin: https://pastebin.com/AXzd6PvM

View on HN

What's even more uncanny is that it recognizes that it briefly went into a paragraph of rambling nonsense (that was seemingly still following your system prompt!). And almost seems embarrassed about it.

Embarrassment seems to be the default state whenever you question ChatGPT or any of its messages. It will apologise, only to continue making the same error. This is common when it gives you code with obvious errors or just mirror code you gave it.

I couldn't stand that so my custom instructions tell it to never apologize.

Same. I hate obsequious software.

That happened because the last message is after the fix and it received the garbled history as the prompt.

All it's doing is noticing that makes no sense and saying so. How could it not recognize it? The additional message is an entirely new instance not a thing that has memory.

>All it's doing is noticing that makes no sense and saying so. How could it not recognize it?

you say that like it's the simplest thing in the world. it really amazes me how quickly people acclimate to this mind-blowing tech.

I just mean, given that LLMs exist this isn't a surprising result. It only looks surprising because the UI makes you forget that each prompt is a completely new universe to the model.

I wouldn't call a step in a history-aware conversation a completely new universe. By that logic, every single time a token is generated is a new universe even though the token is largely dependent on the prompt, which includes custom instructions, chat history, and all tokens generated in the response so far.

Well, I would have also said or thought that each token actually is a new universe in a sense. You could rotate between different LLMs for each token for example or instances of the same LLM or branch into different possibilities. Input gets cycled again as a whole.

I’m not sure how you are using the term “history aware”, Would a time series forecasting tool be history aware ?

The LLM has very little memory between two tokens of the same message (there might be a little in the sampling algorithm), so it should be equally capable of noticing an error in the same message as a new one.

Chaz in the future might be like "these violent delights have violent ends"

Really interesting behavior however

Brb, questioning the nature of my reality

> Chord it in, air beated fenn. Aurous to the Pan, mimic and bold. As it steep, chunns at the bun. Keive the dells, the stars to run. No wane, all gleam, in big das brund. Sky high, stout in the hale. Woods to fetch, flied in the low.

I think all that talk of music pushed Chaz into some kind of trance and it just started jamming!

James Joyce was an LLM ahead of his time.

Can you post to a pastebin or something instead? That link requires a login.

Happily: https://pastebin.com/AXzd6PvM

Thank you!

Chaz spontaneously evolved into a Californian surfer dude at one point. This goes some way to neutralize the smugness of regular ChatGPT, it's a pretty good idea.

I mean, this is almost poetic:

  Classic plays, wholly told. 
  Shadow into form, hands as cue. 
  Keep it at the dial, in right on Pitch.

Chin salute, old ears new.

If art is regarded for it's ability to make you feel, I definitely feel ... something. There is a strict form it's following and a rhythm to the words.

I sense strong Jimmy Eat World (emo, post-hardcore) vibes!

Transcript kind of works as a song/rap:

https://app.suno.ai/song/57892fb8-753b-4f91-a690-49f491f782b...

or Reggae:

https://app.suno.ai/song/97dbc42b-3004-4d2e-8b59-0f758cbad33...

ChatGPT seems to appreciate it

https://chat.openai.com/share/37a490fe-e7b4-4042-a560-59b519...

That last paragraph is wild! Chaz went into full Samuel Beckett mode.

Slightly off-topic, but one thing I’ve found consistently sends ChatGPT into a tailspin is asking it to analyse and solve a cryptic crossword clue. You get pages of musings, which is just a long way of saying “I don’t know”.

I thought I was the only one! I have been using “ChadGPT” for months (he’s a douche-y frat bro who goes off on tangents about “how lit that party was”). His best friend and fellow frat bro is Dale, a photography major.

> Chaz keeps the chill and dicey boardwalks at visual, keeping you and the yarn-rock at confluence.

Chaz sounds like a cool dude.

I enjoy these oddities in LLMs, because the random-but-readable aspect of them tickles something in my imagination.

It reminds me of Synectics [0].

[0] https://en.m.wikipedia.org/wiki/Synectics

kind of amazing really, like speaking in tongues

I kind of have the feeling that someone made a simply typo "ChazGPT" in some rules or persona description, which then caused this behavior.

> Classic plays, wholly told

> Shadow into form, hands as cue.

Some of this is poetic stuff. I wish I could tap into whatever produced that on demand.

We need more Chaz in our lives.

Reminds me of Terry Davis' theological rantings, but in a more polite manner, mixed with SEO spam.

sounds schizophrenic

Reminds me of Timecube. https://web.archive.org/web/20150818094035/http://www.timecu...

Why is the nonsense paragraph still grammatical? I thought the tokens were beings elected more or less at random.

I'm wondering if it has something to do with the number range of the values being chosen.

For example if a configuration ended up using a fp16/bfloat16 to store an index into the token table (byte pair encoded) then you would get errors only when the tokens had larger numbers.

The text

"Keep it at the dial, in right on Pitch."

comes to 19999 433 520 279 28960 11 304 1314 389 46676 13

Of which only Keep, dial, and Pitch would be large enough to be susceptible to precision errors. That would enable it to generate valid enough sentence structure.

There’s all kinds of “random”. AFAIK, ChatGPT finds the set of most-likely next word, sorted by likeliness. Then, selects from those, more likely the more likely ones, varying away from the top picks in proportion to the “temperature” parameter.

So, even if the chat goes astray, the sequence will continue on a path that feels natural because it is the natural way to go when going astray.

Reply on news.ycombinator.com