Hacker News Reader

Comment by FrustratedMonky | Hacker News Reader

Yes When I have a conversation with GPT4, it keeps the history, and the entire history is a prompt for the next prompt.

I would say, not totally unlike a human memory. just GPT4 is limited to 8000 characters or something. Humans have a bit more, but with added 'forgetfulness', so who knows how many characters for real. Humans don't actually recall perfect text very well.

View on HN

But we do manage to remember stuff that happened "quadrillions" of tokens (of sensory input) back. Sure, we are wrong a lot, but we have a context that goes way back. A LLM has a fruit fly's version of what we work with, although in some versions it has perfect recall in that context.

Or it can be said that we boil down our experience to a few millions of tokens, only a set of experiences, emotions and often wrong/made up memories.

Your comment assumes a human can recall all (potentially incorrect) memories and use those to make some judgement, but fact of the matter is we don't.

Yeah, Humans do ton of aggregation and filtering. Thus you can't remember individual days of commuting, because it is so similar, the brain just lets it go.

I'd think something like this would be added to the models eventually. Of course, over-simplifying.

I think that human memory "keeps" the next prompt by baking it into the parameters of the "neural net" (the brain) immediately

this is has gotta be somehow an analogue of dreams andor sleep andor AI hallucinations.

I think this because for many of us, to not sleep triggers hallucinations, which is the conscious experience of whatever the organic version of 'backpropagation' (training the model?) really is.

This is rather pseudoscientific.

99% of conversations on Internet are pseudoscientific.

Think here we are just 'conceptualizing', 'spit-balling', 'blue-sky-discussion', 'wondering out loud', 'hypothesizing'. etc...

Reply on news.ycombinator.com