I'd be surprised to hear that lots of people don't do just this. As soon as the memory features came out I gave them a quick try and quickly turned them off.

I don't want to be held accountable to all of my previous ideas. I want each conversation to start fresh with the context that I provide. If I am exploring some library in one language stack and then I later want to look into something completely different, I don't want the conversation polluted by what it thinks I want based on the previous discussion.

I suppose for those who use it as a companion the memory is a core element. But when used as a tool it gives a significantly worse experience IME.