Note, all impressions based on Claude 1.2, got an email from Poe in the last week saying it was version bumped to 1.3 with a focus on coding improvements.

Impressions:

Bad enough compared to GPT-4 that I default to GPT-4. I think if I had api access I’d use it instead, right now it requires more coaxing, and using Poe.

I did find “long-term” chats went better, was really impressed with how it held up when I was asking it a nasty problem that was hard to even communicate verbally. Wrong at first, but as I conversed it was a real conversation.

GPT4 seems to circle a lower optima. My academic guess it’s what Anthropic calls it “sycophancy” in its papers, tldr GPT really really wants to do more like what’s in the context, so the longer the conversation with initial errors goes, it’s actually harder to talk it out of the errors.

I have access to claude. It's not bad, but decently behind gpt4 for code

And is code generation ability equivalent to code understanding and search ability?