I'm not super impressed yet either, but I do think there's a lot of value. I tried a few times to get it to write some simple python scripts, and usually it'd get some things wrong. But it did help to get the basics of the script started.

If you haven't already, look into trying out Copilot (I don't think there's a free version anymore?) or Codeium (free and has extensions for a lot of editors, just be careful with what code you're okay sending to a third party). Using AI prompts as a fancy auto complete is what has been giving me the most benefit. Something simple like a comment "# Test the empty_bank_account method" in an existing file using pytest, I go to the next line and then hit tab to auto-complete a 10-15 line unit test. It's not always right, but it definitely helps speed things up compared to either typing it all out or copy/pasting.

My biggest annoyance so far, at least with Codeium, is that sometimes it guesses wrong. With LSP-based auto-complete, it only gives me options for real existing functions/variables/etc. But Codeium/copilot like to "guess" what exists and can get it wrong.

> Like a buddy who sits nearby with a beer, sees what you do and is seasoned enough to drop a comment or a story about what’s on your screen any time.

I agree with you, this is probably where it would be most useful (to me). I don't need someone to write the code for me, but I'd love an automated pair-programming buddy that doesn't require human interaction.