It has a lot. I find by challenging it often, getting it to explain it's assumptions, it's usually guessing.
This can be overcome by continuously asking it to justify everything, but even then...
This can be overcome by continuously asking it to justify everything, but even then...
However, constant skepticism is an interesting habit to develop.
I agree, continually asking it to justify may seem tiresome, especially if there's a deadline. Though with less pressure, "slow is smooth...".
Just this evening, a model gave an example of 2 different things with a supposed syntax difference, with no discernible syntax difference to my eyes.
While prompting for a 'sanity check', the model relented: "oops, my bad; i copied the same line twice". smh
I guess I'm more interested in understanding what it can and can't do.