I've asked it to give me a confidence rating for its replies to my questions but it states that it can't give one
For example:
> I'm going to share some information, I want you to classify it in the following JSON-like format and provide a responses that match this typescript interface:
> {
> "isXXXXX": boolean;
> "certainty": number;
> }
However, I got either 0 or 1 for the certainty every time. Not sure if it was because they were either cut-and-dry cases (certainty 1) or not-enough-information (certainty 0).
I'm actually trying to think of a good example of text I could ask it to intuit information from and give me a certainty
For example, ask it to subtract 2 20-digit numbers. It will come up with an answer X where the first couple of digits are correct, and everything after that is wrong.
It gets better.
Ask it to correct itself. It will come up with a different wrong answer Y.
If you then ask it to explain why the answer is right, it will give you an explanation. At the end of the explanation it states the answer is X again, and then in the very next line concludes by telling you that is why the answer Y is correct. :)