A really good point that there’s no guarantee that the reasoning tokens align with model weights’ meanings.
In this case it seems unlikely to me that it would confabulate its exif read to back up an accurate “hunch”
In this case it seems unlikely to me that it would confabulate its exif read to back up an accurate “hunch”
That’s why I mentioned the case where it made up things that weren’t in the photo - “drives on the left” is a valuable GeoGuesser clue, so if GPT looks at the EXIF and determines the photo is in London, then it is highly probable that a GeoGuesser player would mention this while playing the game given the answer is London, so GPT is probable to make that “observation” itself, even if it’s spurious for the specific photo.
I just noticed that its explanation has a funny slip-up: I assume there is nothing in the actual photo that indicates the picture was taken in mid-February, but the model used the date from the EXIF in its explanation. Oops :)