GPT-4V is mind blowing, it surprising to me that it gets so little attention here on HN, because after playing around with it I get the same sense of excitement I got when I tried the original ChatGPT. The level of understanding of what is going on in an image is leagues ahead of what we had until this point, ahead of Bard and basically everything else I already saw.

I tested it with a bunch of photos I made and images it could not have seen in its training data, and most of the time it nailed them perfectly. Its OCR capabilities are top notch, but this is combined with a spatial understanding of how text relates to other parts of the image. It can take a photo of a wall monthly calendar with hand scribbles on it and give you a list of events for each day. It can guess where a specific photo was taken just by analysing the elements present on the photo like the foliage, architecture, car license plates, etc (without being specifically prompted to do so). It can correctly identify multiple plants from a same photo. Gave it a photo of a Montessori set for teaching math (some wooden blocks with numbers and dots on them, no branding on them) and it guessed exactly what it was. And all of that just from two days of testing.

Here are just few examples:

[1] https://i.imgur.com/cV3dVOf.png - Gave it a screenshot from Final Fantasy VII from a boss battle. It correctly identified the party members, and their stats even though the text and labels are a bit all over the place.

[2] https://i.imgur.com/WeXhP7V.png - A photo I shot on my vacation, that didn't really contain any major landmarks, and yet it still somehow figured out from the architecture (and house colors) the exact location of it. I tried this game with several photos and it is very good at it, far better than I could ever be if I saw these photos for the first time.

[3] https://i.imgur.com/HgwYv6q.png - A screenshot of a worksheet from the Human Shader Project. I just asked it to solve it for given X/Y values and it did, its answer was 100% correct (here's the second part of its answer: https://i.imgur.com/RZF2r7v.png)

[4] https://i.imgur.com/12xg4qU.png - A photo of a highly reflective microwave inside a shopping mall. This was given to my by a friend who shot this personally and to be honest I didn't catch at first that this is a microwave, and yet GPT-4V figured that out.

[5] https://i.imgur.com/qSifni5.png - a good old fashioned "find the path connecting one object to the other" puzzle. Correctly identified the right path (this one was taken from the internet so there is a slight chance it saw it in the training data and somehow got the solution for it from the accompanying text, although I couldn't find any instance of it).

Edit: To confirm that [5] was not a fluke I hand drew my own version of this puzzle, took a picture and uploaded it, GPT-4V nailed this one too: https://i.imgur.com/8NgWhzw.png