Hacker News Reader

Comment by ugh123 | Hacker News Reader

Over Easter I asked GPT4 to count a mess of colored eggs on the floor that I was preparing for an egg hunt. They were mostly evenly separated and clearly visible (there were just 36).

I gave it two tries to respond and it wasn't even close to the correct answer.

Was it confused on colored eggs vs. "natural" eggs it might have been expecting? Should it have understood what I meant?

View on HN

I imagine it would be better at describing an image in a general sense, but probably isn't processing it in a sense where it would actually be counting individual features. I could be wrong about that, but it seems like a combination of traditional CV and an LLM might be what's needed for more precise feature identification.

The use case of “counting objects” is basically completely solved already by yolov8. There is no need to use an LLM for that.

Seems like we're pretty far away from "AGI" or even anything resembling a legit "multi model"

multi modal

I need an AI model to tell me which AI model/tool to use.

Then you might want to look into query routing or function calling.

I'm pretty sure LLM vision capabilities are right now limited to something similar to subitizing in humans at best, ie. being able to perceive the number of items less than ~7 without counting. Expecting it to be able to actually count objects is a bit too much.

LLMs are very bad at counting.

I've been disappointed with both GPT-4's and Gemini 1.5's image recognition abilities in general, not just counting. When I have asked them to describe a photo containing multiple objects--a street scene, a room--they identify some of the objects correctly but invariably hallucinate others, naming things that are not present. Usually the hallucinations are of things that might appear in similar photos but definitely are not in the photo I gave them.

Reply on news.ycombinator.com