Hacker News Reader

Comment by giarc | Hacker News Reader

Am I missing something or in this image, is the confidence score 99.23% and has the incorrect license plate? The plates clearly have a 5 but it's predicting an S.

https://camo.githubusercontent.com/5138d6bce5c840d15e533db6f...

View on HN

Is that confidence for the value or the category. It comes after LP in the image which makes me think it's the confidence that it is a license plate, not necessarily the confidence of the reading of said license plate.

You might be right. The gif on github, at times, shows no plate prediction but still includes a confidence score. I would suspect that if it can't even show a prediction, it's confidence score wouldn't show either (being that it would be 0%).

> I would suspect that if it can't even show a prediction, it's confidence score wouldn't show either (being that it would be 0%).

Recognizing that something is a license place, and being able to read the plate are mutually exclusive things. People can identify license plates even if they can't read them, and people who don't know what a license plate is my still be able to read the numbers on them.

The ML has to do two things; the first is reject anything that isn't a License Plate, the second is OCR License Plates.

Typical model output is not actually a probability, but a mostly arbitrary number between 0 and 1. Models usually have to be calibrated to match probability.

It is really confident in its answer, but that does tell you much about how likely the answer is correct.

For example, if I built a model that always returns the same plate number, it would be 100% confident, but wrong 1947791 times out of every 1947792 plates it sees.

That said I would have thought confidence would be lower for plates with 5 and S in it.

For neural networks unless it's calibrated otherwise the confidence score has little meaning to the accuracy. [1] see figure 4 and you will understand what I mean

[1] https://arxiv.org/pdf/1912.03263.pdf

ANPR Tip: Correlate between successive images to improve confidence, and use (if available eg in the UK) legal plate format information to rule out unlikely characters.

Also, humans aren’t perfect at reading plates from highway footage still images - which is why ANPR systems are never 100% accurate.

The 5 prediction probably had a confidence score of 98.52% or something, and the model chose the one it believed to be most likely.

I don't know anything about neural networks, but the fact that you can have two incompatible predictions each with more than 50% confidence tells me that confidence is not what my intuition would predict (with 87% confidence)

99.23% is not 100%. 1/130ish But, the wrong examples don't seem a good demonstration of that...

Nope, you are correct. Wow, both examples from that image are wrong.

Reply on news.ycombinator.com