It's also stunning how poorly it performs in a real world (i.e., non-human-prefiltered photo) scenario.
Like autonomous cars or security cameras or robotics.
Like autonomous cars or security cameras or robotics.
We're not yet at a 'general vision is a solved problem' level and this may take a long time still - if it is ever solved. But many problems can be reduced to the point that they become tractable even absent 'general vision'. Affordable LIDAR is a huge step, so is radar. Those two reduce the problem of vision with two crappy stereoscopic cameras set 5" apart to something much closer to the domain.
As far as I can see the race is on.
Our GPU kit has got 6 times better in 2 years.
Exciting times.
Garbage in, garbage out. When was it any different?
The question is: can a human do much better on that garbage?
> Public Datasets have style of photo that represent it's own bias.
That I readily agree with.
Scoring is also critical, you want to gracefully degrade classification systems not getting the correct dog breed is insignificant vs calling a dog a sofa. Further, people base real world classification on stereoscopic video footage. Training robots to classify photos is a handicap for building robots to operate in the real world.
I started from the dataset bias position myself - billions of family snaps and selfies can't provide the reference for arbitrary images of stuff from arbitrary angles. My reading of the results that we got is that the claims of lower level features extracted by the cnn are correct, and that a network trained on massive public data can be the basis for specialised tools trained on more constrained proprietary datasets.
Your milage may vary though...
Different tasks have different error tolerances. When an MI tool can get within the acceptable margin of error for certain tasks, then there is a possibility for the MI to take jobs.
As long as accuracy is critical to a classifier, in many cases it's better to pay for human eyeballs to look at the criteria.
We're talking about a huge spectrum of applications here. Most of us probably don't care that much about the difference between human and MI error rates about classifying images used in broad-scale advertising on the internet.
We do (or perhaps should) care very deeply about human vs. MI error rate in terms of granting or not-granting things like parole.