Hacker News Reader

Comment by devit | Hacker News Reader

Tried to remove an elephant in an hard instance (what to do with the woman on top?), and a bicycle in an easy instance (lone bicycle), both found from Google Image Search .

http://deepangel.media.mit.edu/showcase/Works%20terribly

http://deepangel.media.mit.edu/showcase/Another%20greyish%20...

Terrible results in both. It looks like it just works on their examples and doesn't do any better than object detection + paint a solid rectangle on others (arguably, it does worse than that).

View on HN

unfortunately in my experience that's the case for most of opensource AI projects out there, while the showcase results are hand-picked or the algorithms was trained and tuned to solve that specific image.

It makes me suspect that the overfiting concerns on deep learning are correct.

I doubt this AI gets perfect performance even in the training set. Deep generative models are known to underfit more rather than overfit, i.e. they can't even do a good job of the full training set let alone the test set. The cherry-picked examples you see are just statistical outliers corresponding to VERY easy examples.

Statistically-random excellent performance in complex tasks is very unlikely. More likely is that examples are in-sample from a small training set or very similar. Big NNs can memorize anything.

They can memorize any supervised learning task, but so far, we haven't been able to see any deep generative model successfully memorize something more complex than MNIST.

I think a 3 year old would do a better job than this AI. Not sure why they bothered with such a fancy website for this to be honest.

This happened with my image as well. AI just put a big gray rectangle on top of the persons and called it a day.

https://s3.amazonaws.com/deepangelai/results/person/thomasah...

damn this is embarrassing considering how much PR and marketing MIT puts out. That image is almost comical, it's as if somebody just got tired and holds a loose definition of "removing an object from a picture".

Reply on news.ycombinator.com