Show HN: ObjectCropBot – Interactively Crop Objects from Photos with AI
andreykurenkov.com
andreykurenkov.com
They basically took the same idea and instead of just producing a hack finished a nice full version of the idea - very nice write up!
But, they don't have any web UI for it yet, so I am getting tempted to revive my dormant side project and likewise make a proper polished version of it with a web interface. If anyone on here is a talented web dev with an interest on working on this as a side project (for free, and for fun, though we could explore monetization if it works), feel free to get in touch! (ps extra get in touch if you are near south bay area/Stanford)
PS Incidentally, I independently came up with the same idea as in Deep Interactive Object Segmentation (https://arxiv.org/abs/1603.04042) and implemented it for my Stanford CS 229 (Machine Learning) project first - the hack came later. The ObjectCropBot hack allows only cropping and not clicking because it was faster to hack in that, but I think it is ideal to allow both cropping to constrict around target object and clicking (clicking alone leaves too much ambiguity , eg do you want to crop the person, or their shirt, or a subset of that shirt).
Pizza Photo Editor is still early-stage -- we're still adding features and increasing the coverage of different object classes in our training data set -- but the core tech is there, and it does have an interactive web UI, so it's already fun to play with. :)
Looking at the example inputs & outputs for ObjectCropBot, it's evident that the challenge is not just detecting the object and clipping it approximately, but tracing a tight precise boundary around it. We've found that DNN/ConvNet-based approaches don't offer the necessary precision, so it's necessary to perform some pre- or post-processing using other Computer Vision techniques. At Object AI, we've developed an "object boundary deduction" pipeline that combines ConvNets with other tech. It's interesting that the blog post that you've linked to makes the same observation!
As to the lack of precision of ObjectCropBot, I think a lot of that is due to it running based on DeepMask and not SharpMask; Facebook's insight to run the final low-res features back through increasingly less downsized images with sort of skip connections is a good one, and I'd bet the resolution of the results would be a lot better if I just hacked it to use that instead of DeepMask (I think the thing I linked just uses that, pretty much).
Still impressive, just needs a better name. There are two problems in Computer Science...
Are you available for consulting? I have a project in this domain that you may be interested in.
(My email address is my HN username @ my startup domain, if you'd prefer to discuss over email instead.)