Mobile Visual Search | Hacker News Reader

Mobile Visual Search

omoby.com

27 pointskljensen17 comments

I gave this a try too with some borrowed items around the office; here are my results:

  Blackberry - "Blackberry Cell Phone"

  Flat Screen TV - No match (but there were tons of reflections)

  Motorola Cell phone - "Cell Phone"

  Ikea French Press - "Glass Coffee Press"

Extra impressive since it was transparent!

  Klean Kanteen metal water bottle, non logo side - "Silver Metal Water Bottle"

  Klean Kanteen metal water bottle, logo side - "Kleen Kanteen"

  Black Stapler, odd angle - "Black Stapler"

  Roll Call Newspaper - "Roll Call Capitol Hill Newspaper"

  Vanity Fair cover - "Vanity Fair"

  Economist, Feb 13 - "Economist New Dangers for World Economy" (The title)

  Microsoft Natural Keyboard 3000 - "Ergonomic Keyboard"

(I was a bit disappointed in this one, it's a very distinct product)

  Front of the book "Ambient Findability" - "Ambient Findability"

  Back(!) of the book "Ambient Findability" - "O'reilly Ambient Findability"

  Trader Joes store brand Apple Juice - "Apple Juice Trader Joes"

  Urban Outfitters sunglasses - "Sunglasses"

  Ken Cole messenger bag - "Black leather pouch"

Overall I'm VERY impressed, although it looks like it's mostly just reading any available text, and that they're not canonicalizing their entries (So 'ambient findability' and 'oreilly ambient findability' aren't really records pointing to the book, just strings of text).

Still, identifying an upside down stapler from the side, and a book from the back cover is pretty darn impressive.

I also noticed experienced a noticeable battery drain and warm iphone from using this thing for 10 minutes or so, but definitely nothing show stopping.

I'm pretty positive that they have a bunch of people typing these things up as they come in.

I took roughly the same picture of the same book twice, and it returned two distinct results:

"Godel Escher Bach An Eternal Golden Braid" vs. "Godel Escher Back Book"

Then I tried a WarCraft III CD case:

"Warcraft Dvd" vs. "Warcraft Video Game"

Some noodles:

"Neoguri Snack" vs. "Neoguri Spicy Seafood Udon Noodles"

JBL Creature Speakers:

"Jbl Speakers" vs. "Computer Speaker" vs. "Jbl Computer Speaker"

I just did an experiment: I took a picture of a box of "Nueva Cocina" brand red beans and rice, once with the logo obscured, and once with the logo visible.

Neither image returned a brand name ("Red Beans and Rice" & "Red Beans and Rice Box"), even though most of my other attempts have. Most puzzling though - I took a third picture, a close up of the logo with only the letters "VA CO" visible in the image. The response: "Viva Cocina".

What does this mean? If it was truly an algorithm, it should have been able to associate the logo with "Nueva Cocina" correctly, which it did not. But the fact that the response was remarkably close to correct leads me to think that not only is this crowdsourced, the same person must have gotten two of my images in a row, and just forgot the exact brand name. WTF? A bit creepy.

They use crowdsourcing:

We're a collaboration of scientists at UC Berkeley and UC Davis and our image labeling engine can take any photo and accurately label it. We use computer vision and an innovative crowdsourcing network to recognize images, and the architecture learns over time, so it gets smarter with each search.

I figured that this was too good to be true. They seem to be exploiting people with a fairly decent command of English, though.

No, they don't use live crowd sourcing. They mean that they learn from the "crowd" of users. What they wrote is misleading.

I think they do. I am getting different labels for the same thing quite frequently.

I stand corrected. Jesus they do in part.

Desk objects are fine but the people that would really use this, to me, are fashionistas trying to figure out whose shoes that girl over there at the bar is wearing...if they can make this work in 'real world' social environments, they've got something pretty interesting.

I couldn't resist to download this app, so I did, and tested it. Took a picture of a red logitech wireless mouse and it was able to recognized in a few seconds. It knew it was a red logitech brand mouse, though it didn't' get the model right, but nonetheless it was able to classify it pretty well. I also took a picture of my toshiba laptop, this time it only knew it was a laptop.

Overall it's pretty awesome! They're doing a fine job with this thus far. I applaud them.

This is very cool, if the tool works as shown for most queries.

One thing I noticed: The battery meter on the iPhone went from nearly full to nearly empty in the course of the demo. I was wondering whether that type of app could draw that much juice, or if there was another explanation -- lots of outtakes, failed search examples that didn't make it into the demo, etc.

Has anyone used their API? I can see a lot of potentially interesting applications, but my gut feeling is that this is still a problem in the academic realm -- though it might work in some textbook cases, it would probably fail/tag incorrectly in a lot of real world cases.

Can someone point the foundation paper for their technology? I've Googled some of their scientists, and most of them are working on neural related area (some work on sparse coding, I guess that is remotely related to this).

Check the IQEngines link on the bottom and their demo on http://www.iqengines.com/demo.php . Pretty impressive (iff it's legit).

I see Amazon, eBay, etc.. it would be cool if they added something like Milo for local search. Might be a better use case (save money by finding a lower price for something locally)

Video w/ fingers over book is uber-cool. Need to touch up their website & make video more like the original dropbox vid. (I.e. entertaining)

Also try Google Goggles, I suppose.

Reply on news.ycombinator.com