AWS DeepLens – Deep learning enabled video camera for developers | Hacker News Reader

AWS DeepLens – Deep learning enabled video camera for developers

aws.amazon.com

173 pointsirs48 comments

This is neat but sort of strange - I love the idea of a hardware package that makes fiddling with deep learning easy, but I wonder why Amazon went through the trouble of producing such a relatively niche product?

If I had to guess, this is going to be a great "user education" tool for AWS, designed to get new developers on the platform as early in the learning process as possible.

Do you really think its niche? Sure its not going to be a GoPro consumer product but the market for "smart video cameras" or whatever you want to call video+insight is going to be pretty darn big. This seems like this easiest foray into combining a video camera with learning with limited bandwidth issues. My only concern is whether or not Google will come out with something similar in the near term.

Surely Google will come with one. Or something under Android umbrella. Only concern is, Google's hardware is at best meh level, while Amazon had better reputation in that front.

Considering the engineering and manufacturing investment in this product, it's hard to see where they are expecting their ROI. It certainly won't be from the product sales themselves, and since the target market seems to be individual as opposed to corporate developers, I wonder if increased use of AWS is going to generate enough revenue to offset the costs.

Because they open everything they develop and in this case they want to see innovations happen around their new home cameras! Which they use already with deliveries.

Don’t forget the weird fashion selfie cam.

A gadget to hook you into using their ML cloud?

This is great for my chicken coop project :) I have been working with a raspberry pi and camera to train a system to recognize and respond to squirrels (close the feeder to stop them eating food). There are many other options to build this, but it is nice that this is an integrated system that is easy to train!

Nice! I bet you could start a business with this if you get it robust. I have been thinking about a bird feeder that works only for birds below a certain weight like hummingbirds. We often have crows taking our feeder apart.

I would really like a camera that would log when deer, or other wildlife aside from squirrels, are in my backyard.

Please submit a post about this project, with juicy details!

Sure will do, mostly I've been training squirrel recognition models on my laptop and also learning what it takes to run those models on the raspberry pi (low frame rates) with OpenCV. I did also buy one of these though https://www.adafruit.com/product/997 so once I get everything running I may also have videos of squirrels being sprayed by the garden hose :)

If I may give some dog trainer advice, you are going to spend water and energy making some squirrels just a bit more challenged to the feat, consider just attracting the offending ones to elsewhere where they find some cheap/disposable food source so they let the chicken's alone :-) — this approach is usually cheaper and renders more results with dogs, cats and even chickens themselves! Say, once you detect them, you lure them to a certain path or place, maybe with some LEDs or scent or something you can automate dispensing (I don't known squirrels).

It's it kind of funny... one of the first home-brew CV projects I ever saw was an engineer's system that would unlock the cat flap door when it detected the cat's silhouette. A desirable side effect is that it would not open the door when the cat was carrying a rodent corpse.

https://web.archive.org/web/20080913222639/http://www.quantu...

You might want to take a look at project ELL that runs DNNs on Raspberry pi: https://github.com/Microsoft/ELL

That is awesome!

Evil tech, taking food from little squirrel.

Related: I recently came across the "Google AIY Voice Kit for Raspberry Pi" in the Adafruit store: https://www.adafruit.com/product/3602

That, as well as this new "AWS DeepLens", look like interesting kits/toys to try out new on-device and hosted systems for voice processing and computer vision.

Then again, I assume you have to buy these knowing that the hardware will likely cease to work in the next year or two or three... or whenever the hosted aspects change, and AWS and Google move on to their next trend.

Any other similar kits/toys others have found?

Well, with the Google AIY kit, as it's Raspberry Pi based, you have the ability to do what you want with the hardware. It's really just a microphone and speaker on top of it.

At some point you could run offline models to take voice input and output intent. What you then do with that is up to you. What the cloud services provide are common lookups, such as searching wikipedia, your calendars, etc. You'd need to implement that yourself.

AWS DeepLens seems to run Ubuntu 16.04 under the hood, but it's unknown whether you can get SSH access to it. If so, it's could be a really nice hackable camera on which you could run anything you want.

Just read from the site FAQs that it does support SSH.

100 GFLOPS can process deep learning predictions on HD video in real time? I thought even GPUs like the GTX 1080 (8.8 TFLOPS) had difficulty with that.

Most of the time you don't need pixel-level classification. Suppose you want to make a cat tracker. How small can you shrink an HD image and still reliably detect a cat (using your own brain)? Pretty small. Probably less than a quarter of the size at least. Do you need colour? Often you can get away without it, so that's a third of the data gone too. For full-image classification you could go even smaller and still be accurate.

Also, you don't have to train all the time. Most of the time, you're in inference mode.

It's not very much. 100 GFLOPs is about 1-2 Haswell/Broadwell cores (32 FLOP/s * ~2 GHz AVX frequency). Even 2015 era mobile GPUs like the Apple A9 GPU (iPhone 6S) is higher throughput (~150-200GFLOP/s achievable).

If you're executing these models at 30FPS on an HD stream (say ~1mm pixels/frame), you have roughly a 100 * 10^9 / (30 * 10^6 ) = 3,000 flops/pixel budget if you're operating at peak, which is about the same flops/pixel of a 16 channel -> 16 channel 3x3 conv. In practice of course you can usually operate on a much smaller spatial input.

With customized model u can do that. Models like MobileNet/SqueezeNet are designed specifically for fast inference on constrained device. Although during inference, the precision of your model can be quantized as low as single digit bits. This is an active area of research, but a lot of exciting stuff is happening and the progress made is massive.

Not really sure why you would use this over an Android/iOS phone and Tensorflow Mobile, or Jetson TX1+camera if you need more FLOPS at a small size

Jetson TX1 is much expensive than this one, though. As with most phones. And most importantly, this looks like a Dropcam style camera, I think the expectation of use case is different here.

Jetson is a dev kit. Pricing for a consumer product with the same chip is lower. You can get a Shield TV box for $180 today; plug in a webcam and it's equivalent to this Amazon thing but cheaper with 10x the performance. For a video processing application that extra performance is going to be essential.

Perfect! I will train one of these to watch my pained facial expressions when Alexa is triggered without using a keyword. Then it will fire a speech script "Bad Alexa."

Does that work? I usually say STFU! Which does work

Shameless related plug : We recently developed a deep-learning powered 2-axis-gimbal for a webcam : https://github.com/GistNoesis/Linn-Photobooth . We do use deep-learning either for deep-art or for pose tracking. I guess we can probably control the gimbal with this new camera though I'm not sure It will be able to process deep art. In my other usages, usually I have multiple cheap webcams with cheap board (IP cams or other) streaming it (over wifi/ethernet) to the GPU processing station located somewhere. With powerful GPU boards at €650, it's probably a better strategy once you start having plenty of cams.

That sounds cool, but after reading the webpage I am still not sure what it actually does. It prints a picture, you can change art style, what's the point of the gimbal?

The gimbal allows you to orient the camera (avoid people to touch the lens with greasy fingers), it can be substituted by a cheap 3d printed piece for manual adjustment if the functionality is not needed. It has some awe effect, and is fun with kids. It makes the camera more alive, specially in the tracking mode. The gimbal allows you to shoot videos where you are always at the center. We think this may grant some freedom of movement to some video makers and make their video more dynamic without the need of a buddy filming them. Also this project has a DIY component to it, so doing the gimbal is a great introduction to robotics.

Just make sure that if you're going to be discussing disconnecting DeepLens that you turn the pod so that the camera can't read your lips.

You can do much more with your phone's camera. I think a better use case would have been an SDK for Android/iOS. There are several open source libraries that you do a lot more interesting that simple object recognition.

They don't specify what this has to enable deep learning. Intel GPU? Atom processor?

"HOT DOG NOT HOT DOG" really makes it useful for adult film industry :D

I think it's rather a Silicon Valley reference.

And the amazon info site actually has a picture of a hotdog and the reference to "HOT DOG NOT HOT DOG"

Here's the clip: https://www.youtube.com/watch?v=ACmydtFDTGs

Wait.. why? If you were a company developing a product based on this, why would you want an AWS logo stamped on top?

I think it's more of a devkit. Not something you build a consumer product around.

Intel Atom processor? What a silly choice! A Tegra would be literally 10x faster for machine learning. Why on Earth would anyone choose Atom for this application?

Much better hardware for this application would be an NVIDIA Shield TV box and a USB webcam. 10x the performance for cheaper.

The tiny JeVois smart camera has been doing this for a while now, and lots of other kinds of computer vision too, although in a smaller and lower resolution device, also much cheaper at $49 and fully open source.

How soon before someone attaches a gun and legs to this thing? (see: Portal)

I know at the military level it does in the Phalanx https://en.wikipedia.org/wiki/Phalanx_CIWS which basically shots a stream of lead which must require one hell of a gimble!

I was imagining hooking up an old airsoft gun to one, legal, cheap, effective deterrent for small animals.

This is another example of tech beating the hell out of laws.

Am I the only one triggered by the fact that they used blue USB 2.0 ports?

Does this device "phone home"?

Reply on news.ycombinator.com