Hacker News Reader

Comment by deepnotderp | Hacker News Reader

I think that custom deep learning chips will be the best enabler of edge device deep learning, it's just too difficult to deploy anything useful onto most smartphones or other computationally constrained devices, and to compound this you often have to use the CPU because the GPU is unavailable either due to the framework or due to the drivers unavailable.

View on HN

I think that custom chips won't be made but just better graphics processors that can do inference even faster. Gamers that want more performance on their mobile games will push mobile graphics to the point that you will able to do gpgpu.

The problem is that even the Titan x drowns when doing real time inference, so a mobile gpu beating it without low precision is very unlikely.

That's news to me! I'm pretty darn sure the Titan X has no problems with inference on the amounts of data that a single user would want inference for.

Well, I was talking about real time object detection on 640x480 video. Perhaps most users would be okay with a 5 second or so delay when processing an image and perhaps you could use Facebook's trick of fast, bad quality style transfer and better quality style transfer once the image is in the servers. But the point is that the current paradigm is very restrictive in terms of deep learning applications.

I think computing optical flow remains a major time bottleneck. Any attempt at temporal coherence would be great, and I'm sure there is some attempt on Messenger, but it really only works well for the last style transfer filter, all the way to the right in the app, and that one only really looks great in well lit scenes. Also, the phone seems to heat up a lot.

I you would downsize the frames first and work on multiple frames in parallel (e.g. however many nets I can fit in VRAM). I find it hard to believe that it wouldn't work after these things.

What's the killer for doing real time inference?

Reply on news.ycombinator.com