Andrew Ng: Unbiggen AI | Hacker News Reader

Andrew Ng: Unbiggen AI

spectrum.ieee.org

16 pointsleptoniscool8 comments

Totally. With computer vision especially, you often want to deploy the model on an edge device with limited bandwidth, power, and footprint (think remote camera monitoring, drones, robots, and autonomous vehicles).

Requiring a huge desktop or server-grade graphics card (much less a box full of many of them) to fit the model into memory misses the mark.

We’ve done a lot of work getting models to be performant on the Luxonis OAK (OpenCV AI Kit) and NVIDIA Jetson devices.

I've had several discussions with friends about how I think AI should be more modular. You could have a general model which, eg. classifies an object as a "fruit" then passes it off to a separate, more specialised model which could classify that fruit as a "banana".

This way you can improve your fruit classifier without needing to make changes to the general classifier. I think it also opens up the possibilities for things like having a general "offline" model on a smartphone but when connected to the internet it could make use of more specialised models.

It would also be cool if you could download offline models for things you're particularly interested in, like birds species etc.

I think one of the problems with having a really large AI that attempts to classify everything would be a sort of "tunnel vision" problem and eventually the AI has to make a guess as to what something is instead of saying "best I can do is this is an animal, but let me go ask a buddy of mine who's an expert on animals".

It's an idea reminiscent of Marvin Minsky's Society of Minds , viewing intelligence as the process of works of diverse agents who solve specific problems in their own ways rather than relying on one formalistic 'master algorithm'.

https://en.wikipedia.org/wiki/Society_of_Mind

If my memory is right, modelling like that does poorer overall than having your general classifier find the banana.

The mental models we have to describe these things don't match well to how the ML breaks down such classifications. Finding a fruit without having already figured out that it's a banana is pretty hard.

So then you're kinda stuck with "looks like a cylinder" so your second one could distinguish hotdogs from bananas, but that's already what the layers of the neural network are doing

> ML breaks down such classifications. Finding a fruit without having already figured out that it's a banana is pretty hard.

Can you elaborate? If I train a fruit detector that just gets the fruits bounding box vs a fruit detector and classifier, are you saying the latter will work better?

> You could have a general model which, eg. classifies an object as a "fruit" then passes it off to a separate, more specialised model which could classify that fruit as a "banana".

The problem is that the distinctiveness of “fruit" vs “not-fruit” to a visual system is less than that of “banana” vs “not-banana”. Visually, we don't to top-down classification in an ontological heirarchy, and there's no reason to think that would be an optimal (or even reasonable) approach for AI, either.

I am not sure why Ng seems to suggest he pioneered the use of GPUs for ML. It was pioneered by the Toronto lab led by Hinton, afaik.

It seems like he was one of the people advocating using CUDA for deep learning before Krizhevsky’s big breakthrough in 2012.

Not a primary source: https://medium.com/analytics-vidhya/gpu-for-deep-learning-7f...

Alex Krizhevsky’s home page has a bunch of CUDA ConvNet stuff from ~2010

https://www.cs.toronto.edu/~kriz/

Edit: found a better source. Deep learning on GPUs dates to 2005, and Ng’s group has the first GPGPU ConvNet report in 2009, and Krizhevsky has the breakthrough in 2011/2

https://www.deeplearningbook.org/contents/applications.html

Reply on news.ycombinator.com