Andrew Ng: Unbiggen AI
spectrum.ieee.org
spectrum.ieee.org
Not necessarily, at least on the first point. Someone could be getting coached.
A few years ago, a coworker of mine hired a contractor onto his team and was convinced the person who actually showed up was not the person who he interviewed (over the phone). He also thought the guy who did show up was getting a lot of help day-to-day from somewhere. The guy was a contractor, so it wasn't a huge problem because we could drop him quickly, but I would have never expected someone would do anything like that. However, it kind of makes sense as a scam: be a decent developer, get a stable of unhirable incompetents, and rotate them through companies while taking a cut of their salary.
The only way to prevent those kind of scams is to put all employees in probation for the first months of work and fire them if they don't perform, like it's common in the UK.
> designing a human process around pathological cases leads to processes that are themselves pathological
Eventually he got caught trying to manage all this
Possibility which, by the way, makes the interviewer's cautionary move generally useless.
I assumed that keeping looking in the direction of the camera was relevant (in their idea).
1. https://www.amazon.com/glasses-eyes-them/s?k=glasses+with+ey...
It's what you would do anyway, unless you suppose that the interviewee would ever install dubious software on his core machine.
Some of us believe instead on the advantage of being a polymath, (also) to be able to export wisdom from other contexts into the current work.
Also in terms of the proper ground to facilitate innovation.
"Jack Ma says his employees should work 12 hours a day, 6 days a week" https://twitter.com/i/events/1116787491707731968?lang=en
The fact that he tries this in manufacturing makes the case stronger. In most manufacturing companies you do not have access to top ML talent.
You have Greg who knows python and recently visualized some production metrics.
If we could empower Greg with automated ML libraries that guide him in the data preparation steps in combination with precooked networks like autogluon, then manufacturing could become a huge beneficiary of the ML revolution.
OR is perfect when you can describe explicitly what the decision space is and what the restrictions are.
ML is great fit when you want to identify and use patterns. Quality control with machine vision is a good application for ML. NLP for PDF documents is a huge field for manufacturing as well. Companies have so much data in email attachments that they do not currently take advantage of.
As opposed to having to figure it out later from the outputs of a black box?
> Quality control with machine vision is a good application for ML.
I can't imagine CV could be an actual replacement for actual SPC in many industries. There's a reason we need to take samples and stress test, analyze composition, etc.
> NLP for PDF documents is a huge field for manufacturing as well.
NPL could be big everywhere... if it provides actual value, which is not a given. ML has a lot of tangential applications (you could also say, better forecasting), but how will directly improve manufacturing processes?
I apologize for being abrasive, but I'm so tired of cs people descending upon all industries, plugging shit data into pytorch and doing shitty ML like it will automatically add value. Even more so in industrial engineering, which in my experience is full of people way better at math than computer scientists and requires a deep understanding of the product and the manufacturing process.
Not all problems can be formulated as a set of explicit equalities, constraints and variables (e.g. machine vision). If explicit modeling is an option, of course you should do it. I am seeing efforts to try reinforcement learning on systems that we know how to describe with equations, and of course the results are laughable compared to the traditional methods.
> I can't imagine CV could be an actual replacement for actual SPC in many industries. There's a reason we need to take samples and stress test, analyze composition, etc.
In one big manufacturing company they were using Machine vision and a cheap web camera to control flaring. Could they do it with fancy sensors instead? Of course, but it would be more expensive, and they never did in the past.
Another manufacturing company is using machine vision to raise an alarm if the door of a cargo car of a train is not closed after loading. Could they install sensors in all of the doors of the train instead? Sure, but it would be cost prohibitive.
>NPL could be big everywhere... if it provides actual value, which is not a given. ML has a lot of tangential applications (you could also say, better forecasting), but how will directly improve manufacturing processes?
In manufacturing we have multiple people opening pdfs from emails to copy contract numbers to excel spreadsheets. Others are getting orders in emails and then type them in SAP manually. I think that these tasks can be automated specially with the recent versions of NLP networks.
>I apologize for being abrasive, but I'm so tired of cs people descending upon all industries, plugging shit data into pytorch and doing shitty ML like it will automatically add value. Even more so in industrial engineering, which in my experience is full of people way better at math than computer scientists and requires a deep understanding of the product and the manufacturing process.
All is good :) There has been a lot of unsubstantiated hype in ML, made even worse by big consulting companies and cloud providers who just sell the hype.
However, that said, in a tightly controlled environment such as a manufacturing line trying to spot defects I would imagine they would have a good chance at performing a lot better than deep learning.
A lot of the advancements in deep learning have also come out of ideas from that research. While they didn't use the techniques directly, there is a lot of knowledge that we'd be lost without.
This is one thing that scares me about ML. We are losing research into the fundamental physics/science to deeply understand these things and instead just throwing models at them.
However I have seen CV and NLP useful here and there... but it is not the bread and butter.
so funny, because so accurate :)
My thought is that Goldratt's "The Goal" / theory of constraints is a useful way of thinking about optimizing throughput in a computer system. http://www.qdpma.com/Arch_files/RWT_Nehalem-5.gif plus an instruction latency table is something like a well modeled factory. (The Phoenix Project applies these principles to project management, which I think is a somewhat less useful analogy!)
I'm curious about applying existing tools to modeling things like: how will this multi-tiered application behave when it gets a thundering herd of requests? What if I tweak these timeouts, adjust this queue, make a particular system process requests on a last-in-first-out basis? Can I get a pretty visualization of what would happen?
Big data is fairly important to a lot of things, for example I was listening to Tesla's use of Deep net models where they mentioned that there were literally so many variations of Stop Signs that they needed to learn what was really in the "tail" of the distribution of Stop Sign types to construct reliable AI
You brain already knows how to select the most important features of a sign. The shape, the size and the color. You have also learned how to understand the text on the sign.
A new born baby does not have that ability.
This is applied in ANN as well. Transfer learning is using a pre-trained neural network, which has already learned identifying objects, and then using it to train on identifying a new, usually smaller, set of objects using, usually, a lot less training data. That is what Andrew is talking about in the article.
And the context. For example, self-driving cars need to account for "Pizza Stop" restaurant signage, placards stuck to telephone poles that say things like "Stop Cancer", stop signs retracted into the sides of school buses, signs with additional instructions like "Stop when lights flashing", road workers with handheld stop signs, and the unconventional stop signs you see in parking lots.
You can probably get pretty far by checking the proximity to the road, height, dimensions, orientation, what it's mounted on, and if the sign incorporates any other text. But you can't just scan some pixels for "red octagon with STOP on it".
I have to disagree. They spend an inordinate amount of time trying to understand what they see, taste and hear.
> able to correlate them with prior information that was retained and learned
This is what we call inference.
> Case in point: if you take a child born blind, give them the ability to see, they are immediately able to recognize and correlate objects around them.
Not everyone who is legally blind can see absolutely nothing, but people who have recovered from complete vision loss [1] have problems. Mike May [2] lost vision as 3 year old child and regained it in his 40s. Despite seeing for the first three years of his life, years after regaining vision he was unable to see in 3D or recognize people from faces alone.
Blind people do not lack spatial awareness, so being able to recognize objects with context if they regained sight with would not surprise me. There are blind people that can "see" with echo location using parts of the brain associated with visual processing [3] But for example in Mike's case, he was unable to recognize close family by their faces years after regaining vision, he needed additional context.
Many things we take for granted as being innate to the human experience, are in fact learned (trained) behavior.
[1] https://en.wikipedia.org/wiki/Recovery_from_blindness
To understand yes, however I meant "identify" in the sense of recognizing an object. I show a picture of an apple to a child a bit over a year old, hide it, put it in a basket of other things and present it back to them, they will be able to identify it. Or if you show an object to a child, then hide the object, they can realize that the object is no longer there, regardless of their understanding of what happened to the object.
> Many things we take for granted as being innate to the human experience, are in fact learned (trained) behavior.
I don't think it's that simple, I would say it's both. I don't question learning plays a big role in recognition, just pointing out that a large amount intrinsic knowledge also exists from early child development. Many of those cases where someone regains eyesight happens much later in life, at a time when their brains have largely matured to the point that neuroplasticity is pretty much over for them. Having someone's brain develop with almost no visual input at a young age is bound to mean that their visual cortex and its connections to everything else doesn't develop as it should.
From a quick search, it appears that to some degree children born categorically blind can recover all the way up to teenage life. [1] But indeed it's likely less effective than a younger child undergoing a similar procedure (which I don't think is really that rare: it's hard to diagnose vision problems at young age, and a lot of children who get necessary corrective surgery at young age turn out fine).
We can go back even further - your genes carry information about the structure and function of your brain and this has been refined by natural selection over the course of human evolution. Humans don't start from scratch with randomly initialised weights.
Like the NN State of the art models of today are so different from state of the art 12 or so years ago which was SVMs.
FTFY.
Yet Tesla have been working on both the hardware and software for 10 years? Amazing progress right?
It seems the right parallel to evolution might include predecessor inventions in vision, computation, and beyond.
For instance, an English speaker and a non-English speaker may listen to someone speaking English and while the auditory signals received by both are the same, the meaning of the speech will only be perceived by the English speaker. When we’re learning a new language, it’s this ‘knowledge’ aspect that we’re enhancing in our brain, however that is encoded.
This knowledge part is what allows us to see what’s not there but should be (e.g. the curious incident of the dog in the night) and when the data is inconsistent (e.g. all the nuclear close calls). I’m really not sure how this ‘knowledge’ part will be approached by the AI community but feel like we’re already close to having squeezed out as much as we can from just the data side of things.
Somewhat related, we have a saying in Korean – ‘you see as much as you know’.
Animals solve this problem by having bodies and moving around. It is that we take the bent stick out of the water which allows us to impart a theory to the "data" we receive... a theory implicit in our actions.
Since we are causally active in the world, sequenced in time, and directly changing it -- our bodies enable us to resolve this problem. The motor system is the heart of intelligence, not the frontal lobe -- which is merely book-keeping and accounting for what our bodies are doing.
The notion that 'all we need is data and data is all there is' seems to summarize a lot of ML sentiments.
It does in general, but what is elaborated and how? Structuring patterns is not the same as "knowledge" (there are missing subsystems), and that fed data is not fed efficiently, with ideal efficiency - compare with the realm in which "told one notion you acquire it" (this while CS is one of the disciplines focusing on optimization, so it would be a crucial point).
Anyone knows if this might be true mathematically speaking? Does order of data matters?
Did this make any of you a little queasy?
it's easy to get complacent and focus on building big datasets. in practice, looking at the data often reveals issues sometimes in data quality and sometimes scope of what's in there (if you're missing key examples, it's simply not going to work).
most ml is actually data engineering.
I wonder if, assuming the data is of highest quality, with minimal noise, having more data will matter for training or not. And if it matters, on what degree?
In general you want to add more variants of data but not so much that the network doesn't get trained by them. Typical practice is to find images whose inclusion causes high variation in final accuracy (under k-fold validation, aka removing/adding the image causes a big difference) and prefer more of those.
Now, why not simply add everything? Well in general it takes too long to train.
How do you identify these images? It sounds like I'd need to build small models to see the variance but I'm hoping that there's a more scientific way?
Of course few shot learning is important for models, but for example for Pathways it was already part of the evaluation.
At a first glance it seems like the hassle of integrating such a product into an existing ML codebase/pipeline is larger than solving the problem by hand.
I also want cars that run on salt water.
I'm not saying that small data ai is equally impossible, but simply saying "we should make this better thing" isn't enough.
Besides the references to his company which has customers and a product that already works on these principles the literature currently shows that this is very much possible if you dig into the correct niches. Besides the SOTA in few-shot and meta-learning it is possible to smartly choose the correct few samples for the network that yield the same results.
It has also been my primary focus for the past 5 years and the core of the company I founded.
And then, someone is using pretrained 500B model, and fine-tuning your few examples, and getting new SOTA.
Simplest way to weigh by sample efficiency: multiply accuracy by ratio of test set to training set sizes. Everyone's training/testing on 80/20 splits, so everybody's SOTA would go down by 3/4s.
Already in 2018 SenseTime reported that for face recognition, clean dataset surpasses accuracy of 4x larger raw dataset.
Only, the article seemed to show a very conservative Ng about the algorithms, a focus on data management - so it's still ML.