The revolution of machine learning has been exaggerated
nautil.us
nautil.us
I think this is a problem that deserves more attention. A large portion of CS students nowadays choose to focus on Machine Learning, many brilliant CS students decide to get a PhD in ML in hope of contributing to a field that will continue to develop at a pace similar to the progress we have seen in the last 7 years. This view, I think, is not justified at all. The challenges that AI faces such as incorporating a world model into the techniques that work well will be significantly harder than what most researchers openly admit, probably to the detriment of other CS research areas.
I've not following that very much in a decade though. Did that whole area end up maturing?
From a pure numbers perspective 95% of jobs in our industry have nothing to do with any reasonable definition of machine learning. Even those applications of ML have a large amount of traditional CS going into acquiring, reformatting and storing large datasets.
(Of course it's a bit hard to assess the impact of a revolution like ML (esp DL) when your company already has hundreds of statisticians who have been employing similar data/experiment analysis techniques for decades, thereby diluting the signal of how disruptive novel forms of ML are within the enterprise.)
Isn't this a good thing? I would think that If it were easy we wouldn't need graduate degrees
Edit: if you don't get what I mean by "not be forgotten", he will be the most cited scientist so why AI will exclude all his papers from some of its training sets (assuming that at least some stage of AI's development it will learn how humans do research)? Sounds unlikely.
Edit: I realized these comments might be very mean to you (assuming you're the lab director with the hundred workers). I should disclaim that what I say is something I assume is true about most people who gloat about their citation counts, but it's certainly not true about all such people, and might not be true about you. Myself, I'm a bit resentful because I'm so bad at the whole academic game, so that probably makes me quite biased.
You make wrong assumptions and proceed. I am not that person and thus I don't get why you attack me personally.
While AI does not exist yet, that guy contributed to its development independently of your opinion about it.
Don't attack people on HN.
Don't attack people on HN.
Of course, there will always be a need for people with deep expertise, but unless you work in a very technical or prestigious field, how many compiler experts have you worked with?
you meant CS PHDs? because it is a better analogy. My answer would be with many, because I am one of them.
I mean, I know there are some shops that are doing really advanced stuff, but I'm talking about your normal tech hub employer.
See http://data-mining.philippe-fournier-viger.com/too-many-mach...
Roughly speaking, about 50k papers are published every year in arxiv/ML by perhaps 20k different researchers. Since some people still don't publish there, you may say there exists about 50k ML/AI/DL researchers.
Say 5k of them are pushing the field forward, most of them have a PhD. The remaining 45k deliver various local optimizations/adaptations, many of them don't have a PhD. Now it depends on the size of your normal tech hub employer. If it is big, then you have a few people from 5k and many from 45k. Otherwise, a few people from 45k.
I'm currently studying a part-time MSc in CS because not having one is notable here (and being around academic people has inspired me to learn more).
I think this is not wise career bet at all. There will always be spots for a select few 'pure ML' grads, but at the post phd level the really good jobs are getting pretty thin unless you come from the right adviser + right school + right publication track.
On the other hand, if you add a secondary useful skill, like having a really good understanding of distributed systems, or networking, or databases, or embedded systems, or being 'just' a solid overall software engineer, you (i) have something to fall back on if ML dreams do not work out, and (ii) have an edge by being just the right fit for specific teams, while also still competing in the general pool.
I think there will be a rude awakening on this in 3-4 years for many.
this is me. I can't recommend it enough.
[1] https://www.google.com/search?rlz=1C5CHFA_enGB720GB720&sxsrf...
It was one of the best classes I've taken and some of the things still feel like magic.
I do want to mention that on top of any self-studying, try to attend a talk or two or start following some feeds online that are close to actual healthcare operations. Operational teams are the ones who have to figure out what to do even when there is no good answer. the easiest way to keep up with which topics are most valuable right now is inside knowledge.
There is no purity in the OR field outside of phd's doing their research - it is entirely about getting shit done efficiently, however possible, to the extent that the operations team can understand. That last part of that sentence is a big catch. For example, if your 'solution' has interns making judgement calls on data entry (because moving the work upstream is efficient!), you are fucked if you assume that data will be accurate.
BUT there's obviously plenty of skillset stuff you can learn to help you in a general way so here are some important areas: 1: Linear & Nonlinear programming (tool: AMPL) 2: Markov Chains (good knowledge) 3: Statistics & probability (necessary knowledge) 4: Simulation (start with monte-carlo [it is easy and you will be surprised it has a name]) 5: databases: SQL / JSON / NoSQL 6: data structures and algorithms (big O notation / complexity)
OR work in general overlaps a lot with business analysis. The core stuff they teach you in school is listed above.
Healthcare right now has a big focus on Natural Language Processing, and applying Standardized Codes to medical charts - and then working with those codes. The most common coding standard in US is ICD-10 I believe.
Other than that it is mostly solved logistical items like inventory control systems that need a qualified operator. You do not want your hospital running out of syringes. You do not want your supplier to be unable to fulfill your order of syringes because you need too many at once. You do not want to go over budget on syringes because there's a hundred other items that need to be managed as well.
Now the important thing to keep in mind is that almost all operations problems at existing companies/facilities have solved their problems to SOME extent since if they didnt then their operations teams would have fallen apart and failed. So in practice it is rare that you are going to implement some system from scratch, or create some big model. Youre probably going to work with a random assortment of tools and mix data together between them on a day to day basis to keep track of when you need to notify people of things. With a lot of moving parts, you will have to task yourself with finding improvements and justifying them. Expenses are very likely to be higher than optimal, and you can earn extra value for yourself by finding incremental improvements.
No one is going to say: "work up a statistical model for me". They are just going to be doing something inefficiently with no idea how to do it better, and you are going to have to prove to some extent why doing it another way will be better - and also be worth the cost of training people to do it a new way. It will be monumentally difficult to convince anyone to make any sort of major change unless the operations team is redlining, so your best skill will be in being resourceful and adapting to the way things are no matter and improving THAT mess - not creating a new way of doing things.
Databases house the data you need to make your case. SQL was the norm, but a lot of stuff is JSON now. You might need to work with an API, add django to the req list.
Simulations let you test multiple scenarios of resource allocation on a complex system with moving parts (for example, resources includes # of hospital rooms as well as employees and their work schedules). Statistical analysis lets you verify the output of your simulations as meaningful or not. There are proprietary simulation programs that do the A-Z of this for you if you know how to configure it (ARENA), and theres pysim + numpy + pandas + ...
Markov chains are related to building a model for your system. It's theory stuff, but helps wire your brain for it. Laplace transforms are "relevant" somewhere in this category
(non)linear programming is the calculator of the fields distribution problems. In practice you create 2 files: a model file and a data file. Model file expects the data file to be a certain format, and is a programming language for processing the dataset.
For example, if you manufacture doors and windows at 3 locations, and sell them at 10 other locations, and you have a budget of $10000: how much wood and glass do you buy for each manufacturing location and how many doors and windows do you make and ship to each selling spot from each factory? The answer depends on data - price each sells at, cost for each plant to produce, capacity of each plant to produce, cost of transporting from factory to selling location, etc. So you make a model file for your problem and then you put all the numbers in a data file. You can change a number, run the model again, see if the result changed. You can script this to test many different possible changes
Data structures and algorithms: There are a lot of different optimization algorithms, all with different use-cases and theres no real upper limit on learning them; so this area can be a good time sink... since someone else will have already coded the implementation but you are providing value in knowing how to use it. Therefor - you dont need to learn how to make the algorithms or what magic they perform outside of whatever helps you understand what its good at. Outside of research, its unlikely this stuff will really get you anything other than maybe being able to impress at an interview - BUT who knows, maybe you find a use-case for some random alg that is career-defining.
I know a ranted a bit, and I didnt proof read, but I hope there was some helpful info in there
They might be disappointed graduating post-ML bubble, especially if they have their hearts set on half a million dollar salaries, but they'll do fine with a PhD in CS.
Then again, I'm not part of the SV world, so YMMV.
What's hot in technology changes all the time. The smart strategy for an undergraduate is to:
1. cover the fundamentals
2. take the honors track (where the professor and the students want to be there)
3. avoid "hands-on" classes, which you can learn on the job as required
4. take at least 3 years of math. Really. Don't avoid it. If you can't do math, you can't do engineering or science.
As for me, my degree is in Mechanical Engineering, with a side of Aero/Astro. And yet I work on compilers :-)
Trivially, being good at frontend coding and at UX design (cognitive aspects, etc) helps. Being good at designing software and being good at talking the language of your business is super helpful, this what's required at the director / VP Eng level.
And yes, do take the math. After just 2 years of university math (as an embedded systems major, I had to later study myself to grab some missing algebra, some category theory, etc, to grasp SICP on one hand, and Haskell on another. This helped me become seriously better at writing and understanding code in industrial languages.
I have a theory it was because, "they are honors students, they'll learn the subject regardless, and we need to put this bad teacher SOMEWHERE".
I once got out of a differential equations honors class because it was very obvious the first week, that the teacher was awful. Later, my fellow honors students told me how "lucky" I was to have switched. They were suffering through the course, while I was learning and enjoying my new class.
Would you recommend a specific branch of math?(discrete, linear algebra, etc.)
Also, which math or general skills/fundamentals would you say has helped you the most when implementing compilers?
My friend is doing a CS degree now and is being told by profs that you won't get a job without an ML background.
If you want to get a job in industry, you need to get advice from alumni that have (recently) gotten a job in the industry you are interested in.
You could end up at a medical school or a business school, a ton of mediocre PhD's take these jobs because Stanford and MIT never called them back. The medical and business schools think they hit the jackpot- they know they got someone mediocre, but at least they found someone. The mediocre PhDs usually end up happy because they found a job and medical and business schools usually pay well.
I'd say that a lot of the jobs that favor ML are from schools/departments that are playing catch up and trying to cash in because they saw a wave of grant money going to ML research. They are either smaller and less prestigious, or interested in the application to their field, rather than any direct interest in ML itself.
An ML focus probably isn't the boost you may have thought if you want a job at Stanford. The big schools have their pick of the litter and are already flush with ML talent, so being "an ML guy" isn't a magic serum that hides all your other flaws. You still have to be smart enough and positioned well (good school, good advisor) to get a job at Stanford.
Originally I learned programming, so I can write classical software, working from home. Never wanted to do anything else, but that does not pay. Only for AI research I could get funding
ML is sexy and pays a lot, sure, but it's far from the only thing going.
To quote the article, "Model-based reinforcement learning aims to address this issue by first learning a model of the environment’s dynamics, and then planning with respect to the learned model."
So they've sped up learning from examples by learning a model first, but it's still learning from examples.
These claims are massively exaggerated. Students don't "choose", "decide" etc. In the US, lets be generous & say there are 50 top universities with good CS/ML depts. After students beg plead & submit their GRE scores & recos & transcripts etc, each dept chooses on average 20 students for incoming PhD cohort. Of them easily 20% will wash out for sure. So atmost 800 top students graduating with a PhD in ML each year. Let me caveat by saying there are nowhere close to 50 top univs offering ML PhDs or having an intake of 20 per dept. So overall, the number is probably close to 200-300 students for the whole USA, not 800.
So all this handwringing for what some 200 kids will do ? I don't care if they do a PhD in basketweaving, in the grand scheme of things, 200 is not even a drop in bucket.
I'm also not sure about why the exact number matters - 200 kids matters a lot when future professors are drawn from the pool of students who have successfully completed a PhD.
I saw the same thing happen in the late 1990s as everybody and his dog got into web design while it was hot. Few of those folks are doing that now, nor did that skill translate well into other roles since the skill set isn't fundamental to other careers.
I'm not sure ML is any different, especially deep learning, since few companies have anywhere near the necessary amount of data to successfully play that game and win.
ML is pure alchemy for a business operating at scale. It’s as if a coal plant could turn its waste emissions into profit. You have this “data exhaust” from all the activity happening within your system that can be used to optimize your system Atleast a few percentage points beyond what is other wise possible. A team of 5 ML engineers can improve an ad targeting system by 5 % and if the company is google that’s billions.
ML creates feedback loops of improvements in product that improve usage that lead to more data which further strengthens the moat a business has.
It totally makes sense to jump on this train. It won’t solve AI but will make a lot of people wealthy optimizing systems.
If research stagnates, application willl also stagnate. For the industry money to keep pouring in there has to be growth and for there to be growth there has to be progress- scientific progress.
So if something is "not ideal for fundamental research" it is also "not ideal" for business.
Not going to pick up on your arbitrary 5% constant here, but please elaborate how these ML engineers are any different from anyone with a quantitative background?
At scale it makes a big difference.
It's very unclear to me that there are a large number of companies that can increase their efficiency by much more than 5% through ML. And it's not clear to me that there is more than a couple-year's-worth of projects in turning data exhaust into money for each company.
So if I were a new grad looking to get into ML, that might somewhat concern me.
You can just use a smaller amount of compute with transfer learning style stuff.
I can probably fine-tune a transformer model with my pocket money and beat any NLP solution from two years ago.
It is much easier to learn the basics of ML if you have good software engineering skills than the opposite in my experience. The one thing that needs time learning is experimental design and quantitative analysis, and this is rarely taught well at university before PhD.
The pragmatic, already realized effects of machine learning are mostly in improving existing systems, especially classification systems, and have increased metrics by big percentages at big data companies, and more applications are being found every day. That is why real demand and interest is so high for ML.
These generative, speculative models are interesting experiments by researchers but inconsistent and far overstudied by the media, because "AI invents new physics" is more understandable and sexier than "AI improves lead generation by 6.8%"
For example, there's already an ongoing replication crisis that's been ongoing for several years with ML papers in computer science. I've spoken with people in the industry who have even speculated that there's straight-up academic fraud taking place. If you get a splashy, sexy finding you can get scooped up by a FAANG-like company with a fat salary before anyone gives your work a second look.
The AI could be optimizing to attract people who are willing to give out their email address than actual potential customers so it doesn't help the business. We tried using Google ads to optimize for leads before and we saw ads were on sketchy websites promising people free stuff.
Lead generation was a bad KPI to use but it was chosen can't optimize for sales because the volume is to low to train the ML models. This is the reality of many businesses that experiment in AI and they produce junk.
ML does work well for certain domains where you have the scale of data to pull it off, but now the hype has spread where it is not appropriate.
I would contest that there are widely read articles saying those things. Yes, surely someone publishes those , but they are generally dismissed. the field is conscious of its hype (e.g. check r/machinelearning), and i don't think students are stupid either
>"It's part of the history of the field of artificial intelligence that every time somebody figured out how to make a computer do something—play good checkers, solve simple but relatively informal problems—there was a chorus of critics to say, 'that's not thinking'." AIS researcher Rodney Brooks complains: "Every time we figure out a piece of it, it stops being magical; we say, 'Oh, that's just a computation.'"
There is progress, though. Machine learning does do a lot. And it makes money, so effort will continue at a high level. AI used to be a dinky field - maybe 20-50 people at MIT, CMU, and Stanford. Now it's huge.
But AI in its meaning use contains a certain set of fields in a way that makes sense. It's just the puplic misinterpration nonsense that makes it look so weird.
I’m happy to concede that software is artificially intelligent just as I’m prepared to concede that most animals are naturally intelligent. It’s just a question of drawing an arbitrary line when things are intelligent enough to warrant the description. The AI effect is the process of that line getting moved closer and closer to “capable of doing everything an average human can do.”
I guess it seems very likely to me that we will have Skynet/paperclip maximizers before AGI. I mean, I think we kind of do right now in what the internet is growing into.
We're like single cells imagining what a brain would be like but we'll never know because probably we'll be part of something brainless that dominates the ecosystem, or if there is an organism with a brain, we won't be able to perceive it because we're components.
How about "the AI effect effect"?
This happens all the time:
- X suggests that success at a particular task is a good proxy to general intelligence.
- Years later Y solves that task with a clever hack and lots of computation, but no general intelligence.
- The media breathlessly suggests we've made progress on general intelligence.
- Z points out we haven't really.
- Someone says "Oh that's the AI effect!", by which they mean the goalposts have been unfairly moved on AI.
No, they have not. There's something called general intelligence, humans can do it, and Deep Blue cannot, and it's fine to say that Deep Blue beating humans at Chess was not thinking.
It just turned out that Chess wasn't as good a proxy to thinking as we initially thought, and mentioning "The AI effect" just clouds the discussion.
As AI is developed, it is also championed as "closer to real thought" and might someday solve the more general intelligence problem, only for those same experts to later realize that would never be the case and then retroactively proclaim "this was never going to solve general intelligence, but it is still AI".
The history is full of this kind of revisionism, so I agree with you.
That the current AI techniques couldn't work for every imaginable task, and that people believed they might isn't a case of the AI effect. That's just hype, and possibly another coming case of the AI freeze.
The AI effect will be how in 5 or 10 years, CS students will learn neural nets as part of their first or second year curriculum and the whole machinery will be behind a single API to some Apache library. And someone will then use that to some cool effect and say they're doing AI, and others will laugh and say, you're not doing AI, you just used a neural net to learn weights from a big data set silly, that's not AI.
This effect happens for every task, even when AI is not involved. Until you understand how, someone doing something you can't comprehend will have you think they are a genius and maybe of a higher IQ and probably very intelligent. But learn the "how" for yourself and it'll stop being so impressive. Rubik's cube is a good example of this. Once you realize there's a trick to it, it stops being as impressive.
Some relevant quotes from the wikipedia article:
> AI effect is: As soon as AI successfully solves a problem, the problem is no longer a part of AI.
> Software and algorithms developed by AI researchers are now integrated into many applications throughout the world, without really being called AI.
> AI advances are not trumpeted as artificial intelligence so much these days, but are often seen as advances in some other field
> practical AI successes, computational programs that actually achieved intelligent behavior, were soon assimilated into whatever application domain they were found to be useful in, and became silent partners alongside other problem-solving approaches, which left AI researchers to deal only with the "failures", the tough nuts that couldn't yet be cracked
> The great practical benefits of AI applications and even the existence of AI in many software products go largely unnoticed by many despite the already widespread use of AI techniques in software. This is the AI effect. Many marketing people don't use the term 'artificial intelligence' even when their company's products rely on some AI techniques. Why not?
While that is true the set that contains instances of Y is only growing and the members are each becoming less narrow.
Deepblue was only chess with many handcrafted parts. AlphaZero does chess, shogi and go with no prior knowledge. Then they went from perfect information, turn-based games to RTS with fog of war.
It's not general intelligence, but at least further down the road in behavioral complexity than jellyfish, despite some crippling limitations (e.g. separation of training and inference)
And when machines start to play chess (or go, or Starcraft) they just say "Oh well it's just computation."
So I think it's justified to bring up "AI effect" when people say "machines will never do $(insert your favorite activity here) because it requires general intelligence!"
It makes me think of the saying about atheism, that an atheism disbelieves just one more god out of thousands than a (mono)theist. You disbelieve in intelligence in very nearly all of the things it's been used to explain.
There's no such thing. General is a strong word. Humans can't for example handle more than 7 objects in working memory. We have an inbuilt limitation to how much complexity we can grasp. Programmers know what I mean.
General intelligence requires a general (infinitely complex and challenging) environment. Without it it will always be a specialised intelligence. Human intelligence is specialised in human survival, as individuals and part of society.
Also, we may not be able to keep some number N of objects in memory simultaneously, but we can definitely reason about an infinite number of objects, like I did above. And if you want to handle more than N objects, you just do it N objects at a time. You can always write things down etc.
You can always extend your memory, I mean.
Think about trying to explain the stock market to an ant. Then scale up, and imagine a similar situation for humans.
I think it would be very difficult to find a process that _cannot_ be understood by humans, let alon explain why. It'd be a bit of a paradox really. "Let me explain to you this thing that is impossible to understand".
Would it still be a general sorting algorithm if someone had just developed it and used it solely to sort numbers between 1 and 100?
It would be.
Why do you think you need to a general environment to develop a general algorithm?
No one is arguing human rationality is unbounded. But our intelligence generalizes, in a practical sense, to a great many more tasks than any computer algorithm I'm aware of.
Mainstream psychologists believe in general intelligence. They even attempt to measure components of it, with good prediction of performance on unseen tasks.
I think there's a big burden if you want to argue it doesn't exist.
Because intelligence is not intrinsic to the agent but the result of the agent trying to maximise rewards in an environment. In the end the driving force is survival - the agent needs only to survive, any method would do. So as soon as it overcomes a challenge it stops evolving and turns to exploiting. And the list of challenges that threaten survival does not scale to infinity. Anytime you think about intelligence you need to also think about the environment and the task otherwise it is meaningless.
citation needed
However stating "can't handle more than 7 objects in working memory" as fact seems a little too assertive to me :) Humans can do it, but are usually not very good at it as the number of items goes up.
So if it's not actually thinking, then technically, from the perspective of an average layperson, it is not real AI. However, the "AI effect" is still real, but it's related to a problem local to academics and AI laypeople.
Most non-AI-laypeople/AI academics believe AI is a machine that can think like how a person thinks.
He has however really turned this into a personal brand which many suspect he is using to advertise his startups. Many people feel he is sort of disingenuous about the actual progress being made in computer vision and NLP, even if there is many ways to go.
To answer your question: You can read a single piece of his and maybe get some value from that, but he keeps rehashing the same points to sh*t over different new work, and there is not much to be learned from that after a while.
Disclaimer: I did not read the above article in full, I am just familiar with Gary Marcus + work in ML industry.
Edit: Keep expanding this but it got me thinking. Earlier in my career, I was also eager for the 'gotchas', for being able to point out why something is crap, exaggerated, cannot do what it promises, and so forth. As I outgrew this impulse to focus more on what is good about some work, I began spotting it on new students a lot. They would come into seminars and smugly crap all over recent papers.
I have since come to the conclusion that it is useful to have realistic internal assessment of progress in your field, but that sharing your harshest criticisms is not necessarily a good way to look smart or make progress.
It often turns out that while feeling clever about realizing limitations of some work, the field just moves forward. The people you just criticized leapfrogged while you were busy thinking about how to one-up them.
Someone definitely needs to stick machine learning's nose into its own dirt, for its own good first of all, but it must be done in a very serious, stick-up-its-arse manner, focusing only on the practical aspects and leaving any emotion aside, and all in a calm and erudite voice, otherwise it won't have any effect.
Of course, now that I say this, there has been one such critique and it was ignored roundly, probably because Noam Chomsky Doesn't Tweet.
This is what I mean:
http://languagelog.ldc.upenn.edu/myl/PinkerChomskyMIT.html
Sorry, I can't find the edited interview (and the transcript doesn't quite get the "calm and erudite" voice across, what with all the uhs it transcribes). And I'm half-wrong about the reaction because there was a reaction, from a certain Peter Norvig, oh yes sirree there was:
http://norvig.com/chomsky.html
But I'm still with Chomsky on this.
1. Find a well-known classical problem with a battle-tested deterministic model representation
2. Generate synthetic data using the model
3. Train a 1-layer neural network to score 99% in predicting the synthetic data (even if the deterministic model can score 99.999% but let's ignore the minor difference)
4. Claim AI "learns" the underlying physics and does a better job than classical models
5. World dominance
1. Take one simple physics equation, such as F = ma
2. Generate 10^6 tuples exactly satisfying this equation (F = 6, m = 2, a = 3), with absolutely no redundant, erroneous, or extraneous information
3. Feed these to a primitive "AI" system which brute-force tries the simplest relations that could relate the three provided variables (m = Fa, a = Fm, F = ma, ...)
4. Recover F = ma, declare physicists obsolete
It's such a poor imitation of the real process of discovery in science that it's frankly insulting. When people hear AI is involved with something they turn their brains off.
Yes, I’m many fields ML has been exaggerated, but it’s just a matter of time. Most companies aren’t using ML effectively yet.
But I know for a fact when you improve NLP, you _can_ improve almost all white collar jobs. NLP the past two years has been crazy, and we haven’t seen anything yet.
For e-commerce search, a common problem is how to give relevant results for misspelled queries or unique queries. Text classification and/or NER can often identify the product type, brand, etc and hopefully give a relevant result which leads to a sale.
Another example is data entry or business analysis roles. Often they will manually extract data from sources and put it into Excel or some other BI tool. NLP ML has gotten good enough in just the last couple years that much of the data extraction can be automated, but you still need a human to verify accuracy.
* NER (name-entity recognition) is becoming insanely good (99+%), this can help with everything from data security to automated form filling, to automated medical billing, etc. Multiple billion dollar industries will be shook up by this.
* My own work on data generation is pointing to reduced surface area of security purposes and improved access to quasi-data for data science and application testing[1]
* Translation from images to text descriptions are also a major thing, which when combined with other systems can do things like make suggested diagnosis.
* Text generation systems are impacting how consumers interact. The chat bots these days are getting to the point where it's very difficult to identify if it's a human or not. I can't go into too many details, but see[2]
* There's also my side business which is a search engine for people who would know an answer to a questions: https://insideropinion.com/
In general, NLP is probably going to have a larger impact that driverless cars on our day-to-day lives. We're seeing insanely good story generation (e.g. GPT-2), chat bots, billing, etc. I also think WAY WAY more is yet to come. The transformer (which is spuring most of this) is only a couple years old. We'll see lots more applications as time goes on.
[1] https://medium.com/capital-one-tech/why-you-dont-necessarily...
That in part is why I don’t buy any of the “AI winter” chatter. We’ve previously never had machines powerful enough to run most useful ML nor enough data to train effectively. Today, as in right now, we have both. Things that were neat theories in the 90s are profitable businesses today.
Nobody in the ML field thinks strong AI is around the corner, or even 10-20 years from now. But that misses the point entirely. ML is useable and useful right now, and that’s not going to revert, no matter how many programmers grumble about “AI hype.”
Regardless, "machine learning" is a very broad field and honestly I have no idea what an "ML engineer" is doing if they are one. It can cover any of the following:
1. Cutting-edge academic research (do better on this test set)
2. Doing data analysis to identify prediction ability
3. Creatively thinking of useful features to evaluate.
4. Implementing data pipelines/logging to obtain the features needed for #3.
5. Production systems to evaluate/train ML systems. (multiple places in the stack).
Because the spectrum is so wide, if you are already an engineer, you can readily get into "ML" categories 3-5 and even 2. Andrew Ng's course is a valuable introduction and not that heavy of an investment -- I found that just with it (alongside my product and infra background), I could readily contribute to ML groups at my company.
It's interesting you put it this way. I think most machine learning researchers who aspire to do "cutting-edge" research would prefer to be the first one to do well on a new dataset, rather than push the needle forward by 0.005 on an old dataset that everyone else has already had a ball with. Or at the very least, they'd prefer to do significantly better than everyone else on that old dataset.
I bet you remember the names of the guys who pushed the accuracy on ImageNet up by ~11%, but not the names of the few thousand people who have since improved results by tiny little amounts.
* Enrolling yourself in a free online course, does not, at all make you a ML expert. A very sizeable portion of those enrolled may not have gone further than the first chapter.
* The ML train (as in people actually knowing ML) is not, at all, crowded
* Even if the train was crowded, learning ML does not make you forget what you already know. Saying "I don't want to learn this skill because so many people already have it", just means there are that many people that now have one more skill than you (unless you decide to allocate this time to learning something else)
I often feel that learning something new takes such energy and mental rearrangement that it does crowd out what I already "know". The brain is a neural network whose weights are constantly being adjusted, just because you learned something at one point does not mean it is permanently there.
For example, I spent many years working in networking, but now that I've been out of that field for several years and working in embedded firmware and data, I would have to relearn much of what I "knew" in my old field in order to be professionally productive. And that's apart from the field itself advancing in ways I haven't kept up with.
It's like riding a reverse-steering bicycle. By learning something that's in direct competition with your existing neural structures, your neural structures change and you no longer "know" what you used to. Jump to 5:10 to see this person, who took 8 months to learn to ride a reverse bicycle, try to ride a straight bicycle: https://ed.ted.com/featured/bf2mRAfC
This seems to me this is mostly caused by being out of the field for so long, not because you did something else while being out.
Science cannot even agree on a practical definition of intelligence. Unless you let biologists, psychologists and neurologists work with computer scientists to advance the discussion, I don't think it's worth it to explore machine learning techniques to pursue intelligence.
Don't call it ML or AI, call it "improved statistical prediction". The buzz will quickly fade.
It certainly works, but magic it ain't. You need an experienced practitioner and lots of patience.
27 years later, not just 99.99% of the humans in general but of my "fellow" programmers, if I can call them so, are still stuck at this petty ignorance level.
What, in your estimation, are some current or near-future trends that represent an objective improvement to the state of the art?
Train a model and then set it loose to do exploratory testing, looking for valid-looking inputs that cause the wrong answer to be returned from our entirely pedestrian Plain Old Business Logic. The AI proposes a scenario, but the human is the final arbiter.
It's very hard to not overstate when overstating brings you clicks, likes, retweets, and dopamine spikes.
The key thing here is that the core ingenuity of ML is statistics - the limits of which are well known.
ML hype is false promises based on uninformed wild extrapolation.
So rather than competing with the human brain...AI will try and trick the human brain instead.
I've yet to see a deep fake that's even remotely indiscernible from the real thing.
https://www.youtube.com/watch?v=k5y4kxhZIBA
A DF has a pretty specific meaning if you are diving into the technical aspects. It's an approach that has a lot of specific limitations that put a practical ceiling on how good it can be. For example, it's a 2D system that won't capture lighting differences with good accuracy. The limitations become more apparent as resolution increases.
Newer systems use source material to generate 3D facial models. The face in the target video is also 'motion captured.' The target video's lighting is estimated and a resulting face image is rendered and composited into the target. That's a lot closer to the pipeline of movies such as Gemini man.
So while DF isn't a technique used in movies, machine learning is heavily involved. It's built into model generation, motion capture, animation, etc. Artists will start with captured/generated assets and hand-tweak them to get the desired result.
The two minute papers guy made a material predictor to cut down on preview render time. Quixel uses ml for search. Substance and Houdini are gaining tools soon. Blender has used it behind the scenes for its new neblua tool. Mocap & animation are bathed in it. There's probably a lot more I can't recall right now.