Mark Cuban on Why You Need to Study Artificial Intelligence
bothsidesofthetable.com
bothsidesofthetable.com
Meanwhile, statistical methods for non-image/text data with identifiable features can often work better than neural networks, but they are not as sexy. (Good discussion on HN about this: https://news.ycombinator.com/item?id=13563892)
I'm curious what Fisher, Neyman, and Pearson would say about the current state of the field. Especially considering how often they and other statisticians disagreed with each other throughout the 20th century.
As for deep neural networks, in my opinion they are just an ensemble of neural networks, and ensemble methods have been shown to produce best-in-class results almost regardless of the base learning algorithm.
Also the whole sport analytic prediction industry mostly uses linear regression... But next hype thing is medical data for them but they're super slow in adopting new thing. Industry seems boring if you're not into sport and into modeling in general not just linear regression.
Many of them don't know how to code. And the stat program of most of these universities doesn't require any real coding classes. The professor give you code templates and you tweak it to get the result. But the stat classes are really good to handle data.
Data science in general like Chapman Hill and UC Irvine seems like a really wide range of stuff from data analyst, a lil programming, data visualization, etc... I don't feel like it goes deep enough in what Statistic does imo.
Most are stuck with Tableau and SAS. Pharmacy companies uses SAS. Startup companies and tech companies uses Python and R.
I'm speaking from one data point sample, my current stat grad program. My undergrad is comp sci and I did a few years in the tech industry before coming back to grad school.
If you don't mind sharing details; why'd you choose your current grad program?
Maybe we should explain the cost/benefit of the buzzwords vs. the science?
Most of the "dinosaurs" around here should know by now that you don't get paid to do what works; you get paid to do what the boss/customer wants. (And that's not always going to be exactly what they asked for, either.)
As long as you continue to throw money at me, I'll continue to chase your wild geese, hunt your snipes, fish for your red herrings, and joust with your wind turbines. Most of us here can learn fast enough to always stay one or two steps ahead of the boss on whatever topic they might latch on to. And that becomes genuine experience if it somehow manages to graduate past the business fad phase.
And even if you don't want to use SGD to fit your model, the deep learning machinery can be reused for other things like bayesian inference, eg PyMC3 and Edward are built on Theano/TensorFlow respectively.
There have even been papers showing differentiable trees integrated into the deep learning framework. There is work on how to input graphs.
Maybe this is just my own desire for unifying frameworks shining through, but joint training seems to have real benefits and feature learning has clearly had a large impact on reinforcement learning.
Maybe it's not the best tool for day to day data science work, but that tends towards what tools are accessible and good enough, rather than what is best.
CNN's are great for some image recognition/classification problems, but that's just the start of the conversation.
Interestingly, the parsimony principle is lent empirical weight by bad results obtained from overfitting.
Machine learning enables us to multiply many more entities than we can using our conscious thought processes. The various image recognition models use many variables that enable results impossible with other techniques. It does not violate parsimony if you can obtain better results on a large data set.
I can predict that if a car drives by it will not teleport to some other location, but will rather continue along its path. I've been able to predict this since I was a child. I did not need to study the laws of motion to do so. I would not take such a prediction with a grain of salt.
I suspect that others using ML, who are inclined to test hypotheses, induce cause, and discover unknowns (via generative ML), are far fewer.
So it's hardly surprising that the eye of discriminative MLers dominates in defining 'sexiness'.
To be frank, the tech community has really been leaving a sour taste in my mouth the last few years. Everyone is anxious to bandwagon on these buzzwords that provide little to no benefit or are applicable only to a small segment of the tech population.
In most cases, even a superficial understanding of the problem space should make it obvious that $This_Weeks_Sexy_Solution is a very bad fit. Do you need a server that persists data and is individually addressable? Then why are you using Kubernetes and Docker, which are still struggling to figure out these very basic things? k8s and Docker have very specific uses, but unless you're Google, they're probably not the right fit for your production environment right now.
This phenomenon seemed to hit a critical mass with document databases and single-page apps, and it continues to iterate with every open-source release that comes out of Google or Facebook. Since Google released TensorFlow last year and the compounded hype of [much worse than advertised attempts at] conversational speech recognition in Siri and Alexa, the "machine learning" bandwagon is starting to try to edge into the spotlight, and it sounds like it's already a mandatory part of any VC pitch.
Fortunately, machine learning is pretty hard and you get into hairy math practically right away, so I don't think the legs on this one will last as long. But we'll see. We'll at least have a lot of faux-ML going around and a lot of people making spurious claims on their resumes.
The economic collisions that make Silicon Valley and the tech industry in general a hive for inexperienced and insecure youth are bearing some really interesting effects this way. How can a company that's not "blown about by every wind of [tech fad]" fully exploit its relative sanity for competitive advantage?
[0] https://news.ycombinator.com/item?id=13572415 ; my reply downthread at https://news.ycombinator.com/item?id=13573978
It's the same urge that makes us all think we're better drivers than average, better looking than average, etc. It's the urge that make people move to Silicon Valley/Hollywood/etc., because everyone's going to "make it" even though statistically, that isn't possible. It's why every parent with the means pays through the nose to get their kids into Harvard, why SUV sales are high even though most people don't off-road, and why people buy technical outerwear for climbing, etc. if they never go near a mountain.
Everyone has a constructed image of themselves and we're lying if we say we don't try to burnish it. I think using all this tech is an expression of the desire for status. The problem is, it really can be a time-consuming, frustrating, customer-data-losing, unsupportable, waste of resources. That's bad for business, bad for your health (spending time fixing shit rather than eating well or going to the gym), and it can make your friends and family irritated with you, too.
I'm building a product right now on the most boring stack possible: python, postgres, pyramid. I've been there/done that on the whole sharded-MongoDB-Redis-Memcache-900 machines-AWS-Docker-startup-ops nightmare-use all the things (we actually used every single thing in that list at my last job). It was unreliable, a huge pain in the ass, a little bit fun, but ultimately quite counterproductive. These days, my bets are on things like the JVM, the .NET CLR, and Postgres. As Knish from Rounders would say, "My kids eat".
Another thing to keep in mind: a lot of this is resume padding. Yes, this is completely unprofessional, reckless behavior, but it happens, many recruiters are keyword-driven, and as interviewers many of us are impressed with a keyword-laden resume. We do this to ourselves as an industry in how we hire, who we let speak at events, and who we praise. It's nobody's fault but our own.
My sibling post is on the right lines - a lot of people in tech are chasing a delusional self-image - though I don't think that's true of everyone. People can make the effort to be self-aware and honest with themselves. I think the rapid expansion of tech in the last few years attracted many, many youngsters, hucksters, delusional types, and so on, and people with maturity and experience became a minority.
Aside - I think it's really ironic for people with no technical know how to go on a talk show and say "learn ML or you're toast". First, the inertia of the world economy is much greater than 3 years. Second, it's unrealistic to ask every 7-11 cashier to go learn ML. Third, programming is a complex art and we're surely not 3 years away from the singularity. Most people I know who have some ML knowledge also realize how far we are from a computer program that can read code, read specs, and then decide how to implement or fix existing code. And last, even if all of the above were moot, I think it's more likely that we write a program that successfully helms a Fortune 500 company before we write one that produces or improves on software. The former could be more easily reduced to a game of chess while the latter requires some massive novel research into "creativity". Maybe it's Mark Cuban who should be learning ML!
That's exactly the wrong way to think about it IMO. My vision of Machine Programming is that it's a fluid OS/UI that adapts application architecture to how people actually use them. So for example you have "Deep Excel" which is trained first to give the basic functionality of a spreadsheet, but as each person uses it, it gets data about most used/combined UI elements, follow on actions etc... that "rewire" through RL the UX to behave in the way that the user indicates it wants to behave.
So each instance of the program running would be unique to each user and constantly changing to make it more adaptive to their needs. Network these in a deeper system and you can iterate on design and functionality of products way faster.
Didn't MS add a "dancing UI" to office at one stage?
As far as ML, we have seen this before. When the tools get cheaper and easier to use, adoption goes up. RoR + cheap cloud servers and easy VC financing greatly increased the number of startups.
Now TensorFlow and the Nvidia Pascal architecture have made ML accessible and fast.
If there is one thing history has shown us, it is that the Hacker News comment section is full of myopic pessimists with an axe to grind (no offense peers!) -- just go back and look at the Dropbox launch thread here.
Even if ML doesn't meet all the hype (it won't for a time, and then later on it will probably exceed it and create things we can't even dream about right now) there will be an entire wave of ML startups, products and jobs.
There is a ton of money to be made on the ML train guys. It's the new social-local-mobile. Much will be shit, some will change all our lives for the better.
I'm old enough now to know how this works.
Edit PS.
I've heard listenable, interesting music completely generated with tensorflow. I have seen artwork that is arguably as good as anything any of the greats have put out generated through similar methods, and we are in the infancy of machine learning.
Machines have beaten humans at chess, go and most impressively in my opinion Jeopardy!
ML is real and it's here and you should leverage it as much as you can, all the knowledge and tools are free.
It's patently ridiculous to say that we are currently living in the singularity, though, it defies the very meaning of the term.
"The accelerating progress of technology and changes in the mode of human life, give the appearance of approaching some essential singularity in the history of the race beyond which human affairs, as we know them, can not continue"
- John von Neumann
I would argue that the intelligence explosion has already occurred. Human affairs as we knew them when that line was uttered have been fragmenting and crumbling all across the world.
The internet itself, the current 10 billion or so machines networked together have resulted in a fundamental change in the human condition.
All of us are operating with amplified brains now. We have access to knowledge from the entire corpus of human information at the touch of our fingers, or now with our voice.
I would say human intelligence is already machine amplified.
The runaway reaction has already begun.
I've a PhD and held ML-engineer positions in a few different companies - I've good industry awareness.
Most applied ML, for most companies, right now, is actually relatively simple models (hand-coded rules! logistic regression! You'd be shocked how common these are.) The bulk of the work is data cleaning, gathering, integration, deployment, productisation, reliability, avoiding pathological cases, special-casing, Product, UX. You do need ML specialists who understand the stuff, to make it all work and come together - but the ratio of ML specialists to the wider team is low. Maybe 1 or 2 specialist on a team of 10 for an ML heavy product.
This is going to remain the case IMO. Yes, there will be small teams, in highly resourced organizations (GOOG, FB etc), academic research labs, or occasional hard-tech startups, who do new model development. Maybe if AI becomes huge, you'll see more traditional Fortune 500s spin up similar efforts.
But there'll be a much wider set of people&businesses applying and tuning well understood approaches, rather than doing new model development. And you just don't need as many ML specialists, for that approach.
Even with deep learning, the tooling will advance. I mean, even look at all the research papers describing applications at the moment - so many of them are using pre-trained models. Industry will be similar. Tooling will advance, and you'll be able to do increasingly more with off-the-shelf pieces.
I think ML is absolutely going to have a big impact - I buy at least some of the hype. But should all developers, or even a substantial minority of developers, start learning ML as a career imperative? I don't think so.
Finally, it takes serious time to learn this stuff. Its easy to dabble (and worthwhile doing - its fun; and sometimes you can do powerful things in a using tools in a very blackbox manner!). But actually thoroughly learning it takes time. It takes serious time to build statistical intuition, as just one example.
We could easily end up with a great many career developers who have a specialization in ML, frustrated they never get to use it.
Basically, the interfaces of the models/tools/abstractions people will use will be 'leaky'.
For example, you can take a machine learning method from scikit learn, which works really well on the scikit learn example, and apply it to your problem.
Any developer can do this pretty quickly. If it works, and gets good accuracy out-of-the-box, then great.
But if it doesn't work, what do you do? How do you know where to look, what could trip it up? Are your features OK, or is it a problem with your model? Or maybe you framed the problem wrong? When you get into this sort of area, that's when you need an ML expert, who knows whats actually going on under the hood - or can at least learn the particularly model quickly - and can make progress faster. A more general developer will slow down drastically once they start reading the documentation for how the model/system actually works.
And hopefully the expert will frame the project better from the very start, solving issues before they even arise, because they know the kind of issue that can occur, or suggest easier paths to solutions.
So, agree with your point there.
But how many such experts do you need? In my experience, only a small number on a bigger team. ML folk are highly leveraged, but need a lot of support to get their product into production - to manage the data (if its worth applying ML to, there's probably a lot of data, so a big data engineering task, maybe connected to a live system), to think about the UX etc.
This will all evolve on several levels of course:
- The tooling will get better; model/data deployment/management will get easier. But also and non-experts will be able to get more done as ML becomes more robust out-of-the-box.
- We'll get better at building ML products (e.g. team structure, data infrastructure, UX (designers learning how these things work), company org (e.g. a lot of friction between Agile and ML))
- Businesses will want to do bigger things
But if I had to guess, we'll be picking up low-hanging fruit for a while, and most of the work, for most companies, will be in the support infrastructure and application, with just a minority of specialist ML roles.
I mean, even look at all the research papers describing applications at the moment - so many of them are using pre-trained models.
I think to the broader point that generally learning how to validate a ML technique is going to be very useful to a lot of people, implementing them less so.
And let's take a look at AlphaGo, how would you do that with SVMs or decision trees? Just get over the fact that deep learning provides a level of "intuition" (Go's search space is famously greater than the estimated number of stars in the universe).
I think that a part of the problem is that older ML PhDs are angry that deep learning is so easy (until the learning rate fails to provide convergence of course...) and would prefer that their preferred methods would still reign supreme.
I'll end this wall of text on the note that OpenAI's Ian Goodfellow notes that all projects at OpenAI use deep learning right now, but they are not dogmatic and will consider other approaches if they work well. I think this is the path that should be taken. On the other hand, I also see a bright future for uniting traditional techniques with deep learning such as attaching a SVM to a CNN and combining decision trees with CNNs both of which have resulted in good results.
I think that's definitely a part of it, and I feel that way sometimes myself (not that I'm a PhD). But there's another side of that reluctance that lies on the axis of model accountability and explicability. A lot of modern ML/Deep Learning doesn't -feel- like we're understanding anything any more than we did ten years ago. Yes our black-box results are better according to the tests we've laid out for them, but there's something more slippery about the 'why', beyond the handwave of 'complexity'. Maybe this is just the way it will be going forward (in the spirit of Quantum's "shut up and calculate"), but it is not easy to give up something that you can wrap your head around with something that kind of just takes care of itself, especially if you're in the business of seeking knowledge instead of results.
On the other hand, we're nowhere near human level intelligence in most tasks (Go, Poker and image classificaiton non withholding) so I can understand the argument for explainability from a practical perspective. I think we're making some good progress in that direction though, and I'll list them below:
1) Attention maps in CNNs can tell us what the net is usually looking at.
2) "Attentive Explanations" use attention mechanisms to point to the object of interest to generate an explanation for VQA tasks, check the paper (warning PDF): https://arxiv.org/pdf/1612.04757
3) A recent project used a similar explanation mechanism that forced the network to output "what it was thinking" while playing an Atari game.
4) NTMs (neural turing machines) allow weighted memory access, which alleviates the back box issue to some extent.
I think this might be where the real divide lies. To present a false dichotomy, there are two schools. One that is interested in building systems that attain human-level intelligence, and another that is interested in understanding the nature of human intelligence by building models using our closes understood analog (computation).
We're definitely making progress on Deep Learning models that tell us more about how they think they work, but philosophically, I'm not sure we're making progress on understanding the nature of what they experience. I don't mean this in any polemical sense, just trying to explain the particular dissatisfaction (and it -is- dissatisfaction, not contempt or disdain. The stuff is amazing) I feel towards these advances.
There are two types of explainability. One is explaining why the system converges on correct solutions in general. Another is explaining why a particular decision was made the way it was made. Human intelligence is usually very much explainable in the second sense. This is not always true, but such cases are called "intuition" and we tends not to trust such judgements for mission-critical tasks.
Also; the nights in the prison cell contemplating that you are held responsible for offing some poor sod. Not so good.
I'm currently enrolled in the Udacity Self-Driving Car Engineer Nanodegree, and while that program is focused on applying ML techniques to a specific problem domain, I can see where and how to apply what we've learned so far to other problem domains. Furthermore, I have in the past taken other MOOC courses (Ng's ML Class in 2011, and Udacity's CS373 in 2012) which taught more generalized approaches (more of the "standard" ML concepts in Ng's course, less so in the CS373 course - but I should note that even in the course I am taking currently, one or two of the lessons did discuss and talk about GD and other simple mathematical models for ML for classification, etc).
I recently completed (late) the "behavioral cloning" project, where we had to implement a CNN to drive a simulated car around a track. At first, it seemed like it would be a simple challenge - slap the NVIDIA model on the thing, gather a bunch of data for training/validation, and it should all "just work".
If only.
I struggled immensely with that one; when trying to explain the problem to "outsiders", I kept running into the issue (which I knew about) of how to describe the fact that this system wasn't something I could easily explain. I can't tell someone "oh change that rule and tweak that test and it does this" - because it doesn't. Its not that simple. I know this, but conveying it to someone else isn't easy. I've found that these systems, for their simplicity compared to, say, a mouse's brain - share the same kind of "black box" issues - I can't tell you why my car likes to veer at a certain point, then recover. I never drove like that with my training data, but for some reason, somehow - that's what it learned.
From a business perspective - this is where things will fall down. We (that is, those people in the deep learning ML domain space) can't explain how these models work. In some ways, they don't seem to be deterministic. In theory, post training, they should be - but in reality, short of perfect data being fed in - the data they get in their inputs is never exactly the same - their ability to generalize and fit this imperfect data, with a result that correct, within a level of error - is a strength, not a weakness. But try to convey this to someone where money or exact results are on the line...
Researchers can't even explain how the C.Elegans connectome works, and it is vastly simpler and completely mapped (interestingly, embodying the connectome as an ANN in a robot produces a machine that acts and can be trained like a C.Elegans - we don't how or why, though).
We humans expect determinism from our machines, because for so long that is how machines have operated; the same input produces the same output. Our machines now, though, can produce different outputs for the (seemingly) same input, and that worries us (perhaps rightfully so, especially in the case of a self-driving anything carrying passengers!). Interestingly, we give a "pass" to human beings doing these same tasks; that is, we don't expect perfect results from these non-deterministic neural networks we hire or employ. We don't worry (much - though we try with imperfect models) about how one person decides to do something one way or another, even if they can't explain themselves.
But we create machines that do that same - and it worries us. I sometimes wonder if that says more about us than about the machines...
> "traditional" machine learning methods ... may be good enough for business tasks, they are not the cutting edge of AI research
That's true - but so? Under discussion is whether every developer should drop what they are doing and learn ML/DL, not what the cutting edge of AI research is?
In that context a lot of the low-hanging fruit for the next while is going to involve actually deploying simpler ML, with all the non-ML work that entails - and maybe using off-the-shelf DL to extract information from unstructured data.
> older ML PhDs are angry that deep learning is so easy
That's very ungenerous. People tend to do PhDs in things like ML (often making sacrifices to do so) because they want to advance the field - not to make it harder! People are excited. They are wary about hype causing another AI winter, or a lot of important projects failing for silly reasons.
This is a poor example. There are Go programs that work very well and use exclusively MCTS without any neural networks.
And yes, MCTS worked well for Go, but notice how the combination boosted it to new levels? And deep nets without MCTS also worked very well on Go, outperforming MCTS only approaches (by FAIR and was the top before being dethroned by AlphaGo).
> This is a fallacy, AlphaGo didn't increase very much with additional hardware at all
It did. Please take a look at the page 36 of the original paper. (https://gogameguru.com/i/2016/03/deepmind-mastering-go.pdf) The performance did not grow linearly, but the increases were vert significant.
>it's elo would still have been higher than lee se-dol's elo
I seriously doubt it would win that match. One-GPU version lost to 8-GPU version around 95% of all games.
The paper is somewhat misleading in estimating relative performance of AlphaGo compared to older programs, because (as far as I know) while all of them were run on the same hardware, only AlphaGo used GPU acceleration.
On the other hand it is true that DL is not required for many (most?) companies as they don't work with the kind of problems you list (vision, robotics)
""" The Upfront Summit is LA's premier technology event, with more than 750 of the country's top investors, startups, and corporate executives """
If he was talking to executives, it may be sound advice. It's extremely likely the hot business opportunities in the short-term will be applying ML techniques to outstrip competitors trying to solve problems with traditional hand-coded solutions. In business-speak, "Learn ML" translates to "Familiarize yourself with the space and hire the people you need who know the topic," because that's how a company "learns" something.
As long as we can reasonably understand the I/O (and that use of the software can pass a security review), I don't see a problem with this.
- I know enough machine learning to be dangerous, but I'm hardly ever asked to use it. I designed a Bayesian classifier for my own startup around 6 years ago, analyzing political donor networks. I've completed the Stanford ML course. Back in college I did a math minor, so I'm comfortable with linear algebra, calculus, etc. I'm pretty comfortable with statistics of both kinds. But my bread-and-butter is freelance web development . . . and I'm not really even sure how to find work doing more MLy things.
- I've read over and over that the most time-consuming part of ML work is data collection & cleanup, and that matches my own experience. It is the same thing that killed so many data warehouse projects in the 90s. You don't need a Ph.D. to do it, but it is a tough and costly prerequisite. So it seems like you'll need non-ML programmers even for specifically ML projects.
- In a similar vein, Google has written about the challenges of "operationalizing" machine learning projects.[1] Having a little experience collaborating with a team doing an ML project, where they did the ML engine and I did the user-facing application, I can say that many ML experts are not experts in building reliable, production-ready software.
- Will there ever be a Wordpress of machine learning? If there is, the author will be rich, but you won't need a Ph.D. to operate it. But because ML requires hooks into your existing systems, I don't know if this will ever happen. What will happen I think is plugins to existing e-commerce systems for product recommendation or other off-the-shelf ML-powered features. These already exist, but I assume they will become more prevalent and powerful over time. In any case, the mainstreaming of ML for business will be inversely correlated with the expense to implement it, which suggests it will be easier and easier for non-expert developers to use (and misuse).
EDIT: Added the (now-)third bullet point I forgot before.
I think this is key - prospective clients won't ask for it because they don't understand where it could be used, and they won't understand the heavy ML methods. An approach there would be to pitch things like improving search results using a bayesian classifier applied to analytics data as a cheap upgrade when quoting other work. Until people are used to even the basic statistical approaches they won't be ready to invest in something more drastic.
So there's some off the shelf ml, which is pretty great, but the problem of automatic feature selection - especially if new features need to be plumbed into a system - is still insufficiently solved. We've got systems for paring down the set of available features, but our systems have no idea how to ask for new features. That's essentially domain research, after all, and probably a 'strong ai-complete' problem.
Yeah, this is a very good point. I trained a simple RNN to play games a while back, and the "RNN" part was something anyone with a decent math background could self-teach pretty quickly. There are lots of tricks and speedups and complications I could have added, but the baseline product was pretty easy.
The "trained" part, though. That took huge effort, and research, and tinkering. Building up a proper dataset with usable input features was 90%+ of my effort. Most of it was too specific to standardize in any useful way, so working with an off-the-shelf system for the learner wouldn't have saved any decent amount of effort.
It certainly isn't at the same level, but Keras (imho) comes real close to being such a thing. Perhaps it, coupled with something like an expanded version of the TensorFlow playground, might make a power but easy-to-approach tool.
That said, even with Keras you still need an understanding of what the parts are for and how to "hook them up".
E.g. when I was reading up on genetic algorithms etc. 20 years ago we also expected the "revolution" to be right around the corner, and that things like genetic programming would change the world in a few years time. And while various of those methods found use some places, most places that could have used at least some of the simpler ones, still don't.
In other words, I think talking about a 3 year timeline is crazy. It's getting more attention, sure, but there is so much low-hanging fruit that most developers could be busy for the next 20 years putting in place the most trivial algoriths all over the place and we still wouldn't have picked off even the low hanging fruit where the computational resources and algorithms and data to make a big impact were well within reach 20 years ago.
This certainly means there is plenty of room for a lot developers to do very cool stuff and build careers on machine learning today, but it also mean most developers will not have to learn the state of the art - or anything near it - for a very long time.
As a concrete example I give to people, consider all of the search boxes out there on various sites - product searches, location searches, site searches - that are straight keyword based searches that don't take into account any clickstream data to improve ranking. The proportion of search boxes I see that take advantage of the available data is vanishingly small, even though very basic analysis can improve the perceived relevance of the results massively.
We certainly will see more companies invest in proper machine learning as the payoff gets higher and difficulty in taking advantage of it drops. But we will also see a huge proportion of sites that could use it continue to ignore it for years to come.
There are big business opportunities in finding ways of making a dent in that portion of the market, though, and so learning this stuff can certainly be well worth it on a personal level, but I don't believe in his timeline in terms of the overall market.
Having looked at search terms users enters on a lot of client sites, I don't believe that for a second. Users are exceptionally bad at composing search terms.
I agree with this wholeheartedly. I think there are pockets of HN that are out of touch with the reality in SMBs across the country (and planet). Often these businesses are struggling to track data, let alone utilize it for decision making or analyze it any sophisticated manner. "Out-of-the-box" ML will be another tool in the toolbelt in these instances.
So, sure, if you are trying to build Massive ML Tech Co., you won't be able to hire a few devs who watched a ML series on Coursera to achieve that goal. But there's ample opportunity to apply that knowledge elsewhere.
"""He thinks even programming is vulnerable to being automated and reducing the number of available programming jobs."""
I believed something similar could happen within 1-2 years of learning/writing AI programs (more than 12 years back). I believed it so much that it consumed most of my weekends as I took on the Genetic Programming approach. Yes! computers can write programs - BUT trying reading those. Eventually after spending hours or days, you will be able to read those programs and you might find a simple "hello-world" program represented by a complex mathematical equation. Good luck trying to get such program fixed by humans. Imagine an experience decoding deep-learning-neural-nets. However, that is black-box from a programmer perspective.
From a business/management personnel perspective - the code is a black box anyways. When they get NNs that can generate required software, they will replace the people-manager with a NN-manager (who is a programmer btw!)
Using Artificial Intelligence to Write Self-Modifying/Improving Programs http://www.primaryobjects.com/2013/01/27/using-artificial-in...
The outputs are like this for some not-so-easy targets:
Op nodes ['ADD', 'ADD', 'MUL'] EXPR: ['ADD[ni_99](ADD[ni_49](I__7[ni_43](), ADD[ni_19](I__8[ni_66](), ADD[ni_79](GET_CONST_3[ni_25](), I__9[ni_71]()))), I__1[ni_61]())', 'ADD[ni_13](I__6[ni_17](), ADD[ni_49](I__7[ni_43](), ADD[ni_19](I__8[ni_66](), ADD[ni_79](GET_CONST_3[ni_25](), I__9[ni_71]()))))', 'MUL[ni_68](GET_CONST_8[ni_73](), FLOAT[ni_42](I__1[ni_91]()))']
With genetic programming (using an AST), it can solve complex equations:
However, this simple equation i.e. correct answer "(a + b + c - d) / e" could be evolved and will result into either of these (depends on my luck maybe)
Case1: ((int)((b+((c-d)+a))/e)&(int)((b+((c-d)+a))/e))
Case2:
((((int)a&(int)(((((((mod(e,a)c)/e)(((((int)(d-a)&(int)b)+e)/a)/e))/((((int)e&(int)c)+e)+(c+(e/a))))c)/e)e))/((((int)e&(int)c)+e)+((b/e)+(e/e))))+(((((((((mod(e,a)c)/e)(((((int)(d-e)&(int)b)+e)/a)/e))/((((int)e&(int)c)+e)+((b/e)+(e/e))))c)/e) <..........10383 characters here.........> ))))))/e))/((((int)e&(int)c)+e)+(((d-e)/e)+((d/b)/e))))+(((mod(e,a)/e)+(b/e))+(((c/b)+((d+b)/e))/e)))))))))
The GP output (case 1 and 2) above was generated with a tweaked version of https://github.com/rogeralsing/go-genetic-math
I wouldn't bet against it. It's very risky to extrapolate progress in AI.
A DARPA project named CHI [1] succeeded in automatically producing programs from a high-level knowledge description language. And when they fed it its own description, CHI rewrote itself, with a better implementation. Do you fear for your programming job yet? Well, that was 36 years ago.
[1] www.dtic.mil/get-tr-doc/pdf?AD=ADA105661
Maybe if it was coming from Bill Gates, Mark Zuckerberg, or another tech titan with some actual coding experience and a deeper level of learning about what ML even is. Cuban's a business man, and most CEO's I know don't have a clue about the stacks that run their own company, let alone what's popular.
That said, I do think ML will be important, but I develop ecommerce apps and things of the such in Laravel, unless I move into AI and Neural nets I don't see needing to know a lot about ML (though I wouldn't mind moving in that direction as that space picks up) -- but there's still plenty of opportunities without it.
I suppose there will be (and to some degree already are) machine learning magic wands, but they are going to be the kinds of things that suck capital out of the companies that utilize them (through loss of proprietary data and competition blocking moats.)
What causes AI winters is when an advance such as deep learning can be applied to new problems and leads to increased interest. And while this new thing is really good as a subset of problems and impresses the public, of course it can't displace humans at everything and naturally has it's limitations.
So funding pours in, everyone gets hyped, and then those natural limits are (re)discovered and everyone gets all anti-AI research. Of course many people knew the limitations all along, but the dream is gone and so is a lot of funding until the next thing comes along.
This is probably natural to a lot of fields but AI just seems more prone to these boom and bust cycles because it's really exciting stuff.
This is more VC/founders who are hyping up AI and need more ML folks so they can drive down costs.
The modern tools for ML/deep learning are accessible to all open source and well documented. And as I note in my top-level comment, old-fashioned statistical methods like linear regression are more than sufficient for real-world business problems, and definitely do not require a PhD to grok.
Stop spouting this bullshit. You don't need a PhD, and you don't need to advance the field to be doing 'serious ML'. All you need to be able to do is know how and when to apply it to solve crucial business problems.
It isn't required for any serious research effort, except by the accident of inertia. And it certainly isn't a necessary indicator of determination.
That said, there is definitely a place for non-PhD level ML practitioners. I don't think the industry has stabilized in this regard, but I can definitely see a "machine learning developer" type position becoming quite common. This is not the same as someone doing original research, but would definitely meet the needs of a great many business use cases.
This counts as name-calling in the sense that the HN guidelines ask you not to do it: https://news.ycombinator.com/newsguidelines.html. Your comment would be much better without that bit.
We detached this subthread from https://news.ycombinator.com/item?id=13599533 and marked it off-topic.
Yet, I can still apply machine learning to solve Ax=b problems. More importantly, I can use business analysis and write code to transform business problems into an Ax=b problem, and then optimize it.
You don't need a PhD to grok optimizing a vector to transform a matrix of inputs into a vector of observed outputs, then apply that trained vector going forward. Neural nets are slightly more complicated than a straight linear regression, but only slightly. I'd call decision tree methods like GBM even more complex, but still eminently grokkable for a decent programmer.
Obviously you will want a theoretical expert on your team. But if your startup is counting on having a room full of them, good luck.
Good thing the vast majority of businesses won't need "serious" ML, but instead will require only simple implementations to help solve business problems.
I mean, we don't need a phd in image compression to create a service that streams videos. We just use libraries. Same for everything in computer science, it always end up packaged in some kind of reusable code or service, and only some specialists remain working in the field to work on marginal improvements.
Why would ML be any different ?
If you don't just move on to another problem you will need people who know how to push these systems further.
I am rather newbish in the whole space, but most everything I've seen is quite often just knowing enough to turn the right dials the right way. And I do wonder how long will it be before some of these dials can start turning themselves with more iterations with a simple, "yes that's what I want, or no that's what I don't want"
Sure, it might take some more cycles for it to find optimal in this example.
Understanding what models are doing and adding new features/data transformations is standard work for ML practitioners now. If you believe the deep learning hype, architecture engineering is the new feature engineering, and to the extent that that is true, industry will essentially resemble academia to some extent. Maybe not making anything groundbreaking, but mixing ideas together.
Luddites of 18th century thought they would never be replaced and continued on their trajectory.
A lot of white-collar jobs may be automated (or otherwise changed beyond recognition due to technology) after about thirty years maybe, but not three.
Reading through the comments, I see that Cuban's statement upset and even angered several HN commenters. That is a strong emotional reaction.
I am not saying it is the 5 stages of grief, but the first 3 fits: denial, anger, bargaining, depression and acceptance.
Also from Howard Aiken: Don't worry about people stealing your ideas. If your ideas are any good, you'll have to ram them down people's throats.
With this prosthetic knowledge we will have to learn much more what to ask and know how much the machine knows.
One has to quickly grasp the abstract that is one level too high or detailed for the machine to find and than find the seperate answers of the level below to recombine them. You can't yet ask where to open a restaurant, but you can google for demographics and write a program to map ratios between income, foot-traffic density and restaurant density.
Once we can ask what and where is the most profitable business in to start in town, we probably still don't get a step by step guide how to do it, interior design included. Where the rubber meets the road there is still a lot of oppertunities to decide on and complex data we can grasp more easy than the machine.
Currently, it means: "Jumps on bandwagons, caution."
In three years, it'll mean: "Brainless buzzwords, avoid."
He's talking out of his ass here. This won't be happening anytime soon (if ever)
Perhaps a good time to be an ML grad coming fresh out of college.
By all means, start learning actual machine learning methods too, rather than just statistical methods, but as I pointed out elsewhere on this thread: there is low hanging fruit everywhere.
Not nearly all of those will need "proper" machine learning, and even fewer will be willing to pay what it will cost to hire people with in-depth machine learning experience or pay the development costs or computational costs anytime soon. But a lot of them will buy into the buzzwords and look for a cheaper halfway-house or be open to pitches.
Your assertion, though:
> This is so dumb. You arent going to understand or get a job in machine learning unless you have at least a masters in the subject. Its extremely difficult and complex.
...couldn't be further from the truth. You can understand this stuff without a masters in the subject. It really isn't too difficult or complex. Sure, I will admit that understanding how to take a derivative might be useful, but despite not having that knowledge (but I'm working on obtaining it), I have still been able to implement successfully working ML solutions - at least in a classroom-type environment.
My last success was getting a virtual car to drive around a virtual track, staying on the track and negotiating the curves, using an implementation of the NVIDIA End-to-End CNN architecture and some data I generated (plus augmentation and some other fun stuff). I used Keras and Python 3, running on my workstation at home, with a 750 ti SC as my "GPU" (I really need to upgrade this). My model converged very well after 10 epochs, but after 20 the loss was pushed pretty low to sub 1%. As far as I could tell, there wasn't evidence of overfitting (I need to do more investigation on this, though).
This was all done as part of Udacity's Self-Driving Car Engineer Nanodegree, which I am taking part in. Prior to this, I also completed Udacity's CS373 course in 2012, and Andrew Ng's ML Class in 2011. My motivation for all of this has mainly been my interest in autonomous unmanned ground vehicle robotics technology. I have an ongoing side-project in developing such a platform (seemingly back-burner'd a lot, though - life getting in the way, I guess). Even so, if a job offer comes about because of it, I'm not going to complain.
As it is, I believe the knowledge has helped me land positions, since it shows my dedication to improving my skills in problem domains outside of the everyday software development tasks. When potential employers have asked about it, I can show them some code I've worked on, while mentioning how some of the more simpler ML methods could help in a business problem domain. It sets me apart somewhat from other candidates, I believe.
Especially those who think the topic isn't worth their time to learn, because it may be "difficult and complex".
In the long term, possibly not - there are already efforts and approaches being done in ML/deep learning to get models to label unlabeled data (google "deep learning unlabeled data" for tons of research info).