No Phd, No Problem: New Schemes Teach the Masses to Build AI

To transition from practitioner to engineer, one will need to know low level details including directly relevant math and popular prior art. One of the distinguishing difference between a practitioner and an engineer is that later needs to be able do systematic debugging to identify and fix the problem. I often ask folks during interviews how they would go about when their ML model to do X doesn't work as expected. This usually reveals the difference between practitioner and engineer with high accuracy.

To transition from engineer to scientist, one needs to know much broader body of prior art and beyond directly related math. A typical interview that would differentiate between engineer and researcher would be to discuss some known unsolved problem and see if the person can elaborate details of prior art, opinions of top researchers, wins achieved so far and directions for future efforts.

This is again not to say that practitioners can't do debugging OR that engineers can't make ground breaking innovations. Faraday, Edison, Wright Brothers - all either started out as practitioners or engineers and successfully attacked some of the hardest unsolved problems of their time, never obtaining formal academic credentials. On the other end far many more academics have broken new grounds then non-academics. At the end of the day its all probabilities. Claiming that PhD is no longer required to do "AI" doesn't reveal all these subtlety and marginalizes tools and foundation that PhD equips oneself on this journey.

I think even that level of knowledge is not always necessary. Often, just having the intuitive understanding can be enough to get great results e.g: understand that this is a black box that takes features in, predicts things, and you have to help it by giving good features; or understand that word2vec builds vector representations based on words that co-occur in the same context.

--

I had someone contact me for a "consulting" position, to replace their rule-based insurance claim denial system, with "ML". Contract-work, nobody in-house to review its performance, no insight to the problems of the domain and they wanted to throw ML at it to "solve their problem".

When I refused, and pointed out that it would be highly irresponsible, to "contract-out" this type of work, particularly given its life-or-death implications; the CEO got angry. Told me; its "post-claims processing" and it isn't life or death. To which I told him bullshit; your denial of claims is going to directly influence how doctors practice their treatment. There is a direct feedback cycle. The fact that you don't see it, makes it even more dangerous.

They simply, didn't understand, how HIGHLY inappropriate it was to just throw random ML at a problem, particularly as a one-off consulting project (and no in-house expertise).

^ thats real-world.

Personally I don't think you should be allowed anywhere near ML, UNLESS you have that PHD in computer science. I don't even think you should be allowed to HIRE people for ML until you fully understand the hazards with letting a computer control critical decisions.

So yeah, with respect to your distinction of "ML research" and "applied ML".

"NO."

It's not models that would necessarily be bad when made by something like "script-machine-learners". What has a lot of potential for a lot of badness is taking these models as given by "magic". Machine learning is essentially a giant exercise in piecing-together correlations by sort-of clever, sort-of brute-force methods. What's hard is knowing the proper application areas and the improper application areas.

My basic rule with stories is that they must come from me casting my attention out into the world, not via a bidding war for it in my inbox. I run a whitelist and never read inbound PR. I only email comms people when I a) can’t find a person’s details b) the person asks me to

I'm a few weeks in to Fast.ai and the professor keeps it high level and has more focus on application than theory. You jump straight into working on image classification which is broken down into clear concise steps that gives you some confidence moving forward. If you do take the course make sure to plug in your own data and get a feel for how everything works before jumping to the next video.

I would say its good to have some knowledge of the basics, so watch a few weeks worth of Andrews course while it's still free(7 day trial). Skip over the programming assignments because they are too low level and verbose. Then when you think you have a good sense how things work jump over to Fast.ai and start coding.

EdX has a good one, I wrote about my experience of it here https://gaius.tech/2018/08/08/microsoft-professional-program...

The tooling and libraries are very good now; assuming you already had the data in a good shape, you could do something commercially useful with Keras in a day or two from first picking it up. There is still quite a lot of low hanging fruit around but these skills are well on their way to becoming commoditized so I don’t know how much longer that will last.

Personally, what I see is a lot of AI becoming commoditized. There was a time when you had to have a fairly strong understanding of compilers to program anything complicated. These days you can use a high-level programming language and never get down to the level of the compiler if you don't want to.

If someone wants to make lots of money or change the world with AI, my advice would not necessarily be to start with a PhD. It would be to focus on understanding data, getting very good with a library, and building apps that are useful to people.

If you do that, then getting additional knowledge about the mathematics furthers your career, but your career doesn't block on acquiring that knowledge.

If you do this, you probably won't have a great of chance of working on a team at Google or Facebook improving the implementation of the AI infrastructure. Just as you probably wouldn't get a job at one of these companies optimizing the compiler if you didn't have an academic background or years of experience in compilers. But you could still work on other teams in those companies and make as much or more than those people and have a more direct impact than just making it marginally faster.

I have a PhD in pure mathematics, and I don't do ML. Getting a PhD is a particular journey, and it's not for everyone. Also, the idea of getting a PhD as a credential for industry seems a little odd to me, but that may be my personal bias.

Having taken Fast.ai's Deep Learning course, I can confidently say that their course is enough to help a software engineer with no previous AI experience build extremely powerful real-world AI applications.

Jeremy Howard (the cofounder of Fast.ai, the former president of Kaggle, and the former #1 Kaggle competitor) only has a B.A. in Philosophy. One student who started the Fast.ai course as a violinist is now working as a researcher at Google Brain. It may have been true in the past that you need many years of work to become competent at building AI, but that's not true anymore.

Of course a lot of study and also applying what you learned are important. But having a PhD says very little about a person's ability to do data science or machine learning work. Especially since most phds have an extreme focus on their narrow field.

I haven’t gotten into it myself, but that is more of an interest issue.

In my opinion however it is a good rule of thumb to assume that someone with a PhD in a relevant field will become a good data scientist after an adjustment period.

Someone with amazing ML skills but no knowledge of the problem domain will fail. Someone without domain knowledge will be unable to assemble and curate the data they need and will be unable to make any sense of the output. That is the Achilles heel of ML rockstars who think it's just a matter of getting enough data into their training set and the NN will do the rest.

This is missing the point of a PhD, which is to develop a capacity for doing science (or more generally hard thinking in relevant domain). Most of my colleagues in neuroscience are physicists

Just like how people check out your GitHub repository to see if you are a capable coder, people check out your refereed publication record to determine if you can do research. It's possible to build up a record of publications without a PhD, but it is a hard skill to acquire on your own.

Having a PhD should mean that the person is capable of making consistent progress in ill-defined problems without much external support. Of course not all PhDs can do this, and you don't need a PhD to gain these skills, but they correlate strongly enough that it makes sense to just filter some jobs by whether or not someone has a PhD.

I think this is largely the reason, that a PhD is seen as a semi-reliable guide to general research ability, though of course it's neither necessary nor sufficient, but there is a rough correlation.

A person with PhD will typically like to invest huge chunk of time to digest much of the prior art. They want to systematically understand what worked and what didn't in the past. They then bring in their unique understanding of small set of techniques/cross-domain expertise to combine with prior art to attack the problem. It is important for them to craft well designed experiments and lean on tools like mathematics to guide their next steps. Their goal is to make sure anonymous reviewers of their work are satisfied with proof they present of their hypothesis.

A person without PhD typically doesn't want to take a lot of time going through years worth of prior art. Their approach usually boils down to developing intuition via initial random search and then iterative experiments to strengthen intuition. If this doesn't work out over a period of time, they will probably want to move on to something else instead of keep creating next little insignificant bump of progress. The frame of mind is get to near-instant gratification as opposed to possibly spending all of your life in to something that wouldn't make any impact in real world but might build useful steps for future generations.

It is hard to say one approach is better than other for all classes of problem. However using the first approach requires certain training, rigor, discipline and, most importantly, frame of mind - we call PhD. It is only recently and in few fields like AI that research careers have become desirable from financial perspective. However in most of the fields, things are very different and you would rarely see anyone demanding to be theoretical mathematician or quantum physicist, for example, without a relevant PhD while also receiving modest income with no promotions or stock options.

However, it has become clear that for many advanced problems the random search approach typically maxes out quickly because search space becomes prohibitively large and developing correct intuitions requires climbing knowledge pyramids of epic proportions. For example, invention of transistors, GPS or techniques like initialization or dropout in deep learning wouldn't have been possible without PhD style multi-generation approach and concrete mathematical understanding.

Having said that, I'm against hard liners who create rules that being researcher requires PhD and one without it cannot innovate or do creative work. Many complex and most valuable innovations have been done by non-PhDs such as Faraday, Edison and Wright Brothers - interestingly in all of these cases people with academic degrees gloriously failed working on same turf, for longer time and better funding (Davy, Langley etc).

I.e., they are actually doing the exact job that they are now being hired to do.

> Is it the ability to conduct research - methods, process, analysis, discipline? Or is it the fact that they have knowledge in that particular field?

When you apply to R&D labs, you are competing against people who spent at least five years:

* building research infrastructure (which includes, but is not limited to, knowledge about "methods, process, analysis, discipline"),

* a concrete track record of producing publishable research,

* a professional network and some amount of respect for you previous research output, and

* experience designing a multi-year research project (from writing/helping write a convincing proposal all the way through publishing, evangelizing, and managing relationships with people who want to build on your work).

Do you have 5+ first author publications in good conferences? Have you contributed to R&D software? Do you have multiple influential collaborators who have a high opinion of you and are likely to help popularize/contribute to/build upon your research goals? Have you written and followed through on grant proposals?

If "yes" to all those things, you should apply for these research positions! Even without a phd, you are a good candidate.

If all you have is a six month MOOC, or some R&D software development but no experience with the full research process, then sorry, but there are people with who have already demonstrated they can successfully design and execute on research agendas. You might still be a good candidate, but preferring strictly more experienced and better qualified candidates who have demonstrated, empirically, that they can perform the full job, is not an "arbitrary blockade".

> Every time I speak to someone with a PhD at work (I work in top 10 tech companies in US), in 100% of the cases in my experience do not engage in anything related to their thesis.

"people I have worked with" is a huge selection bias.

IME, in CS this is almost always either a) not true, or b) a conscious choice on the part of the phd holder (and is more of a pivot than a "totally unrelated thing"; you just might not see that there's a deep connection between the thing that the person was doing before and what they're doing now).