n3ur0n

I would say UDL should be very accessible to any undergrad from a strong program.

I would not call the notation ‘dense’ rather it’s ‘abused’ notation. Once you have seen the abused notation enough times, it makes just makes sense. Aka “mathematical maturity” in the ML space.

My views on this have changed as a first year PhD in ML I got annoyed by the shorthand. Now as someone with a PhD, I get it — It’s just too cumbersome to write out what exactly you mean and you write like you’re writing for peers +\- a level.

Given your background, I think it would be worthwhile for you to pick up ESL [0] and read some relevant sections (supervised/sparse/linear methods). It's a great book and a good starting point for thinking about ML methods for high dimensional data.

Also, might be useful to took at webpages of some researchers in this space and courses they teach [1,2].

  [0] https://web.stanford.edu/~hastie/ElemStatLearn/  
  [1] https://scholars.duke.edu/person/dunson  
  [2] https://www.cs.princeton.edu/~bee/

Depends on the scope of the project. Would the goal be to come up with a better algorithm for cell classification based on histological images? Or to apply an existing algorithm to a new dataset?

The former would be quite difficult without much background in ML/Computer Vision (you would have to spend some time self-teaching basics of ML/Deep Learning and the pre-reqs for those — Basic Linear Algebra and Probability).

The latter is doable. I would recommend a very hands on approach. Pick some computer vision object classification tutorials and code them up (using a high level library). Make a mind map of the concepts and look them up as and when you’re unclear about a concept. Then move on to replicating some well cited, peer reviewed papers. Often papers will have their code on GitHub. Try and relocate their results on their dataset. After this you would have the basic working knowledge to modify the algorithm slightly for your specific use case.

I think it's partly the incentive structure that is to be blamed. Historically, quantitative PhDs in healthcare(medical physicists, statisticians, comp. genetics) have been underpaid (in my opinion). Now with FAANG and Quant Funds willing to pay $400K+ comp packages to these PhDs, there are far more exit opportunities for these PhDs.

On a positive note, I'm so glad that clinicians are taking interest in ML! As a practicing ophthalmologist, the fact that you were able to self teach is really impressive! I do know that a lot companies are looking for people like you, who have clinical experience. If you are interested you should explore roles/potential collaborations with some of these health research teams in tech.

I do respect your experience and take on the matter, however, let's replace this statement:

"I'm an eye surgeon and self-taught machine learning practitioner, I started to learn Python in 2016 when the deep learning hype was at his highest."

with:

I'm a [machine learning researcher] and self-taught [ophthalmologist], I started to learn [ophthalmology] in 2016 when the [clinical medicine] hype was at his highest.

In this hypothetical situation, I bet you would instantly discount what I would have to say about ophthalmology because I clearly would not have the depth or experience to have an informed opinion on ophthalmology.

Over the past few years with the ML hype, I have noticed quite a few clinicians who have self taught some deep learning methods claim expertise in the subject area (not targeting you, a general observation). I feel like many clinicians do not understand the breadth of machine learning approaches. There is just so much to know! from robust statistics, non-parametric methods, to kernel methods. Deep learning and deep generative models are by no means the only tools at our disposal.

I absolutely agree with you though. Applied machine learning practitioners have been over selling their accomplishments -- which I believe is detrimental to progress in the field.

I would highly encourage you to collaborate with ML researchers who have spent a decade or more working on hard problems. From the other side, I can tell you I gained a lot discussing ideas with domain experts (neurologists, radiologists, functional neurosurgeons). They have insights that I could never have picked up by self teaching.

As a rising 5th year PhD in ML -- I could not agree with this advice more! I have very hands-off advisors. I spent the first 3 years "wandering the woods to find something". Last year, I really had to sit down and think about how I can finish up my PhD on time. I pretty much did what you outline here. A lot of tools I used were organization tools I learned from my business school/product manager friends.

I honestly think the PhD system needs to be overhauled. even at a "top-tier" program like mine, it is amazing to me that at no point do we receive any training regarding practical components of research. 100 years ago, the way you became a physician was to follow around a physician and one day you were ready to be a physician yourself. In the year 2020, this is how PhDs are trained. I do realize that a PhD is not a "professional" degree like MD or JD, however, given that most PhD grads will 1) go to industry 2) go into academia, we need to teach students about project management, planning etc.

> just means trying out

this statement trivializes a very hard problem.

> literally just means trying out, or simulating

Simulating is an incredibly hard problem and MC methods and theory is an incredibly rich area of study. Some tools I use for my work in probabilistic machine learning models are MCMC techniques like HMC (Hamiltonian Monte Carlo), variance reduction techniques (Rao Blackwellization). If you would like to learn more, here is a great course: https://statweb.stanford.edu/~owen/mc/ -- you can take a look at the syllabus. Also, Casella Berger is a standard MC method book.

There is really no difference. You can frame the Kalman filter as a Bayesian posterior inference problem.

For example, for a stationary linear Gaussian model, you have a transition model of the form: z_t = Az_{t-1} + Bu_t + e where e ~ Gaussian(0,Q) and an observation model of the form: x_t = Cz_{t} + Du_t + d, where, d ~ Gaussian (0,R)

Since, z_t and x_t are both multivariate gaussians in this model, you can compute the posterior distribution on z_t's, which will also be a Gaussian. That is basically the Kalman filter.

As the writeup mentions, you might choose a non-Gaussian noise model, in which case the posterior distribution is not a Gaussian and then you employ something like a unscented Kalman filter or extended Kalman filter.