Hacker News Reader

The SciML tutorials are a good start: https://github.com/SciML/SciMLTutorials.jl

And also the 18.337 lecture notes (probably also has videos available): https://github.com/mitmath/18337

> but still— I'm a long-term Julia fan, professional data scientist, mathy PhD, would hope that's at least table stakes.

I don't understand this part. Are you saying you're a "professional data scientist" with a "mathy PhD" who has a "light-to-moderate amount" of machine learning knowledge? How did you get the job?

I would expect anyone with a mathy PhD to understand ODEs and PDEs, and neural ODEs are commonly understood (by those who read the papers, where I think it is an appropriate assumption that people who want to understand this stuff would do) to be effectively infinite-depth neural networks where every layer represents the same function.

Have you been in interview loops or worked with bread and butter data scientists performing common tasks? I am curious what your view of what most data scientists do day in and day out?

What about tasks being common makes a light-to-moderate understanding of machine learning sufficient?

Processes initiated by data scientists during the execution of their role will tend to fail silently. What is meant here is, throwing an inappropriate model at otherwise good data produces unreliable (catastrophic in certain situations) results, but produces results nonetheless. Without the proper discernment of the reliability of the results, we have an unequivocal failure to execute the role. This is the oft-unmentioned companion to, but decidedly more insidious than, the "garbage in, garbage out" (i.e., right model, wrong data) aphorism.

It is up to the person performing this operation to deduce whether or not the conclusions are trustworthy. I don't see how someone can be confident of this without either relying on a pre-defined workflow verified by someone else qualified to assess the consequences, or to have those qualifications themselves.

What follows is a contrived example, but illustrative of the problem:

Consider e.g. user privacy: it is by now well-known that e.g. embedding vectors (or even merely the relationships between them) can leak a lot of information about the person or object it represents. It is not enough to understand how the forward pass of such a model commences, but also what is stored in those representations, which, having gone through a master's with quite a few people who now call themselves data scientists, I am not confident is commonly understood.

This is helpful, and I agree with you. There are some "data scientists" who are able to use PyTorch, TensorFlow, etc. and modify code in a Jupyter Notebook without knowing the larger ramifications of the work they are doing.

This is a pretty shitty, non-constructive response. I think neural differential equations are not easy to wrap one's head around even if you have a solid understand of deep learning and differential equations.

Sure, if they spent a lot of time wading through the literature they'd probably understand fine, but the point they were making was that the post was quite unapproachable without having delved into the specific literature on neural differential equations.

I think this is a reasonably valid complaint, and does not warrant you implying that they don't deserve their job.

Valid complaint how? If I have never studied carcinogenesis why should I believe I should understand the description of a new treatment for bone marrow cancer?

The way any article is written reflects the audience it is suited for. If this article was intended for people unfamiliar with neural ODEs they would have put more effort into writing it in a suitable way.

There is a big difference between a blog post and a scientific article. A blog post should be expected to popularize a topic and reach a wider audience.

You also seem to be quite unfamiliar with breadth of the topic of mathematics. It is quite possible to take a mathematics phd without touching differential equations, except as an undergrad.

BTW, implying that people don't deserve their job is just shitty behaviour, and way out of line.

I think it would have helped if the people writing that paper had not confused the issue by introducing a new name for something that is a well known thing in optimal control and had been invented even before neural networks, namely adjoint sensitivity analysis. There even appear multilayer networks of switching components in Pontryagin's book on the subject.

> Are you saying you're a "professional data scientist" with a "mathy PhD" who has a "light-to-moderate amount" of machine learning knowledge? How did you get the job?

Indeed! My job involves approximately zero machine learning, at least for a narrow or stereotypical definition of machine learning. I work on optimization, domain-specific models inspired by queueing theory, various phsyically-motivated structural models, writing production code for data-heavy products, testing hypotheses in data, brainstorming how existing data can be used to solve new customer problems, writing documentation, communicating with customers, etc.

If you think this isn't data science but know of better nomenclature: please share! I've struggled to write a great job posting for this kind of work, and surely improved nomenclature would help :)

in what is your phd?

Operations research / transportation

cool!

I would say you do machine learning if you're doing optimization on bespoke models. These worlds blend together once you get into the weeds. It doesn't have to be gradient descent (via AD or otherwise) to be machine learning.

By "light to moderate amount" do you mean that you haven't catalogued the menagerie of statistical models that are commonly taught in machine learning / deep learning courses? Because that to me is secondary to having the underlying principles (optimization theory, measure theory, etc.) down pat. Recognizing the fundamentals in the theory which describes how machine learning proceeds is invaluable for comprehension.

If you wanted a flashier name for your role I might suggest "Solutions Architect" or even "Machine Learning Engineer" based on what roles you want to aim for. Because honestly you're doing a lot of what is already entailed by those titles. "Data Scientist" also fits for sure, being such a broad title nowadays.

I interpreted what you said more glibly(?) than it seems you intended, and apparently I expressed my surprise snarkier than I intended.

What's funny is how most of them are revolving around a similar culture of concepts (ODE, automated differentiation, analysis) which is very unlike mainstream computing.

ps: I need to read those adjoints articles.

I consider myself to be reasonably good at ML and work for one of the cool labs and this stuff is impenetrable to me. Stuff like this are deceptively written and a cynic might say that they are primarily to show off rather than help.