Hacker News Reader

Comment by wfalcon | Hacker News Reader

yes! our goal is to completely remove any engineering from the AI research -> production lifecycle.

Not just a marginal improvement on that experience but a 10x completely different approach.

Dead end. Train researchers to be engineers. Not the other way around.

I wish you luck though. We’ll see in ten years whether programmers are as effective as researchers, or whether researchers are as effective as programmers. In 60 some years of computing, no one has achieved the latter, despite many attempts.

Also, I was surprised that this webpage is basically a waitlist and nothing else. No discussion of technique, no docs, no substance. Just a “you like pytorch? Pytorch rules!” type hype.

I do like pytorch, but I also like knowing one or two substantive points about what the proposal here is. If you want to train a model from your laptop, it’s a matter of applying to TFRC and kicking off a TPU.

The whole ecosystem is in need of massive overhaul. I like the ambition. But I dislike trying to pretend we aren’t programmers. ML is programming, and pretending otherwise will always cause massive, avoidable delays.

I would pick engineers over researchers any day as you can teach an engineer how to do research but the opposite is rarely a case.

It is much harder to do research then it is to do engineering. I do both at work where we have about 15 ML engineers and only 2-3 of them are capable of doing something novel. Maybe you mean something different by “research” but it is very hard to teach someone how to come up with an idea or a novel method publishable in a top conference. Teaching engineering on the other hand usually goes like this: “Here’s how this works. Here are your options, tradeoffs, pitfalls.” After you’ve done it a few times you mastered it.

I forgot who said it: “In engineering, if you don’t know what you’re doing you shouldn’t be doing it. In science, if you know what you’re doing you shouldn’t be doing it.“

Very little of the work done in machine learning is truly novel versus simple extensions of what has been done before. There's probably the same amount of novel work in machine learning and engineering except the ML crowd publishes it while the engineering version stays in closed source code bases. There's also a distinction between doing novel machine learning work and publishing successfully on novel machine learning work.

I agree with you. However very few "ML engineers" do any novel work, let alone "truly novel" work. Typically they just take the existing code and apply it to their specific problems with some tweaking. Sometimes they would try to reproduce papers when no code is available. Very few engineers are capable of the NeurIPS quality research - it does not matter if it get published or stays in a private repo. That's much harder to do and to teach than what 90% of typical ML engineers do. That was my point.

There's also a ton of bad research being published in second rate conferences, but I don't consider that "research". You won't get "simple extensions of what has been done before" published in a top level conference, the acceptance rates have been extremely low recently.

>Research is "creative and systematic work undertaken to increase the stock of knowledge"

I meant this. Academia-related BS is just a one way to do that.

I would not have touched TensorFlow (or AI in general) at all if it weren't for Keras, and I wouldn't be happy with PyTorch if it weren't for PyTorch Lightning.

Easy onboarding for ML tooling is very valuable for the industry as a whole.

not sure where is "you like pytorch" coming from. Grid.ai is not limited to pytorch :)

As a scientist: The amount I struggled with getting distributed training on a HPC cluster to work vs. how easy it was with Lightning was eye opening. Almost no code change and finally I can run across 20 nodes with 4 V-100 each :). Plus the automatic SLURM checkpoints and restarts <3.

Lightning is built for researchers by researchers... we've already taken a much different approach.

100% agree with you that going the other way is likely not the best approach.

Lightning + Grid elevates and turns non experts closer to researchers... ie: focus on building the products and doing science and not the engineering.

That's what lightning excels at today. That's the experience Grid will 10x.

Reply on news.ycombinator.com