Comment by ChrisLomont | Hacker News Reader

Comment by ChrisLomont | Hacker News Reader

Parent Full thread

>At the end the engine that compiles the mathematical expression to hardware is what matters,

This is becoming increasingly outdated as significant parts in scientific machine learning require parts to be written that don't simply compile to GPUs. Think, for example, when you mix a physics model, say for RF or some such, and deep learning to model parts of the function. In python, you cannot write the RF model because python is vastly too slow, so you're forced to write the RF model in something fast, like C/C++, then integrate that to python, then integrate that to your favorite tensor network, needing more languages than you can do immediately with Julia.

Deep learning is moving rapidly out of simply being a tensor engine, and being a tool in much larger problems, where many high performance pieces need developed. Julia is light years ahead of Python for these domains, and I cannot see Python ever catching up because it suffers from performance and/or multiple language problems to solve these.

If you've never learned about scientific machine learning - go read some or watch some videos. It's fascinating and growing rapidly.

I’m seeing just the opposite as you: if it’s good enough to do protein folding for Google and video classification for Tesla, it’s good enough for me.

I’m sure that Julia is better for some specific tasks, for example phyisical simulations, that are important for some types of scientific tasks, but I don’t see any valid argument on why a simple tensor engine in Python is not smart enough to simulate the human brain, which is just a bunch of connected neurons.

This comment seems a bit odd given that the connections between physical neurons are qualitatively very dissimilar from the connections in artificial neural networks, so invoking the human brain seems like a red herring in discussing AI methods.

>if it’s good enough to do protein folding for Google and video classification for Tesla, it’s good enough for me.

That's funny, since protein folding uses the methods I described to achieve it's results. Multiple stages in their work [1] uses physical modeling, not NNs, to do protein folding, most likely because the problem becomes currently intractable to solve with simple Python + TF. NNs are a step to adjust the physical model, exactly like I described above.

Simply read their paper and note all the physics models incorporated at about every step of the process to enforce physical constraints - this means vastly less parameters, less training time, less training data, faster evolution of the process, etc.

Here's their repo [2]. They did that work in Python and TF, likely because it was started years ago. As Julia becomes a much faster develop tool for this type of work I expect this will change. They also used TF 1.14 - showing the age of their development. They did not release the feature generation code which is a significant component; the released code only works on the specific dataset they provide. This is likely because this component is not simply simple python they wrote, but an amalgam of things written to make the physics parts of the chain fast enough. But they don't clearly state either way.

Also, by your argument, since it's possible to solve any NN problem with a network only 3 layers deep, why not just claim that's all one needs? Because it's also not computationally feasible.

The point is that by adding outside knowledge, such as physics models, you can have vastly smaller networks, require less training data, train and infer faster, with the end result of being able to solve a much larger class of problems efficiently.

So yes, you can do it in python, or any language, if you want to waste orders of magnitude more effort and resources to do it, effectively limiting the things you can practically do.

This is why Python incurs a unnecessary cost for such development.

By willfully ignoring learning about these methods and being ignorant about even the results you cited you will miss out on extremely useful knowledge.

[1] https://www.nature.com/articles/s41586-019-1923-7.epdf?autho...

[2] https://github.com/deepmind/deepmind-research/tree/master/al...

Reply on news.ycombinator.com