Hacker News Reader

Comment by mg | Hacker News Reader

After a cursory glance, my feeling is that "The Mathematics of Neuronal Networks" would be a better title?

In recent years, the term "Artificial Intelligence" is often used instead of "Neuronal Networks".

I wouldn't be surprised if this will change again. If there is evidence that it will not and Neuronal Networks are for some reason the optimal medium for intelligence, I would love to read about it.

View on HN

I like to explain that AI > ML > NN.

Now, NNs are the ones getting results at computer vision and natural language, and more. I think most people would say that other ML approaches are computational statistics. The goalpost for AI keeps moving.

If you are truly interested in the math of AI I think PAC Bayes learning is more appropriate and your book is Understanding Machine Learning [1] (not an easy read). A more gentle intro would be Learning From Data [2]. If someone recommends a book/paper it would be awesome, I'm always on the look.

[1] https://www.cs.huji.ac.il/w~shais/UnderstandingMachineLearni... [2] https://work.caltech.edu/telecourse

Seconding this. "Learning From Data" is one of the best intros to machine learning theory I've seen through the years!

NN > ML. The proof is that nematode or fly, I dont remember, with its simple NN fully mapped, and still remaining a mystery how it works. ML, which is just a matrix multiplication at its core, is a laughably simplistic model of NN.

GP means artificial neural network, not an actual nervous system.

Machine learning contains ANNs as a sub-discipline. Other non-ANN topics in ML include ensembled trees, Gaussian processes, and sampling theory.

linear-algebraic matrix multiplication complicated by non-linear threshold discontinuities evaluators, no?

I think one can't compare deep learning to machine learning since they have different purposes. While there have been great strides in the interpretability of NNs, the analytic models of classical machine learning are favored from a computational and interpretability perspective. Conversely, if the relationships in our data are constantly evolving (e.g. the change in the stochastic process followed by a time-series from one interval to another), then NNs are more appropriate.

Although it tends towards Deep Learning as opposed to AI, I highly recommend Bishop's Pattern Recognition and Machine Learning. It not only provides a solid Bayesian perspective, but also comments on the subtleties of applying theory. For the latter, its discussion of overparameterization in (I think?) the first chapter comes to mind.

[1] - https://www.microsoft.com/en-us/research/people/cmbishop/prm...

Because AI basically "stole" the term neural networks from neuroscience, neuroscientists now often speak of "neuronal networks" while referring to biological ones, to avoid confusion. So I would recommend to not use "neuronal networks" in this context.

This is a very good point. To anyone interested in what Artificial Intelligence looked like before neural networks, the "Artificial Intelligence - A Modern Approach" by Stuart Russell and Peter Norvig is a good book on the subject.

Agreed. I was teaching from it (AI 101, search/agents, no ML) and it's possibly the best textbook I have encountered. It's structured very well.

As an aside, I will forever remember the rough geography of Romania. I still plan to do an A* route visit of the country some day :D

> Neuronal Networks are for some reason the optimal medium for intelligence

No, that's not the same 'intelligence' ("General Intelligence") as the "I" side in Artificial Intelligence.

The term 'intelligence' applied to Artificial Neural Networks makes sense, as such: to reach a procedural solution it takes an engineer; the engineer is said to have reached the solution because "intelligent"; ANNs are (semi-)automated builders of function approximators; ANNs are said "intelligent" because they "reach solutions" (like the engineer would have done - "approximator":"engineer"="Artificial":"Natural" Intelligence). And of course they are not "intelligent", while they are in some sense. It's just an expression, it's rhetoric (it requires considerate interpretation).

That some sort of "intelligence" is achievable in other ways, or that ANNs spawn as an idea from anatomical considerations of naturally intelligent entities, or that ANNs could help in modelling general intelligence (etc.), is tangential.

ANNs dont have the key component of intelligence - ability to learn. If our brains worked like ANNs, they'd be wired once at the brains factory, they'd produce the same output given the same input, and learning anything new would need a complete rewiring at the factory, possibly erasing other learnings (often unpredictably).

Even if ANNs could update themselves on the fly, in a reasonably incremental manner, they'd just imitate insect-level reactions. Intelligence would need those ANNs to have a virtual reality, run thought experiments there and learn from that.

Well this does not mean that some mechanisms similar to those we are studying cannot be part of the original or that they cannot be part of a General Intelligence system. Even if that were simply "adjusting a structure to function and information retention". The very "compositional" nature of the perceived that makes Convolutional ANNs work was noted after biomimicry.

This, of course, as an aside.

You're asking for online learning. This is certainly possible with modern techniques - define an unsupervised loss and update the weights with gradients as new data is encountered. Federated learning also works along these lines.

'Raw' online learning is unpopular because you have no garauntee that the network won't do something funky in the field.

That said, I think there are production systems in the world which learn on a day by day basis. Eg, take all of the logs from the last day and use them to update the production model for tomorrow. Then there's enough data that you don't risk a bad step. Think of it as learning while dreaming...

Came here to say the same thing. This is not a paper on mathematical artificial intelligence, at last as I understand it, not in the way Goertzel or Hutter write about it. I was actually thinking this would be something like Hutter's universal artificial intelligence.

We usually say "neural networks" and not "neuronal networks"

I think in Greek 'neura' and 'neuron' are equivalent. The perception that '-on' indicate the single or ideal entity ("ion, electron, muon...") comes from Whewell and Faraday ('-on' ex "ienai": the "goer" particles) and fully postdates the use of 'neuron'.

Edit: also see the comment from member martopix, nearby ( https://news.ycombinator.com/item?id=30988150 )

You have the correct etymology, but the convention is for "Neural Network" in ML and "Neuronal Network" in Neuroscience for we laymen.

As noted in the reference to the post from martopix :)

Reply on news.ycombinator.com