Hacker News Reader

Comment by bogomipz | Hacker News Reader

The post states:

>"Perceptrons are the simplest form of machine learning and lend themselves to somewhat easier hardware implementations compared to some of the other machine learning algorithms."

Can someone explain what is it about perceptrons that make them easier to implement in hardware?

View on HN

There's a very approachable explanation in the following (around the 1:01:40 mark): https://youtu.be/8I_1TSs695I?t=1h1m40s -- part of Design of Digital Circuits - Lecture 18: Branch Prediction II (ETH Zürich, Spring 2019). Related readings: https://safari.ethz.ch/digitaltechnik/spring2019/doku.php?id....

These lectures are pretty great, by the way, highly recommended to anyone interested in computer architecture: http://people.inf.ethz.ch/omutlu/lecture-videos.html

Incidentally, this year's High-Performance Computer Architecture Test of Time Award has been given to "Dynamic Branch Prediction with Perceptrons" referenced in the lecture (from 2001, https://www.cs.utexas.edu/~lin/papers/hpca01.pdf): https://engineering.tamu.edu/news/2019/02/jimenez-receives-h....

These are fantastic links, thanks! I've watched many of Onur Mutlu's CMU lectures he's a wonderful teacher. Cheers.

All you need for a perceptron is to add up all of the inputs after they’ve been multiplied by their weight. This would probably be done in parallel with a fused multiply add circuit which is fairly simple and some place to store the weights. When they change the perceptron all they need to do is record the current weights and load in the new values.

Makes sense, thanks for the explanation.

Wait, how is K-NN harder to implement than a perception... Even in hardware? What about linear regression? What about decision trees? What about naive Bayes?

Performing nearest neighbors search is much more expensive than a simple multiply-add. I don’t know about the other, my idea is that a perceptron is much easier to implement in hardware/low-level operations.

A single layer perceptron IS linear regression + some non-linearity of the output. I think in reality they could have used the word "logistic regression" but chose perceptron because it sounds better.

Reply on news.ycombinator.com