Two definitions of a kernel? Why "kernel trick"? | Hacker News Reader

Two definitions of a kernel? Why "kernel trick"?

syhw.posterous.com

21 pointssnippyhollow8 comments

This is a great example of why mathematicians have difficulty getting others to use and understand their work. What the OP says is perfectly true, yet completely incomprehensible unless you already know what he's talking about, at which point you don't need to read what he's saying.

I'm not claiming to explain why kernel machines are called this name to the layman. I just think it could be beneficial to people with a background in mathematics or CS (BA would suffice I'd say) that are interested in ML. If one is interested in ML (particularly kernel methods) but does not want to learn optimization and algebra, one will not be able to use (perform ? ;)) kernel tricks. I would have loved to read that during my Master for instance. (But perhaps it is just me scratching my own itch... Not a big deal anyway.)

I'm an undergrad CS major in my fourth year, and I have no idea what you are writing about, although it sounds like it might be interesting or useful if I understood it. The first stumbling block came right away, since I don't know what the notation you used here is supposed to mean:

"In algebra (kernel of a linear transform, of a homomorphism, of a matrix), a kernel is the set of elements that map to the neutral element. T : V → W ker(T) = {v ∈ V | T.v = 0}"

What is T.v? Is 0 the "neutral element"? What is meant by neutral element? (I am not an expert at algebra, since I am a CS major, and my major has been focused more on things like compilers, machine architecture, programming languages, algorithms, etc. rather than machine learning) What does V -> W ker(T) mean? I assume you're saying something like "T is a kernel of a function that maps V to W if T.v (where v is a V) is the neutral element", but this means nothing to me. Aren't you trying to explain what a kernel is...? How is the reader supposed to understand this if they don't already know what it is?

T is a linear transform/map or (it is the same thing!) a homomorphism, or a matrix: an application between 2 vector spaces that preserves vector addition and scalar multiplication (i.e. f(x+y) = f(x)+f(y), f(a.x) = a.f(x)).

T : V -> W is the signature of this homomorphism, whereas ker(T) = {v \in V | T.v = 0} means "ker of T is the set of vectors v in the vector space V such that T applied to v is the neutral element of W". The neutral element is noted 0 because, in a vector space, it is such that v + neutral_element = v. So neutral_element is the null vector.

I learned algebra kernels in my first year of college, so I assumed wrongly. But that's not the point: I was not thinking about explaining what a kernel is but instead (higher level) trying to group what seemed to be scattered definitions into explaining why we call "kernel machines" this (and why a kernel trick involves a kernel).

Now, I'm thinking about explaining (with pictures) what an algebra kernel is...

Here it is: http://i.imgur.com/8V7Xg.png

I'd suggest tossing that figure into the original post. It's a nice visual hook.

The video you linked "explains" it very well to visual learners such as myself: http://www.youtube.com/watch?v=3liCbRZPrZA

If a picture is worth a thousand words, a second of video is truly worth 29,970 words.

This video helped me too. Understanding something with one explanation is very good, but making the link with a second and a third one is even better.

Reply on news.ycombinator.com