Hacker News Reader

Comment by doyoulikeworms | Hacker News Reader

I’m able to follow neither the article nor the discussion. What would I have to learn in order to be able to?

Even if it was, like, years of studying. I’m just curious how deep this rabbit hole is.

Quaternions are an extension of the idea of complex numbers. Complex numbers have a real and an imaginary part, while quaternions have a real part and multiple imaginary parts (3). So the basic idea is that these richer types of number, when used to build a network (instead of plain real numbers) have benefits.

So to get started with reading this paper you just need to learn about deep learning, and then also the very basics of quaternions, which would be taught in, for example, a first course on abstract algebra.

I don’t think quaternions would be taught in a typical first course of abstract algebra. Do you know of a textbook where they are featured prominently?

Probably some of the introductory level Computer Graphics books. The field uses a lot of quaternions, so they might have a good explanation about it.

I don't know of any book which features them "prominently", but I also don't think you'd really need one. They are taught in various abstract algebra books, they're just taught in the fashion of, "Here's an exercise that introduces a peripheral topic it's useful to know about." For example, groups and rings of quaternions show up in MacLane & Birkhoff's Algebra (62, 426; 282) and Lang's Algebra (9, 545, 723, 758).

Edit: In an effort to find more applied information I put down my math books and picked up the information theoretic ones. You can find more information about the use of quaternions in the two volume Handbook of Digital Signal Processing and Salomon's Data Compression. More generally, when quaternions aren't explicitly referred to it's helpful to look up the coverage of complex rotations, especially with respect to the Discrete Fourier Transform.

For a discussion of rotations with quaternions in the context of animation, this is a reasonably short paper: http://www.cs.cmu.edu/~kiranb/animation/p245-shoemake.pdf.

When I took abstract algebra in Berkeley years ago, they were taught but they weren't the focus of the course. Basically, they're an example of skew field (division ring) so they have some interesting properties that was briefly studied. But obviously, one has to study more to understand their applications.

https://en.wikibooks.org/wiki/Abstract_Algebra/Quaternions or https://books.google.ie/books?id=ouvZKQiykf4C&pg=PA98&lpg=PA...

I am from Dublin, where quaternions were invented, so they get mentioned a lot by mathematicians and physicists here, maybe getting a higher billing than they do elsewhere. Computer graphics is obviously a place to go for introductions also, but it is typically going to be a more applied and less rigorous treatment.

I believe you missed a step there. After learning deep learning, GP would want to learn about how complex numbers are an improvement. Only then would one want to consider how quaternions are an improvement.

Quaternions are used to describe a rotation in 3D-space. Three numbers give the rotation axis and the magnitude of all four is the rotation angle. They are used instead of euler angles because they don't have a singularity problem (gimbal lock). I don't know if this plays a role here. Could these neural networks represent 3D transformations really well?

Geoffrey Hinton's idea of "capsules", which I don't really know anything concrete about, tries to address the recognition of objects subject to rotations, etc. That's a topological/structural strategy within the neural network, though, so quite removed from an idea like quaternions.

It's worth observing that what distinguishes complex numbers from 2-vectors like (x,y) is that there's a multiplication rule that corresponds to rotation around the origin. Similarly with quaternions. But you can also just use them as glorified vectors of 2 or 4 elements.

See this https://www.technologyreview.com/s/610278/why-even-a-moths-b... for a recent interesting finding on dimensionality.

In summary: Just as the complex numbers are defined by adding a new element i such that i^2 = -1, the quaternions are defined by adding elements i, j, k such that i^2 = j^2 = k^2 = ijk = -1.

Also, see https://en.wikipedia.org/wiki/Cayley%E2%80%93Dickson_constru...

Reply on news.ycombinator.com