Geometric Algebra for Python | Hacker News Reader

Geometric Algebra for Python

github.com

186 pointsalex_hirner69 comments

Hi everyone, I'm Hugo one of the maintainers of this package, glad to see lots of interest! If anyone is interested in some of the applications of GA in robotics etc then check out the talk Eric and I gave at GAME2020 back in February https://www.youtube.com/watch?v=bj9JslblYPU

A few months back I made an effort to learn GA. I did get some traction, but eventually shelved the effort. The thing that put me off is that no one could explain how to do units analysis in GA. I am mainly interested in physics and engineering applications, especially robotics and also RF engineering to some extent. Without being able to see how dimensions combine in GA, I have zero confidence in the correctness of my results. Arithmetic without dimensional analysis is not physics. Full. Stop.

So, if you could point to any resources that show how to properly dimension GA values, and how GA operators combine dimensions, that would get me started on GA again.

The units work just how you’d expect; as far as I can tell there’s nothing new or unique about units in GA.

If you multiply two vectors with units of meters, you get a scalar + a bivector, both with units of square meters.

If you divide two vectors with units of meters, you get a scalar + a bivector which are both dimensionless.

Etc.

What are units of the geometric product of two vectors?

Edit: didn't read clearly. Better question is why would we want to have a scalar with units of square meters in this case? Shouldn't 0-grade objects all have the same units?

> Shouldn't 0-grade objects all have the same units?

No, and this is a really important point. Scalars can have any units. For example, you can have a scalar with units of time, or mass, and you can take the dot product of a force vector and a displacement vector to get a scalar with units of [force]·[distance] = [work].

If you instead form the wedge product of those vectors, you get a bivector with the same units: [force]·[distance] = [torque] (note that the units of [work], [torque], and [energy] are all the same).

If you take the geometric product of a force vector and a displacement vector, the result is the sum of a scalar and a bivector, both with the same units of [force]·[distance].

> If you take the geometric product of a force vector and a displacement vector, the result is the sum of a scalar and a bivector, both with the same units of [force]·[distance].

This part is where I have problem with GA: what the hell is the physical[0] interpretation of such a sum? For example, a four-vector <p_x,p_y,p_z,c·E_k> (momentum and kinetic energy) can be thought of as kinetic energy being the temporal component of (4-)momentum, but no similar interpretation seems viable to combine work and torque into a logically unified quantity.

0: I'm not sure if this is the right word - the interpretation as single unified value with no special-case treatment of its components might be another, equally not-quite-right way of putting it.

The physical interpretation depends on context, but the concrete arithmetic is the same whether you write it using GA language or some other mathematical formalism (matrices, Gibbs–Heaviside style vectors, differential forms, tensors, complex numbers, ...), so if you have a problem with it your problem is with the physics.

Like with any expressive language (e.g. English, C++, or matrix algebra), GA makes it possible to state a wide variety of nonsensical things. But that’s not the fault of the language.

Where GA really shines IMO is in the ability to divide by vectors, something that is extremely useful but gets super cumbersome using other mathematical languages.

The biggest “problem” I have with GA is that it takes a lot of practice to get familiarity with all of the powerful stuff it can do. There are pages and pages of (extremely powerful and useful) short identities which are impossible to memorize by just looking at them, and can only be learned as far as I can tell through years of hard-won experience. I feel like I still only really have a handle on the most basic stuff.

I find regularly myself working on some complicated coordinate-based calculation for 3 pages of scratch paper full of mistakes and wrong turns, only to experience déjà vu, re-express the original thing in GA language, and end up with a clear and geometrically interpretable 4 lines of simple algebraic identities which solve the problem. But I’m not always sure if I’d be able to figure out which 4 lines to use right off the bat, without first going through the coordinate slog.

> some complicated coordinate-based calculation

Isn't that just what distinguishing scalar/vector/bivector/trivector/etc is for? I don't have problems with coordinates because I don't bother with coordinate-based calculations in the first place. What I don't get is how adding two multivectors of different [ranks? eg scalar + bivector] is supposed to simplify anything.

> > > the geometric product [...] is the sum of a scalar and a bivector

> any expressive language [...] makes it possible to state a wide variety of nonsensical things.

Yes, but if everything the language makes it possible to state is nonsensical, then what's the point?

I assume there's some sort of point to geometric algebra, but I have yet to encounter any convincing explanation of what that point is, and why I shouldn't keep my dot and wedge products properly separated.

I just said that for many single calculations, GA has saved me literally hours of headaches, in addition to making the result dramatically clearer and more insightful.

Nobody is going to force you to try something you don’t want to try. I don’t think I’ll be able to convince you.

> I just said that for many single calculations, GA has saved me literally hours of headaches, in addition to making the result dramatically clearer and more insightful.

Er, right; I was[0] asking for a example of a such a dramatically clearer and more insightful result, relative to just using normal dot and wedge products. Using vectors at all saves hours of headaches if all it's being compared to is coordinate-based calculations.

0: Well, the comment before that one was asking about the interpretation of the geometric product.

It’s kind of the same, actually. You can treat the individual entries as “components” in a 2^d-vector with special multiplicative structure, which essentially transforms in specific ways. These axioms of GA lay that out more clearly: https://arxiv.org/pdf/1205.5935.pdf (specifically axiom 6)

As I said, the units of the geometric product of two vectors (which if you like can be broken into the sum of a scalar part + a bivector part, each with the same units) will be the product of the units of each vector.

At some point, any GA computation bottoms out to scalars. Since you start out with quantities you already know how to give dimensions to, I would expect you can just follow standard dimensionsal analysis.

That isn't to say it's straightforward to implement computationally, or that there isn't interesting structure there.

Why is this any more of a problem than units in conventional vector calculus?

I don't understand the "geometric algebra" goal. The people that I've seen pushing for it usually say kids in high school are being taught the wrong thing, we should do away with the cross product and start teaching this geometric algebra. I'm not too familiar with the content, but to a superficial look it seems they are just using exterior algebras and (maybe?) clifford algebras? Assuming that is the case then why the rebranding (and sometime change of standard notation) to "geometric algebra" for concepts that already had a name at the end of the nineteenth century (that is bound to piss the general math community off)? Also why are people so against teaching the cross product? There are many reasons why Hodge duality is important, not just in abstract math but also in physics (it enables the most geometric definition of Maxwell equations for example)

Geometric algebra and Clifford algebras are technically the same thing. The difference is that the term geometric algebra is generally used to refer to the work of David Hestenes, while Clifford algebra is used in more formal settings.

David Hestenes is himself a physicists. He did a lot to bring Clifford algebras into the main public attention (mainly applying it to the study of space-time). He used the term "geometric algebra" because it looks like this was the term used by Clifford in his work.

There is nothing wrong with teaching the cross product, some people (me included) just think that teaching about exterior (and geometric) products would be more productive.

Using the term "geometric algebra" as Hestenes does is in keeping with the "don't name things after mathematicians" essay that was on HN yesterday:

https://news.ycombinator.com/item?id=24385389

My take is that linear algebra should be taught before calculus and that einstein notation should be taught at the same time as matrix multiplication.

Do you also think the nuance of co- and comtravariant tensors should be understood, and the conventions of raising and lowering indexes, which is where a lot of value of Einstein notation comes from?

Yes, much more preferable to 4 years of talking about "row vectors" and "column vectors." Perhaps not so early though.

Introducing the concepts in the context of a euclidean metric seems unmotivated?

While not necessarily in a first course, it is an important difference to appreciate which saves a lot of headache later. Eg: Vanilla gradient descent involves naively adding a covariant and a contravariant quantity, which is generically a recipe for disaster, and nicely motivates second order methods that use the Hessian as the "metric" to raise/lower indices. The picture about Hessian eigenvalues determining the step-size is of course complementary and helps gain an intuition for the meaning of a "metric" and its importance.

do you known of any resources teaching GD this way?

Unfortunately, no specific resource that I’m aware of, but you can search for the term “natural gradients”.

My take is that linear algebra dances around the lack of inverse. Math gets more interesting when you assume closure under inverse.

Here is a video https://www.youtube.com/watch?v=tX4H_ctggYo and the associated HN thread https://news.ycombinator.com/item?id=22282452 that provide some more background on GA and its applications.

I think you're generally right that GA is a rebranding of concepts we already knew about, however representing them in a uniform manner seems like a useful thing to do. The at t=4083 in the video they show some interesting examples of projections, rotations, and other geometric operations that can be done quite simply https://www.youtube.com/watch?v=tX4H_ctggYo&t=4083

AFAICT the proponents of GA claim it explains lots of things that don't make a whole lot of sense without it.

One example is why you multiply normals by the world_inverse_tranpose. Apparently the reason is because normals are actually bivectors, not directional vectors as they are usually explained.

another is that at least in code, they apparently solve lots of issues that other methods don't

See: https://marctenbosch.com/quaternions/

https://marctenbosch.com/quaternions/code.htm

and: https://bivector.net/

and https://www.youtube.com/watch?v=tX4H_ctggYo

It makes perfect sense without GA -- a precalculated normal vector is effectively a stored cross product (this is what the "bivector" terminology is attempting to explain, but is unclear). While the cross product is invariant under rotation and translation, scaling the inputs of the cross product is not the same as scaling the output -- you need to apply the inverse scaling to the output (you can derive this from basic identities).

The "inverse transpose" is a symbolic manipulation to derive a transformation matrix that inverts the scale, but not the rotation, though it's actually a bit overzealous, you can actually just use the cofactor matrix. https://github.com/graphitemaster/normals_revisited

> One example is why you multiply normals by the world_inverse_tranpose

I know linear algebra but I have no idea what the heck this means. Is this a gaming thing?

Computer graphics. When transforming a coordinate system by a linear transformation represented by a matrix, normal vectors to surfaces transform differently than regular vectors. GA/exterior algebra explains this: the useful thing we call normals is represented by the same data (a 3d vector) but actually has to have a a different "type" to make sense. In higher dimensions they are not interchangeable with vectors.

Ah gotcha, why don't bivectors transform normally

e: I'm restating the question in a form I understand, not asking it.

If T:V->V is a linear transformation and "^" is the exterior product then T(u^v)=T(u)^T(v).

For example, let T be a reflection about the plane spanned by e1 and e2. Then T(e1^e2)=e1^e2, but T(e1 x e2) = -e1 x e2, where "x" denotes the cross product.

I should have been clear, I was restating their question, not asking it again.

In LA you calculate the normal of a plane by taking the cross product of two vectors lying in that plane. If you transform all three with the same matrix the normal will in the general case no longer be the cross product of the transformed vectors, you have to transform it with the inverse transpose matrix instead.

Thanks, I think I was more confused by what `world_inverse_tranpose` is.

I understand that for gamedev it's more natural to think in terms of the exterior product of two vectors rather than the normal, because you have to deal with transformations a lot. But the normal is a vector, it's a definition that has been there for hundreds of years. There is no need to rename stuff, just use the thing that better fits your use case. In fact there are other things that are easier to understand in terms of normals, like the shape operator of a surface.

“Hundreds of years” is an exaggeration. The term vector originated in 1837 (according to Dutch Wikipedia). So I’m not so sure that “normal vector” is a term which is, say, a hundred years old.

Yeah right, it explains a lot of things, assuming you don't ask how to make sense of this alien thing called a 'bivector' in the first place.

That's when they start telling you, that you're supposed to be an elite mathematician, well versed in the higher forms of sophistry, and if that were the case you wouldn't need to understand bivector and wedge product and what not at an intuitive level.

I'm sold.

I'm not taking sides, but it's pedagogically important to reframe things from time to time even if there is no radically new math.

Especially when you teach applied math for nonmathematicians. For example teaching analysis using infinitesimals for engineers might be pedagogically better (for the level of understanding required).

I'm not expert in history of mathematics, but If I remember correctly there was lots of fighting over represetations related to hypercomplex systems, vectors and matrices in the 1800-century. Quaternion Society wanted to do it one way. https://www.nature.com/articles/052545a0

I use GA (via Versor) in rigging for animation because it's a lot easier for me to do the math that way vs. linear algebra. How I think about the world maps directly onto the math, and it requires ~1 LOC for each thing I'm trying to do. Debugging is trivial.

Anyway, that's why I use it. I'm 43.

I think there is a two-part answer to your question.

1. What's the difference between Geometric Algebra and exterior algebras? Clifford algebras?

This is a hand-wavy explanation, but the GA community has specific GAs give a geometric meaning to objects and operations of some Clifford algebra. For example, there's a 5D algebra (Conformal Geoemtric Algebra?), which is really a 2^5=32D algebra, where elements of the algebra can represent points, point pairs, circles, planes, etc. On the level of Clifford algebra, any of these objects is a point in a 32D space, and operations on points give you other points. That's sort of the mechanics. And then it's the job of GA to give a geometric interpretation of the points and the operations. To draw an analogy to logic, I think of it as syntax vs semantics.

2. Also, what's wrong with cross products?

I just explained how GA is sort of unifying and uniforming. All sorts of disparate geometric objects can be treated uniformly in one representation.

At the same time GA allows you to distinguish between objects that are different. If you give me a vector `A` and a vector `B`, and I return to you `C=A×B`, on the "syntax" level, A, B, and C look the same. If we're thinking in terms of coordinates, they are all triples of real numbers.

But `C` is unlike `A` or `B`. For example, if you wish to operate under a different coordinate system, you might have a transformation matrix M to convert between old and new coordinates. So for example `Anew = M * A`, `Bnew = M * B`, and `Cnew = det(M) * M^{-1}^{T} * C` [2]. What what? The explanationsSome would use is that `A` and `B` are true vectors or polar vectors, and `C` is a pseudovector or axial vector.

In code that I write, I would very much like to have some distinction between vectors that transform like `A` or `B` and vectors that transform like `C`. You can do this with naming conventions, or just making sure you do keep track in some other way that. A GA framework would give you a way to track this difference. In my opinion (and there are many ways to accomplish the same thing) the programming language type of `C` should be distinct from `A`.

I think Hodge duality and the cross product are important, and I honestly don't know what's right to teach kids in high school. The tradition we currently have gets us pretty far. On the other hand, how awkward is it to invoke the notion of C being a "pseudovector" [3]? I think most people never really learn that. The first time I really reckoned with that was when learning computer graphics, because normal vectors transform differently from tangent vectors. Incidentally, if what I wrote doesn't make sense to anyone, learning about these concepts by thinking of normal vectors vs tangent vectors is probably an intuitive motivation. This page should help: https://www.scratchapixel.com/lessons/mathematics-physics-fo...

For anyone who might not know, "an oriented 3-dimensional Euclidean space, an oriented plane can be represented by the exterior product of two basis vectors, and its Hodge dual is the normal vector given by their cross product" [1].

[1] https://en.wikipedia.org/wiki/Hodge_star_operator

[2] https://en.wikipedia.org/wiki/Cross_product#Algebraic_proper...

[3] https://en.wikipedia.org/wiki/Pseudovector

As a physics undergrad, most of these concepts started being introduced to me once we got into relativity and coordinate shifts were much more common.

I wish it had been earlier because I think it is important to have a strong idea of what the objects you are working with are and what properties they have.

> If you give me a vector `A` and a vector `B`, and I return to you `C=A×B`, on the "syntax" level, A, B, and C look the same.

Here you go:

  A = <1.0, 2.0, 3.0, 4.0>
  B = <5.0, 6.0, 7.0, 8.0>

What exactly is A×B supposed to look like there? :)

Cross product is awful. The wedge product, on the other hand, is:

  # C = <<Ax*By-Ay*Bx,Ax*Bz-Az*Bx,Ax*Bt-At*Bx,
          Ay*Bz-Az*By,Ay*Bt-At*By,Az*Bt-At*Bz>>
  # C = <<1*6-2*5,1*7-3*5,1*8-4*5,2*7-3*6,2*8-4*6,3*8-4*7>>
  # C = <<6-10,7-15,8-20,14-18,16-24,24-28>>
  C = <<-4.0,-8.0,-12.0,-4.0,-8.0,-4.0>>

Which works fine for any number of dimensions. (Assuming coordinate-lexicographic order (resonable) and that I didn't screw up the arithmetic (less so).)

Well if you think of the cross product as returning a single vector that is orthogonal to all the others, then your question "what is AxB" for $\mathbb{R}^4$ seems ill-specified.

You would instead need to do CrossProduct(A, B, C) and calculate the determinant of the 4x4 matrix with the last row being the $\mathbb{R}^4$ basis.

You could also create a function by lazy evaluation of CrossProduct(A, B) : $\mathbb{R}^4 \rightarrow \mathbb{R}^4$

> "what is AxB" for $\mathbb{R}^4$ seems ill-specified.

Thats also true for R^3: does <1,0,0>x<0,1,0> give <0,0,1>, or <0,0,-1>?

the determinant of the matrix {{1,0,0},{0,1,0},{e_1,e_2,e_3}} is e_3 so <0,0,1>.

it's completely specified by the order of A and B - there are two possible orders and two possible vectors.

It's not specified at all because there is no such matrix - the e_i are vectors, not scalars. At best it's a convenient mnemonic abbreviation for something decided on some other basis; that's like saying that the gradient is 'specified' by the 'vector' ∇ = <∂/∂x,∂/∂y,∂/∂z>.

There's interesting stuff like Conformal Geometric Algebra and Projective Geometric Algebra. The idea behind them being that geometric objects like lines, circles and points can be represented as elements of a Clifford algebra, with the various different products in GA having some natural geometric meaning. I don't know if it's useful, but it's certainly interesting.

It's great to see some geometric algebra.

But I'm not quite sure what the package is for.

Is it for playing with geometric algebra? It is really rather simple to implement geometric algebra. As easy as understanding the internals of this package (which is not small for what it does). And in some ways a necessity for understanding the math. Like I wouldn't expect someone to just use quaternions and matrices to do 3-D, without learning how a matrix or quaternion multiplies. And I wouldn't start with a package of dozens and dozens of files in multiple directories, with a ton of inter-imports†.

So I'm assuming the package is for practical use, so it will be nice to see what they envisage it for. The only example is a single trivial operation. I'm aware it can be used for quantum physics. But my experience has been 3-D graphics. And there it is very meh.

So my long-standing problem with geometric algebra in software engineering has been... why? A frustration because it is a lot of fun.

My (imho) advice if you find it fun: you can write your own version in an evening.

---

† I wonder if packages are equivalent of books 20 years ago. I remember that temptation to buy a book, as if that was how you acquired some knowledge. I was guilty of that so many times. Now, want to get geometric algebra? There is a package for that.

A few months back I did a few experiments with this package. It allows GA-ish notation in a Python script by providing GA types and operators. It this didn't exists, you would want to write it yourself, and probably do is far less well.

I can't speak to performance. I was using it as a GA interactive calculator while I tried to wrap my mind around GA.

> I remember that temptation to buy a book, as if that was how you acquired some knowledge.

yeah... what foolish things we believed 20 years ago...

(tries not to look at the large pile of books accumulating dust in the corner of the room)

Tsundoku

It provides a unified language for many areas of mathematics, which would need to be studied separately without GA. Also, with the GA toolbox, things become much more clear and better to understand.

You should check out the bivector community https://bivector.net/.

Join the discord https://discord.gg/vGY6pPk.

Check out a demo https://observablehq.com/@enkimute/animated-orbits

I work with ML. Even a dense vector can be relatively high-dimensional (something like 128-D is common) I'd love to use Geometric / Clifford algebra for rotations of these vectors; the notation is much more intuitive. However, last time I tried using this library, it insists on generating all the N-vectors, of which there's a combinatoral amount w.r.t. the number of dimensions. For rotations, I really only need bivectors (IIRC)

Is there a way to use Geometric Algebra for higher dimensions?

High dimensional GAs are a relatively hot topic of research but something we haven't aimed to target with this library (mostly because its not our main research area). My personal preference for working with high d algebras would be Stéphane Breuils and Vincent Nozick's GARAMON library (https://github.com/vincentnozick/garamon) (https://link.springer.com/article/10.1007/s00006-019-0987-7), it is efficient and easy to use, and the authors are very friendly and helpful. You could also try out TbGAL (https://github.com/Prograf-UFF/TbGAL), I haven't used it myself but the work of the Leandro Fernandes is typically high quality.

My package called Grassmann.jl offers somewhat higher dimensional algebras than the python package here:

https://github.com/chakravala/Grassmann.jl

What are the applications of Geometric Algebra aside from computer graphics?

The GAME2020 event (Geometric Algebra Mini Event) showcased applications in graphics, physics, robotics, neural networks, mathematics. The lectures (from Cambridge, UVA, UPEM, TU Delft, ..) are all available here : https://www.youtube.com/playlist?list=PLsSPBzvBkYjzcQ4eCVAnt...

"Mini Event" = several 1 hour long talks

Was hoping for more (time wise) digestible introductions.

There have been attempts to use it in physics, mainly as a way of unifying the various formalisms that get used there: http://geocalc.clas.asu.edu/html/Overview.html

From a paper (out of a quick google search) [1]:

* neural computing

* image and signal processing

* computer and robot vision

* control problems

1: https://arxiv.org/pdf/1305.5663.pdf

I have also seen some work on doing collision checking for robot arm kinematics -- which is a rather nasty problem when done the traditional way.

[edit to add] Also all the classic RF transmission line problems that one would normally solve with Smith Chart techniques map fairly elegantly onto GA(2)

If anyone wants to play around with the geometric algebra over R^3 in C++, I wrote a little library a while back which handles vectors (3d), quaternions (4d), bivectors (3d, quaternions without the scalar part), multivectors (8d)

https://github.com/markisus/g3

Please check out my geometric algebra software at https://github.com/chakravala/Grassmann.jl

I find it fascinating that geometric algebra and algebraic geometry are two distincts and totally different things and i wonder which term came first.

haha, well they are both probably about equally old i believe.. i think the two topics can be inter-related, but they are indeed separate areas of study with different emphasis, and I don't believe that many algebraic geometers know about the geometric algebra historically, since GA was quite obscure and hard to utilize until computer algebra made it easier

whereas AG is just as obscure and hard to utilize, even with a computer...

Geometric Algebra is useful because it's a generalization of complex number and quaternions.

It explains complex numbers and quaternions intuitively.

I agree with the first sentence.

But I don't think it's intuitive to learn about complex numbers through GA. Complex numbers are a great jumping-off point to learn GA since it is simple and most people eventually accept that the imaginary unit squares to -1, and because complex numbers have a striking geometric interpretation.

But I feel almost certain that people should learn about the special case of complex numbers before they ever learn about geometric algebra.

Reply on news.ycombinator.com