Hacker News Reader

Comment by formalsystem | Hacker News Reader

Generally, I tend to care more about math if it relates to my specific interests. I'm in ML and robotics so I've found it useful to understand algebra, topology, real analysis.

I've tried to get into Category theory a few times but couldn't keep my motivation high enough once things got too technical. Anyone know how category theory could be useful for someone in ML or robotics?

View on HN

Recently, I've been into learning about CRDTs a class of data structures which can be edited independently and merged without conflicts. They can be used to build experiences similar to Google Docs (although Gdocs uses a different approach called OT).

I'm interested in learning Category/Set Theory to better understand the underlying math of how they work.

I have very little experience with ML/Robotics, but CRDTs might be useful for distributed learning, or coordination between independent robots.

It's funny you mention them, I've been thinking a lot about CRDTs for online multiplayer games where you have a bunch of concurrent players changing a game state.

I've been taking a more domain-theoretic approach to understanding CRDTs(focusing more on the semantics than the syntax) but it appears that domain theory does have some connections to category theory:

https://ncatlab.org/nlab/show/domain+theory

What class was this if I may ask?

When I said "class", I really just meant "type".

CRDTs are relatively new so there may not be many classes yet that teach them. If you want to learn about them, I recommend you first watch this video: https://www.youtube.com/watch?v=OOlnp2bZVRs&list=WL&index=56... and then read this paper: https://pages.lip6.fr/Marc.Shapiro/papers/RR-7687.pdf . The video explains a lot of the math in a very easy to digest way and will make the paper much easier to understand.

I’m sorry, I misread. I was aware of CRDTs and I thought that they were taught in some class.

The structure of various ML primitives can be considered and under CT. Doubly so since in many cases you can in fact "go backwards" through a ML structure in a way normal functions can't, meaning that categorical duals might have more useful interpretations in your world.

So if anything, it might be MORE applicable for composing ML networks.

Yes. And other bits commonly associated with functional programming should also work together really well with ML. After all, most ML models are explicitly thought about as a composition of layers (ie functions).

But so far, the best ML libraries can be found in impure / imperative languages like Python.

Is there any paper or book you like that covers this topic in more depth?

Just look for introductions to duality in math and category theory. It's not the sort of subject that'll be broken out individually, but rather an assumed consequence.

My understanding is that category theory can be considered as abstract function theory. If we can prove that a function (e.g. Haskell compiler) is equivalent to automatic differentiation with some transformation, according to category theory, we can write Haskell code to train neural network, instead of writing extra graph building code in, say, Tensorflow. This paper basically shows this. http://conal.net/papers/essence-of-ad/essence-of-ad-icfp.pdf.

You may be interested in this paper too https://arxiv.org/abs/1711.10455

I think given a language with AD you already don't need to have a separate language for graph building and my understanding is that's one of the goals for the Julia team and I've seen other nice looking packages for AD in other languages

* https://fluxml.ai/2019/02/07/what-is-differentiable-programm...

* http://diffsharp.github.io/DiffSharp/examples-inversekinemat...

I'm also very excited about being able to design neural nets using ideas from functional programming. I love the Keras API but I don't think it's going to be the obvious API a couple of years down the line.

And for a completely different application of differentiation to programming, see http://strictlypositive.org/diff.pdf

I'm currently working on developing a double category of hybrid systems with applications to parallel composition and model reduction in robotics. We should have a math paper up on the arxiv within a month, and a robotics paper up shortly thereafter.

Looking forward to it! And if there's a preprint you're comfortable sharing please let me know!

Check out UMAP [1]. It's an alternative to t-SNE for dimension reduction that's grounded in topological category theory.

[1] https://arxiv.org/abs/1802.03426

Have you seen Robert Ghrist's book, "Elementary applied topology" ? There's plenty of category theory in there, even though it is mostly implicit. (And applications to ML and robotics.)

https://www.math.upenn.edu/~ghrist/notes.html

Yes! I skimmed it and really enjoyed it although I mostly remember seeing algebraic topology and not so much category theory, will give it a closer look.

Reply on news.ycombinator.com