Geometry of the restricted Boltzmann machine | Hacker News Reader

Geometry of the restricted Boltzmann machine

arxiv.org

13 pointschromophore12 comments

Just came across this discussion. I'm one of the authors, and if anyone is interested, I'd be happy to answer any questions about the paper (jason@math.stanford.edu). It originated in me hearing from Andrew Ng and other folks in the ML community about how powerful deep belief nets were, and wanting to quantify that power more explicitly. The first step, for mathematicians, is to understand what a RBM can represent. As you point out, the paper is written for a math journal and so emphasizes certain things. I presented a poster from a more ML perspective at the ICML workshops. Cheers!

More than six people on here are able to comprehend this? I'm not one of them. Never cared much for "Algebraic Geometry" nor "Metric Geometry". An introduction to the theory behind Boltzmann machines would be much more relevant here, imo. Maybe something like this: http://learning.cs.toronto.edu/~hinton/absps/pdp7.pdf

"We study this graphical model from the perspectives of algebraic statistics and tropical geometry, starting with the observation that its Zariski closure is a Hadamard power of the first secant variety of the Segre variety of projective lines."

Look, if you don't know what that is right away just from the abstract I don't know how to help you...

... what with me having no clue either.

I've always disliked mathematics' proclivity for naming things after people. I don't have any better solution, but I still don't like it. Even things I do understand look like gibberish if it has three or four proper names like that.

I am reading this these days. I think you are right about this not being here. Sorry. Will keep in mind.

I agree this that article is of very limited interest. A lot of people in my lab study RBMs and I'm not sure I would even forward this article to them.

I don't blame you at all for submitting it. I'm just confused about who upvoted it and why.

LOL. The paper is very recent. And extremely impregnable for someone like me. I was actually hoping for some comments on it by people who work on RBMs. Like I got some good comments on a ML gallery I posted. Sometimes comments help clear up things, and even a little clearing up is good IMHO.

I think they did because they didn't have a look at the abstract, but read it partially by title only. And maybe you know, thought it was something introductory on RBM's ? I mean RBMs are extremely pretty. And I am just beginning to appreciate that.

There is seldom much interesting for ML person in this paper. (disclaimer: I haven't made much sense of its goal either. This reminds me of a saying “Write for lay audience, so maybe specialists will understand; Write for specialists, then noone will”)

It's just Bern Strumfels running his research program, which is somewhat obscure even for algebraic geometrers. He is an expert on algorithms of algebraic geometry and their applications e.g. to optimization, and whatnot. There was a thematic school on that, videos should be buried somewhere here, http://www.ima.umn.edu/2006-2007/. This may be interesting line of research—for a mathematician—or ML person willing to learn stacks of pure mathematics leading to fluency with at least the book by Cox, Little and O'Shea just to make sense of labels on buttons.

Hmmm, Okay! Enough said. Why can't there be much interest? :)

My old professor (who is an authority with RBMs) linked me up and asked me to see and make sure I know certain things about it.Let me give it some months and report back to him. I basically work with SVMs, it's his suggestion to study RBMs (which I know already) and sent me a number of papers. I am not sure why he sent me this one (it's just a month old). This one to be frank looked pretty nice to me, and I downloaded a number of other papers to study it. Even if I give up. I'll be glad to learn some of the math involved. I'd rather go with his opinion. Thanks.

The aim of my submission (as is the aim of most my submissions) was actually to get some comments on it, and links to some resources. Not what a person should learn and what not. I'll know better what to do and why. Given the wide pool of people on HN, I was hoping to get something interesting, say some comments ON the paper. Even one helpful comment would have done.

Sorry for not responding for a long time. I didn't mean to be snarky above, sorry about that too.

I wrote a bit more here, but wasn't satisfied with how it went so its no more. It was anyways mainly about Sturmfels' programme of reasoning about graphical models with algebraic geometry and how I see it and less TO the paper, so it was probably unsubstantial not only badly written.

By this time I'm sure you already know what program of algebraic statistics is anyways and surely can judge it for yourself. This paper fits it and is not directly consumable by machine learning just yet. Other parts of AS are. To actually use either parts the book I mentioned is minimum anyways and will clarify alot. Even if summary of the paper or AS itself is to be presented to a prof, there is no way around this book. The shortest intro to AG is by M. Reid, it freely builds on commutative algebra (short intro on J.S. Milne page at jmilne.org) that builds on abstract algebra. I haven't read the new book by Sturmfels and Sullivant but I'm half-sure it just assumes AG nor doesn't explain statistics.

Simpler, more cynical, explanation: it looks complicated and contains all sorts of fancy math in LaTeX, so people heuristically figure "this must be really important and deep".

Not saying all of the upvotes are due to that, but some significant fraction.

Wow - so e e cummings and charlie the cockroach are now a professor.

Seriously - nice work.

Reply on news.ycombinator.com