charleshmartin

'Technically' this is correct--the RBM CD algo is not minimizing this function; that's not the point.

It is known that when training an RBM, the reconstruction error decreases but not monotonically; in fact it fluctuates. In the words of Hinton, 'trust it but don't use it'.

http://www.cs.toronto.edu/~hinton/absps/guideTR.pdf (which is cited in the post as well)

So in a global sense, yes, I would say that the RBM does eventually minimize the reconstruction error even though it fluctuates.

I can even offer a conjecture here on why the error fluctuates ; in a discrete RG flow map, there could be finite size effects that would give log-periodic fluctuations. This is a stretch--but it is something that could be tested.

I explain this idea here http://charlesmartin14.wordpress.com/2015/01/16/the-bitcoin-...

As to stacking the RBMs to form a DBN--yeah that's the point. "Hinton showed that RBMs can be stacked and trained in a greedy manner to form so-called Deep Belief Networks (DBN)" http://deeplearning.net/tutorial/DBN.html

Thanks for the comments. It is helpful to have others read the blog and make suggestions and ask for clarifications. The motivations here are to (1) summarize and clarify the key ideas of the physics paper that observed this connection, and (2) set the stage for my next blog, where I try to connect Deep Learning to my idea around Spin Funnels.

I will review the comments and think how to update the blog to make it more clear.

I am happy to answer questions. You can ask here or directly on the blog.