Metropolis Algorithm and Bayesian posterior sampling | Hacker News Reader

Metropolis Algorithm and Bayesian posterior sampling

efavdb.com

84 pointsefavdb14 comments

Metropolis Light Transport (http://graphics.stanford.edu/papers/metro/) is a clever rendering technique based on this sampling technique applied to light paths.

One thing people (myself included!) often missed though is that Metropolis is extremely prone to flicker. I think the best summary of why comes from Kalos and Whitlock (2nd Edition):

> The M(RT)2 algorithm is very simple and powerful; it can be used to sample essentially any distribution function regardless of analytic complexity in any number of dimensions. Complementary disadvantages are that sampling is correct only asymptotically and that successive variables produced are correlated, often very strongly. This means that the evaluation of integrals normally produces positive correlations in the values of the integrand, with consequent increase in variance for a fixed number of steps as compared with independent samples. Also the method is not well suited to sampling distributions with parameters that change frequently.

> One thing people (myself included!) often missed though is that Metropolis is extremely prone to flicker.

Doing PCA across frames basically eliminates that. IIRC Pixar did a paper on the technique.

Actually, the Renderman denoiser is based on work from Disney Research.

https://renderman.pixar.com/view/denoiser

PCA would be one way to do that. This recent paper suggests that they may be decomposing the samples mostly by path type, though:

https://www.disneyresearch.com/publication/pathspace-decompo...

He's referring to the older 2008 paper that did PCA, Statistical Acceleration for Animated Global Illumination: http://graphics.pixar.com/library/ShotRendering/paper.pdf

I found this tutorial was a good overview for both the Metropolis algorithm and for Metropolis Light Transport:

https://facweb.cs.byu.edu/graphics/publications/metropolisTu...

What's the rationale behind 100 as the thinning factor? Just a conservative number based on the autocorrelation length?

The thinning here is based on a misconception. If the purpose is plotting a histogram it doesn't matter if the samples are correlated. The bin heights are consistently estimated if you keep everything. Throwing away intermediate steps usually just introduces noise.

Thinning is a good idea if the samples are strongly correlated, and the computation to process them all would be better spent on running the chain for longer. Or if you don't know what computation you want in advance, and you can't afford to store everything.

(Also, as someone points out in the comments, the implementation of the Metropolis algorithm in this post is wrong.)

Can anyone here recommend a good reference (blog, e-book, etc.) for beginners in Bayesian statistical analysis?

You can have look at this

https://github.com/CamDavidsonPilon/Probabilistic-Programmin...

The only book accessible by beginners, while still dealing with real-world level analysis: https://www.amazon.com/Doing-Bayesian-Data-Analysis-Second/d...

Book:

- Information Theory, Inference and Learning Algorithms Textbook by David J. C. MacKay (free pdf: http://www.inference.phy.cam.ac.uk/itila/)

Other things:

- (the already mentioned http://camdavidsonpilon.github.io/Probabilistic-Programming-...)

- https://github.com/markdregan/Bayesian-Modelling-in-Python (a practical introduction to PyMC3)

Bayesian Logical Data Analysis for physical scientists by Phil Gregory is pretty good, particularly if you are some kind of scientist.

source: I TA'd a course in Data analysis, primarily bayesian, that used that as the textbook.

Statistical Rethinking (the lectures are available on youtube):

http://xcelab.net/rm/statistical-rethinking/

Reply on news.ycombinator.com