Hacker News Reader

Comment by asdf_snar | Hacker News Reader

I think Geyer's point is that if you generate samples X_0, ..., X_b where b>0 is a burn-in time, and start collecting estimates at time t>b, then that's not somehow better than starting at X_b and not burning in (the two are equal in distribution).

View on HN

The problem there is that (assuming a source of true noise) X_b is a random variable, in fact if b is sufficiently high its distribution approximates your target distribution pretty well. So if you're arguing "instead of burn-in you can also use an approximate sample of your target distribution" then that's technically correct.

However generating an approximate sample is the exact problem that a MCMC process is trying to solve.

Of course, you are right. I recall Geyer at some point writing some people thought there was a difference between having a burn-in period that happened to have X_b = x_b, and simply starting with X_0 = x_b. I can't find that on this page though.

I see, you're right that those two are entirely equivalent (it's what makes the Markov chain 'Markov'). And using an approximate sample instead of burn-in isn't entirely without merit, though I think I've only seen that being used with things like Brownian motion where the trajectory is the thing of interest, rather than the samples themselves.

Yes, he writes "If you start at x and I start at x then your MCMC run is no better than mine." and surrounding text is making this point.

At that point, though, isn't it just semantics?

If you have a chain, and throw out the beginning fraction, is that any more burnin than starting with X_b, which you probably had to find using some period of... burnin?

This is really all about time series heterogeneity which is a topic unto itself. Eventually the chain reaches some stationary distribution but in the meantime there will be some shift in the distribution potentially. Seems like you could model the chain to do some test of where the distribution shifts, if at all.

I imagine this is in the literature. Giving it a brief look I can't find anything but I haven't looked hard enough.

Reply on news.ycombinator.com