Hacker News Reader

+1 on all of this (except I'm a fan of full MCMC, have you used ADVI?)

Haven't tried ADVI yet, but I think it is (likely) a great idea. In general automatic differentiation is a wildly powerful technique and is underutilized in computational science. Even in pure optimization problems people rarely leverage it to get J or H information as often as they probably should. I'll admit to not being super familiar with all the machinery under the hood in the ADVI paper [1]. It wasn't out when I started working on my current project.

[1] http://arxiv.org/abs/1506.03431

do let me know how it goes if you do try ADVI.

we're also working on making it more robust to initialization and step-sizes. stay tuned.

AVDI is incredibly awesome. Thanks so much for this work. Really nice readable paper on it too.

As someone working with large datasets I think automating variational inference + SGD to work with a broad class of models is really the way forwards and a big force multiplier for machine learning.

Reply on news.ycombinator.com