Goodman came by with his student Bo Zhu to talk about a method for sampling in the special case that the logarithm of the likelihood is quadratic in a residual (like a chi-squared) and the priors on the parameters are Gaussian (or improperly flat and unbounded). They have a very clever method, inspired by Levenberg-Marquardt optimization, that makes use of the derivative of the model with respect to the parameters; their method perfectly (that is, with unit autocorrelation time) samples any linear least-squares problem straight out of the box. They asked me for problems (nails for their MCMC Hammer, if you will); we discussed a few but decided to start by testing it on problems on which we have run our ensemble sampler.
Sounds cool, although I was under the impression that it was fairly straightforward to directly sample such posteriors. I do like the idea of creating MCMC methods by thinking of optimizers as degenerate cases of MCMC: for example, Hamiltonian MCMC turns into steepest descent optimization as the posterior goes towards a delta function at the maximum likelihood point.
ReplyDeleteBTW, here's a nail (although it is strong lensing, so you may want to ignore it): http://arxiv.org/abs/astro-ph/0601493
P. S. Despite the computational advantages, I'm not a huge fan of the methods in the above paper because they violate some of the most obvious criteria that a prior for sources should satisfy.
I started thinking about using MCMC in radio data analysis. My hope is that this allows you to identify FRI automatically by using an outlier distribution, which should be better than identifying it manually (clicking on the screen!!) or by hard clipping.
ReplyDelete