In Bayesian signal processing, all the information about the unknowns of interest is contained in their posterior distributions.
The unknowns can be parameters of a model, or a model and its parameters. In many important problems, these distributions
are impossible to obtain in analytical form. An alternative is to generate their approximations by Monte Carlo-based methods
like Markov chain Monte Carlo (MCMC) sampling, adaptive importance sampling (AIS) or particle filtering (PF). While MCMC
sampling and PF have received considerable attention in the literature and are reasonably well understood, the AIS methodology remains relatively unexplored. This article reviews the basics of AIS as well as provides a comprehensive survey of the state-of the-art of the topic. Some of its most relevant implementations are revisited and compared through computer simulation examples.
Importance Sampling methods are broadly used to approximate posterior distributions or some of their moments. In its
standard approach, samples are drawn from a single proposal distribution and weighted properly. However, since the performance depends on the mismatch between the targeted and the proposal distributions, several proposal densities are often employed for the generation of samples. Under this Multiple Importance Sampling (MIS) scenario, many works have addressed the selection or adaptation of the proposal distributions, interpreting the sampling and the weighting steps in different ways. In this paper, we establish a general framework for sampling and weighting procedures when more than one proposal is available. The most relevant MIS schemes in the literature are encompassed within the new framework, and, moreover novel valid schemes appear naturally. All the MIS schemes are compared and ranked in terms of the variance of the associated estimators. Finally, we provide illustrative examples which reveal that, even with a good choice of the proposal densities, a careful interpretation of the sampling and weighting procedures can make a significant difference in the performance of the method.
Authors: John R. Dixon
Comments: 41 Pages.
This is the technical report to accompany:
Dixon, John R., Michael R. Kosorok, and Bee Leng Lee. "Functional inference in semiparametric models using the piggyback bootstrap." Annals of the Institute of Statistical Mathematics 57, no. 2 (2005): 255-277.