Exact inference via quasi-conjugacy in two-parameter Poisson-Dirichlet hidden Markov models
By: Marco Dalla Pria, Matteo Ruggiero, Dario Spanò
We introduce a nonparametric model for time-evolving, unobserved probability distributions from discrete-time data consisting of unlabelled partitions. The latent process is a two-parameter Poisson-Dirichlet diffusion, and observations arise via exchangeable sampling. Applications include social and genetic data where only aggregate clustering summaries are observed. To address the intractable likelihood, we develop a tractable inferential framework that avoids label enumeration and direct simulation of the latent state. We exploit a duality between the diffusion and a pure-death process on partitions, together with coagulation operators that encode the effect of new data. These yield closed-form, recursive updates for forward and backward inference. We compute exact posterior distributions of the latent state at arbitrary times and predictive distributions of future or interpolated partitions. This enables online and offline inference and forecasting with full uncertainty quantification, bypassing MCMC and sequential Monte Carlo. Compared to particle filtering, our method achieves higher accuracy, lower variance, and substantial computational gains. We illustrate the methodology with synthetic experiments and a social network application, recovering interpretable patterns in time-varying heterozygosity.
Similar Papers
Efficient Inference for Coupled Hidden Markov Models in Continuous Time and Discrete Space
Machine Learning (Stat)
Helps predict how fires spread by watching them.
Gaussian approximations for fast Bayesian inference of partially observed branching processes with applications to epidemiology
Methodology
Makes tracking disease spread much faster.
From Partial Exchangeability to Predictive Probability: A Bayesian Perspective on Classification
Methodology
Helps computers guess better with less data.