Variational Bayesian mixtures enable multi-omics selection but have not been validated against sampling-based sparse clustering

Question: How do variational inference Bayesian mixture models handle joint variable selection across multi-view or multi-omics data, and how do they compare with sampling-based sparse clustering approaches?

Answer

Taken together, the retrieved papers support a narrower conclusion than the question presumes. Joint variable selection plus clustering across multiple omics views is currently delivered by exactly one family of variational Bayesian mixtures — categorical finite mixtures with per-feature saliency and explicit model averaging to escape variational local optima — and that method's evidence is empirical and application-driven, not comparative against sparse sampling-based clustering [15]. The theoretical case for variational inference (VI) in this role is real but indirect: the selection-consistency and cluster-number-consistency guarantees that have been proved for VI are proved for single-view, single-block problems — sparse or over-specified mixtures on one feature matrix, or variable selection in one regression — and not for joint selection across views [2] [4] [5]. Conversely, the strongest formal results in the sparse mixture literature (minimax-optimal posterior contraction, mis-clustering rates) are stated for continuous spike-and-slab priors fit by sampling to a single high-dimensional data block [5]. Meanwhile, the multi-view Bayesian mixture work that does exist has aimed at a different problem — modelling dependence between partitions across views, with no feature selection at all [1] — while genuine joint multi-view selection is largely done by non-Bayesian path solvers and information-theoretic objectives [6] [13]. The defensible position is that VI is what makes joint multi-omics selection computationally possible, while sampling-based sparse clustering remains the source of the guarantees; the two have not been put on a common benchmark.

Variational Bayesian mixtures are the one demonstrated route to simultaneous feature selection and subtype discovery on multi-omics data

VICatMix is a variational Bayesian finite mixture for categorical data in which VI is explicitly credited with outperforming competitors on computational time and scalability while maintaining high accuracy, and which performs variable selection that improves performance on high-dimensional, noisy data [15]. It is the only retrieved method that combines VI, per-feature selection (feature saliency), and a stated potential utility for integrative cluster analysis of different 'omics datasets, with applications to TCGA cancer subtyping and driver gene discovery [15]. Notably, the design does not trust the variational objective alone: it builds in summarization and model averaging to mitigate poor local optima in VI so that the number of clusters and feature saliency can be estimated jointly [15]. Related latent-variable mixture work achieves simultaneous dimension reduction and clustering but does not perform feature selection [17].

Moving the approximation onto the model space preserves the selection-consistency property that motivates the Bayesian formulation

The standard objection to VI — that it approximates coefficients rather than the selection problem — has been addressed directly by developing a variational approximation to the marginal distribution on the model space instead of on the regression coefficients, with a proof that the approximation inherits the strong selection consistency satisfied by the posterior it approximates [2]. For mixtures specifically, matching upper and lower bounds on the ELBO without conjugacy assumptions suggest that model selection by maximizing the ELBO is consistent, and the mean-field approximation is shown to inherit the posterior's stable behaviour of eliminating extra components under over-specification, with an $n^{-1/2}$ parameter convergence rate up to a logarithmic factor [4]. These results make "VI is fast but wrong about the support" a less tenable blanket claim [2] [4].

But the known mean-field pathologies fall precisely on cluster number and uncertainty, which is where multi-view selection is decided

Mean-field classes fail to capture parameter dependence and typically underestimate uncertainty; bagged variational posteriors are needed to recover off-diagonal covariance structure and valid uncertainty quantification [10]. Bayesian nonparametric mixtures fitted to real data yield unbalanced partitions with a few dominant clusters and many sparsely populated ones, a failure traced to the cost function used for point estimation and correctable only by post-processing the posterior [9]. Clustering results from Gibbs-sampled mixtures are likewise highly sensitive to kernel misspecification, with even slight non-Gaussianity within a cluster splitting it into multiple Gaussian components — a failure mode that requires an explicit fusing mechanism rather than a better sampler [18]. Since small, rare multi-omic subtypes are exactly the target of joint selection, these are the load-bearing failure modes, and the retrieved VI literature addresses them by averaging and post-processing rather than by resolving them [15] [9].

Sampling-based sparse mixtures hold the stronger theory, but that theory stops at one view

The cleanest guarantee in the retrieved sparse clustering literature belongs to a continuous spike-and-slab Bayesian Gaussian mixture whose posterior contraction rate is proved minimax optimal, with a mis-clustering rate derived by matrix perturbation arguments and no need to pre-specify the number of clusters [5]. That result is for a single high-dimensional Gaussian mixture, demonstrated on one scRNA-seq dataset, and carries no multi-view or per-view selection structure [5]. Gibbs sampling underlies the comparable single-block frameworks that report honest uncertainty quantification and information criteria for choosing the number of clusters and factors [7], and the explicit trade-off is stated plainly elsewhere: EM returns efficient point estimates, while collapsed Gibbs returns a full posterior and an estimate of the number of components at substantially greater computational cost [12]. Bayesian cluster validity indices further show that the choice of cluster count is treated as a post-hoc, prior-regularized decision rather than something the sampler settles [8].

Multi-view Bayesian clustering has solved cross-view partition dependence and delegated joint selection elsewhere

The multiview case is described as having received little attention in the Bayesian clustering literature, and the response — CLIC, induced by a product centred Dirichlet process — uses a single parameter to model dependence between the statistically dependent but distinct clusterings obtained in each view, with a marginal Gibbs sampler and closed-form partition distributions [1]. It performs no variable selection [1]. Where joint multi-view or multi-omics selection is actually performed, the retrieved methods are not Bayesian mixture models: $\ell_\infty/\ell_0$-constrained weighted sparse PLS selects a subset of samples and a subset of features jointly and is extended to multi-view fusion [6], and a deterministic information bottleneck formulation performs joint feature weighting and clustering on genomics data [13]. Penalized model-based clustering with variable selection achieves selection consistency even under missing-not-at-random data in transcriptomics, but is again single-view [14].

Conflicts and gaps

There is no head-to-head comparison in the retrieved set between a VI Bayesian mixture with joint multi-view selection and a sampling-based sparse clustering method; VICatMix's time-and-scalability advantage is asserted against unspecified competitors, not against a sparse Gibbs sampler [15]. The one direct VI-versus-MCMC statement runs the other way in a special regime: in federated over-fitted mixtures aggregated by variational consensus Monte Carlo, the approach recovers small clusters more accurately than standard MCMC applied to the pooled data when local dataset composition reflects the true clustering structure [3]. Papers also disagree in emphasis about whether VI's tendency to prune components is a virtue or a defect — it is framed as inherited posterior stability and consistent model selection [4], as something needing model averaging to avoid poor local optima [15], and as a general uncertainty-deficit requiring bagging [10]. Two gaps are structural: all retrieved VI-based selection theory is single-view, and all retrieved Bayesian multi-view machinery is selection-free.

What would change the answer

A paired-omics benchmark (e.g. TCGA expression plus methylation or copy number) comparing VICatMix-style variational saliency against a continuous spike-and-slab Gaussian mixture fit by Gibbs with per-view selection indicators, held to a common ground truth, would directly settle the comparison [15] [5]. Simulations varying cross-view feature correlation and the size of the rarest cluster would test whether the mean-field uncertainty deficit translates into false positives in the selected support [10]. Evaluating feature-level error rates, in the spirit of false-discovery-rate-controlling variable-level inference for clustering [16], would convert "maintaining high accuracy" into a calibration claim. Finally, posterior summaries that expose multimodality in the partition rather than a single representative clustering would reveal whether variational multi-view solutions collapse distinct posterior modes [11].

References

[1] Dombowsky A, Dunson DB. "Product Centred Dirichlet Processes for Bayesian Multiview Clustering.". pubmed. 2025-01-01. URL: https://doi.org/10.1093/jrsssb/qkaf021

[2] Yiqi Tang, Ryan Martin. "Variational empirical Bayes variable selection in high-dimensional logistic regression". arxiv. 2025-02-18. URL: https://arxiv.org/abs/2502.10532

[3] Julie Fendler, Francesca L. Crowe, Tom Marshall, Sylvia Richardson, Paul D. W. Kirk. "Variational Consensus Monte Carlo for Bayesian Mixture". arxiv. 2026-06-19. URL: https://arxiv.org/abs/2606.19643

[4] Chenyang Wang, Yun Yang. "Estimating the Number of Components in Finite Mixture Models via Variational Approximation". arxiv. 2026-04-23. URL: https://arxiv.org/abs/2404.16746

[5] Yao D, Xie F, Xu Y. "Bayesian Sparse Gaussian Mixture Model for Clustering in High Dimensions.". pubmed. 2025-01-01. URL:

[6] Min W, Xu T, Ding C. "Weighted Sparse Partial Least Squares With Joint Sample and Feature Selection for Integrating Multi-Omics Data.". pubmed. 2025-11-01. URL: https://doi.org/10.1109/TCBBIO.2025.3605691

[7] Shin H, Ferreira MAR, Tegge AN. "Bayesian Clustering Factor Models.". pubmed. 2026-01-01. URL: https://doi.org/10.1002/sim.70350

[8] Nathakhun Wiroonsri, Onthada Preedasawakul. "A Bayesian cluster validity index". arxiv. 2025-12-24. URL: https://doi.org/10.1016/j.csda.2024.108053

[9] Beatrice Franzolini, Giovanni Rebaudo. "Entropy regularization in probabilistic clustering". arxiv. 2026-02-03. URL: https://doi.org/10.1007/s10260-023-00716-y

[10] Shitao Fan, Ilsang Ohn, David Dunson, Lizhen Lin. "Variational bagging: a robust approach for Bayesian uncertainty quantification". arxiv. 2025-11-26. URL: https://arxiv.org/abs/2511.20594

[11] Cecilia Balocchi, Sara Wade. "Beyond point estimation: explaining the posterior in Bayesian cluster analysis". arxiv. 2026-09-18. URL: https://arxiv.org/abs/2506.16295

[12] M. E. J. Newman. "Mixture models for data with unknown distributions". arxiv. 2025-02-28. URL: https://arxiv.org/abs/2502.19605

[13] Efthymios Costa, Ioanna Papatsouma, Angelos Markos. "Sparse clustering via the Deterministic Information Bottleneck algorithm". arxiv. 2026-04-14. URL: https://arxiv.org/abs/2601.20628

[14] Binh H. Ho, Long Nguyen Chi, TrungTin Nguyen, Binh T. Nguyen, Van Ha Hoang, Christopher Drovandi. "A Unified Framework for Variable Selection in Model-Based Clustering with Missing Not at Random". arxiv. 2025-11-05. URL: https://arxiv.org/abs/2505.19093

[15] Rao J, Kirk PDW. "VICatMix: variational Bayesian clustering and variable selection for discrete biomedical data.". pubmed. 2025-01-01. URL: https://doi.org/10.1093/bioadv/vbaf055

[16] Shen H, Li D, Liu Y. "A general framework for testing clustering significance and variable-level inference in high-dimensional data.". pubmed. 2026-07-01. URL: https://doi.org/10.1093/biomtc/ujag145

[17] Markarian N, Engelhardt BE, Pierce NA, Sternberg PW, Pachter L. "k -spaces: Mixtures of Gaussian latent variable models.". pubmed. 2025-01-01. URL: https://doi.org/10.1101/2025.11.24.690254

[18] Dombowsky A, Dunson DB. "Bayesian Clustering via Fusing of Localized Densities.". pubmed. 2025-01-01. URL: https://doi.org/10.1080/01621459.2024.2427935

How this survey was made

This survey answers one question: “How do variational inference Bayesian mixture models handle joint variable selection across multi-view or multi-omics data, and how do they compare with sampling-based sparse clustering approaches?”

We keep our own collection of recent preprints and refresh it every day from bioRxiv, medRxiv, arXiv, PsyArXiv, SocArXiv and ChemRxiv. For each survey, we search that collection for recent work and keep the strongest matches: papers that use the same terms as the question, and papers on the same topic in different words.

The text was written from the titles and abstracts of those papers; the full texts were not read. Statements that draw on a particular paper are marked with a number in brackets. The reference list just above is the 18 papers whose work this text draws on, chosen from the 138 papers those searches returned; the rest of the shortlist is not printed. It is not a complete review of everything published on the question.

Treat this as a starting point and a reading list, not a peer-reviewed review. The searches behind this survey ran on October 29, 2026 and covered work from the previous two years.