Pré-Publication, Document De Travail Année : 2025

Bridging the inference gap in Mutimodal Variational Autoencoders

Résumé

From medical diagnosis to autonomous vehicles, critical applications rely on the integration of multiple heterogeneous data modalities. Multimodal Variational Autoencoders offer versatile and scalable methods for generating unobserved modalities from observed ones. Recent models using mixturesof-experts aggregation suffer from theoretically grounded limitations that restrict their generation quality on complex datasets. In this article, we propose a novel interpretable model able to learn both joint and conditional distributions without introducing mixture aggregation. Our model follows a multistage training process: first modeling the joint distribution with variational inference and then modeling the conditional distributions with Normalizing Flows to better approximate true posteriors. Importantly, we also propose to extract and leverage the information shared between modalities to improve the conditional coherence of generated samples. Our method achieves state-of-the-art results on several benchmark datasets.
Fichier principal
Vignette du fichier
bridging_the_gap_JNF.pdf (26.11 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04932313 , version 1 (06-02-2025)

Identifiants

  • HAL Id : hal-04932313 , version 1

Citer

Agathe Senellart, Stéphanie Allassonnière, Agathe Senellart. Bridging the inference gap in Mutimodal Variational Autoencoders. 2025. ⟨hal-04932313⟩
0 Consultations
0 Téléchargements

Partager

More