mailitics

Tag: kl

  • Surprisal-R’enyi Free Energy

    Surprisal-R’enyi Free Energy arXiv:2603.03405v1 Announce Type: new Abstract: The forward and reverse Kullback-Leibler (KL) divergences arise as limiting objectives in learning and inference yet induce markedly different inductive biases that cannot be explained at the level of expectations alone. In this work, we introduce the Surprisal-R’enyi Free Energy (SRFE), a log-moment-based functional of the likelihood…

  • Relaxed Triangle Inequality for Kullback-Leibler Divergence Between Multivariate Gaussian Distributions

    Relaxed Triangle Inequality for Kullback-Leibler Divergence Between Multivariate Gaussian Distributions arXiv:2602.02577v1 Announce Type: new Abstract: The Kullback-Leibler (KL) divergence is not a proper distance metric and does not satisfy the triangle inequality, posing theoretical challenges in certain practical applications. Existing work has demonstrated that KL divergence between multivariate Gaussian distributions follows a relaxed triangle inequality.…

  • Tail-Sensitive KL and R’enyi Convergence of Unadjusted Hamiltonian Monte Carlo via One-Shot Couplings

    Tail-Sensitive KL and R’enyi Convergence of Unadjusted Hamiltonian Monte Carlo via One-Shot Couplings arXiv:2601.09019v1 Announce Type: new Abstract: Hamiltonian Monte Carlo (HMC) algorithms are among the most widely used sampling methods in high dimensional settings, yet their convergence properties are poorly understood in divergences that quantify relative density mismatch, such as Kullback-Leibler (KL) and R’enyi…

  • Self-sufficient Independent Component Analysis via KL Minimizing Flows

    Self-sufficient Independent Component Analysis via KL Minimizing Flows arXiv:2512.00665v1 Announce Type: new Abstract: We study the problem of learning disentangled signals from data using non-linear Independent Component Analysis (ICA). Motivated by advances in self-supervised learning, we propose to learn self-sufficient signals: A recovered signal should be able to reconstruct a missing value of its own…

  • A Sharp KL-Convergence Analysis for Diffusion Models under Minimal Assumptions

    A Sharp KL-Convergence Analysis for Diffusion Models under Minimal Assumptions arXiv:2508.16306v1 Announce Type: new Abstract: Diffusion-based generative models have emerged as highly effective methods for synthesizing high-quality samples. Recent works have focused on analyzing the convergence of their generation process with minimal assumptions, either through reverse SDEs or Probability Flow ODEs. The best known guarantees,…