DO-EM: Density Operator Expectation Maximization
arXiv:2507.22786v1 Announce Type: cross
Abstract: Density operators, quantum generalizations of probability distributions, are gaining prominence in machine learning due to their foundational role in quantum computing. Generative modeling based on density operator models (textbf{DOMs}) is an emerging field, but existing training algorithms — such as those for the Quantum Boltzmann Machine — do not scale to real-world data, such as the MNIST dataset. The Expectation-Maximization algorithm has played a fundamental role in enabling scalable training of probabilistic latent variable models on real-world datasets. textit{In this paper, we develop an Expectation-Maximization framework to learn latent variable models defined through textbf{DOMs} on classical hardware, with resources comparable to those used for probabilistic models, while scaling to real-world data.} However, designing such an algorithm is nontrivial due to the absence of a well-defined quantum analogue to conditional probability, which complicates the Expectation step. To overcome this, we reformulate the Expectation step as a quantum information projection (QIP) problem and show that the Petz Recovery Map provides a solution under sufficient conditions. Using this formulation, we introduce the Density Operator Expectation Maximization (DO-EM) algorithm — an iterative Minorant-Maximization procedure that optimizes a quantum evidence lower bound. We show that the textbf{DO-EM} algorithm ensures non-decreasing log-likelihood across iterations for a broad class of models. Finally, we present Quantum Interleaved Deep Boltzmann Machines (textbf{QiDBMs}), a textbf{DOM} that can be trained with the same resources as a DBM. When trained with textbf{DO-EM} under Contrastive Divergence, a textbf{QiDBM} outperforms larger classical DBMs in image generation on the MNIST dataset, achieving a 40–60% reduction in the Fr’echet Inception Distance.
Abstract: Density operators, quantum generalizations of probability distributions, are gaining prominence in machine learning due to their foundational role in quantum computing. Generative modeling based on density operator models (textbf{DOMs}) is an emerging field, but existing training algorithms — such as those for the Quantum Boltzmann Machine — do not scale to real-world data, such as the MNIST dataset. The Expectation-Maximization algorithm has played a fundamental role in enabling scalable training of probabilistic latent variable models on real-world datasets. textit{In this paper, we develop an Expectation-Maximization framework to learn latent variable models defined through textbf{DOMs} on classical hardware, with resources comparable to those used for probabilistic models, while scaling to real-world data.} However, designing such an algorithm is nontrivial due to the absence of a well-defined quantum analogue to conditional probability, which complicates the Expectation step. To overcome this, we reformulate the Expectation step as a quantum information projection (QIP) problem and show that the Petz Recovery Map provides a solution under sufficient conditions. Using this formulation, we introduce the Density Operator Expectation Maximization (DO-EM) algorithm — an iterative Minorant-Maximization procedure that optimizes a quantum evidence lower bound. We show that the textbf{DO-EM} algorithm ensures non-decreasing log-likelihood across iterations for a broad class of models. Finally, we present Quantum Interleaved Deep Boltzmann Machines (textbf{QiDBMs}), a textbf{DOM} that can be trained with the same resources as a DBM. When trained with textbf{DO-EM} under Contrastive Divergence, a textbf{QiDBM} outperforms larger classical DBMs in image generation on the MNIST dataset, achieving a 40–60% reduction in the Fr’echet Inception Distance.
Adit Vishnu, Abhay Shastry, Dhruva Kashyap, Chiranjib Bhattacharyya
Go to original source