mailitics

Tag: matrix

  • Implicit Bias and Convergence of Matrix Stochastic Mirror Descent

    Implicit Bias and Convergence of Matrix Stochastic Mirror Descent arXiv:2602.18997v1 Announce Type: new Abstract: We investigate Stochastic Mirror Descent (SMD) with matrix parameters and vector-valued predictions, a framework relevant to multi-class classification and matrix completion problems. Focusing on the overparameterized regime, where the total number of parameters exceeds the number of training samples, we prove…

  • Split-and-Conquer: Distributed Factor Modeling for High-Dimensional Matrix-Variate Time Series

    Split-and-Conquer: Distributed Factor Modeling for High-Dimensional Matrix-Variate Time Series arXiv:2601.11091v1 Announce Type: new Abstract: In this paper, we propose a distributed framework for reducing the dimensionality of high-dimensional, large-scale, heterogeneous matrix-variate time series data using a factor model. The data are first partitioned column-wise (or row-wise) and allocated to node servers, where each node estimates…

  • Glitches in the Attention Matrix

    Glitches in the Attention Matrix A history of Transformer artifacts and the latest research on how to fix them The post Glitches in the Attention Matrix appeared first on Towards Data Science. Jonathan Williford Go to original source

  • PCA recovery thresholds in low-rank matrix inference with sparse noise

    PCA recovery thresholds in low-rank matrix inference with sparse noise arXiv:2511.11927v1 Announce Type: new Abstract: We study the high-dimensional inference of a rank-one signal corrupted by sparse noise. The noise is modelled as the adjacency matrix of a weighted undirected graph with finite average connectivity in the large size limit. Using the replica method from…

  • Gaussian Equivalence for Self-Attention: Asymptotic Spectral Analysis of Attention Matrix

    Gaussian Equivalence for Self-Attention: Asymptotic Spectral Analysis of Attention Matrix arXiv:2510.06685v1 Announce Type: new Abstract: Self-attention layers have become fundamental building blocks of modern deep neural networks, yet their theoretical understanding remains limited, particularly from the perspective of random matrix theory. In this work, we provide a rigorous analysis of the singular value spectrum of…

  • A Probabilistic Basis for Low-Rank Matrix Learning

    A Probabilistic Basis for Low-Rank Matrix Learning arXiv:2510.05447v1 Announce Type: new Abstract: Low rank inference on matrices is widely conducted by optimizing a cost function augmented with a penalty proportional to the nuclear norm $Vert cdot Vert_*$. However, despite the assortment of computational methods for such problems, there is a surprising lack of understanding of…

  • Holdout cross-validation for large non-Gaussian covariance matrix estimation using Weingarten calculus

    Holdout cross-validation for large non-Gaussian covariance matrix estimation using Weingarten calculus arXiv:2509.13923v1 Announce Type: cross Abstract: Cross-validation is one of the most widely used methods for model selection and evaluation; its efficiency for large covariance matrix estimation appears robust in practice, but little is known about the theoretical behavior of its error. In this paper,…

  • Instance-Optimal Matrix Multiplicative Weight Update and Its Quantum Applications

    Instance-Optimal Matrix Multiplicative Weight Update and Its Quantum Applications arXiv:2509.08911v1 Announce Type: cross Abstract: The Matrix Multiplicative Weight Update (MMWU) is a seminal online learning algorithm with numerous applications. Applied to the matrix version of the Learning from Expert Advice (LEA) problem on the $d$-dimensional spectraplex, it is well known that MMWU achieves the minimax-optimal…

  • Understanding Matrices | Part 4: Matrix Inverse

    Understanding Matrices | Part 4: Matrix Inverse The physical meaning of matrix inversion, related formulas, and how inversion behaves on several special types of matrices. The post Understanding Matrices | Part 4: Matrix Inverse appeared first on Towards Data Science. Tigran Hayrapetyan Go to original source

  • A Bird’s-Eye View of Linear Algebra: Why Is Matrix Multiplication Like That?

    A Bird’s-Eye View of Linear Algebra: Why Is Matrix Multiplication Like That? Since the way we manipulate high-dimensional vectors is primarily matrix multiplication, it isn’t a stretch to say it is the bedrock of the modern AI revolution. The post A Bird’s-Eye View of Linear Algebra: Why Is Matrix Multiplication Like That? appeared first on…

  • Inequalities for Optimization of Classification Algorithms: A Perspective Motivated by Diagnostic Testing

    Inequalities for Optimization of Classification Algorithms: A Perspective Motivated by Diagnostic Testing arXiv:2508.01065v1 Announce Type: new Abstract: Motivated by canonical problems in medical diagnostics, we propose and study properties of an objective function that uniformly bounds uncertainties in quantities of interest extracted from classifiers and related data analysis tools. We begin by adopting a set-theoretic…

  • A Smoothing Newton Method for Rank-one Matrix Recovery

    A Smoothing Newton Method for Rank-one Matrix Recovery arXiv:2507.23017v1 Announce Type: new Abstract: We consider the phase retrieval problem, which involves recovering a rank-one positive semidefinite matrix from rank-one measurements. A recently proposed algorithm based on Bures-Wasserstein gradient descent (BWGD) exhibits superlinear convergence, but it is unstable, and existing theory can only prove local linear…

  • Confusion Matrix Made Simple: Accuracy, Precision, Recall & F1-Score

    Confusion Matrix Made Simple: Accuracy, Precision, Recall & F1-Score How to evaluate classification models and understand which metric matters the most. The post Confusion Matrix Made Simple: Accuracy, Precision, Recall & F1-Score appeared first on Towards Data Science. Nikhil Dasari Go to original source

  • Transfer Learning for Matrix Completion

    Transfer Learning for Matrix Completion arXiv:2507.02248v1 Announce Type: new Abstract: In this paper, we explore the knowledge transfer under the setting of matrix completion, which aims to enhance the estimation of a low-rank target matrix with auxiliary data available. We propose a transfer learning procedure given prior information on which source datasets are favorable. We…

  • Understanding Matrices | Part 2: Matrix-Matrix Multiplication

    Understanding Matrices | Part 2: Matrix-Matrix Multiplication The physical meaning of multiplying two matrices and how it works on several special matrices. The post Understanding Matrices | Part 2: Matrix-Matrix Multiplication appeared first on Towards Data Science. Tigran Hayrapetyan Go to original source

  • Infinitesimal Higher-Order Spectral Variations in Rectangular Real Random Matrices

    Infinitesimal Higher-Order Spectral Variations in Rectangular Real Random Matrices arXiv:2506.03764v1 Announce Type: new Abstract: We present a theoretical framework for deriving the general $n$-th order Fr’echet derivatives of singular values in real rectangular matrices, by leveraging reduced resolvent operators from Kato’s analytic perturbation theory for self-adjoint operators. Deriving closed-form expressions for higher-order derivatives of singular…

  • Understanding Matrices | Part 1: Matrix-Vector Multiplication

    Understanding Matrices | Part 1: Matrix-Vector Multiplication The physical meaning of multiplying a matrix by a vector, and how it works on several special matrices. The post Understanding Matrices | Part 1: Matrix-Vector Multiplication appeared first on Towards Data Science. Tigran Hayrapetyan Go to original source

  • Randomised Optimism via Competitive Co-Evolution for Matrix Games with Bandit Feedback

    Randomised Optimism via Competitive Co-Evolution for Matrix Games with Bandit Feedback arXiv:2505.13562v1 Announce Type: new Abstract: Learning in games is a fundamental problem in machine learning and artificial intelligence, with numerous applications~citep{silver2016mastering,schrittwieser2020mastering}. This work investigates two-player zero-sum matrix games with an unknown payoff matrix and bandit feedback, where each player observes their actions and the…

  • Computational Efficient Informative Nonignorable Matrix Completion: A Row- and Column-Wise Matrix U-Statistic Pseudo-Likelihood Approach

    Computational Efficient Informative Nonignorable Matrix Completion: A Row- and Column-Wise Matrix U-Statistic Pseudo-Likelihood Approach arXiv:2504.04016v1 Announce Type: new Abstract: In this study, we establish a unified framework to deal with the high dimensional matrix completion problem under flexible nonignorable missing mechanisms. Although the matrix completion problem has attracted much attention over the years, there are…

  • Fast Debiasing of the LASSO Estimator

    Fast Debiasing of the LASSO Estimator arXiv:2502.19825v1 Announce Type: new Abstract: In high-dimensional sparse regression, the textsc{Lasso} estimator offers excellent theoretical guarantees but is well-known to produce biased estimates. To address this, cite{Javanmard2014} introduced a method to “debias” the textsc{Lasso} estimates for a random sub-Gaussian sensing matrix $boldsymbol{A}$. Their approach relies on computing an “approximate…

  • Contextual Topic Modelling in Chinese Corpora with KeyNMF

    Contextual Topic Modelling in Chinese Corpora with KeyNMF A comprehensive guide on getting the most out of your Chinese topic models, from preprocessing to interpretation. With our recent paper on discourse dynamics in European Chinese diaspora media, our team has tapped into an almost unanimous frustration with the quality of topic modelling approaches when applied…

  • Matrix Completion via Residual Spectral Matching

    Matrix Completion via Residual Spectral Matching arXiv:2412.10005v1 Announce Type: new Abstract: Noisy matrix completion has attracted significant attention due to its applications in recommendation systems, signal processing and image restoration. Most existing works rely on (weighted) least squares methods under various low-rank constraints. However, minimizing the sum of squared residuals is not always efficient, as…

  • How to Interpret Matrix Expressions — Transformations

    How to Interpret Matrix Expressions — Transformations Matrix algebra for a data scientist Photo by Ben Allan on Unsplash This article begins a series for anyone who finds matrix algebra overwhelming. My goal is to turn what you’re afraid of into what you’re fascinated by. You’ll find it especially helpful if you want to understand machine learning concepts…

  • Deep Matrix Factorization with Adaptive Weights for Multi-View Clustering

    Deep Matrix Factorization with Adaptive Weights for Multi-View Clustering arXiv:2412.02292v1 Announce Type: new Abstract: Recently, deep matrix factorization has been established as a powerful model for unsupervised tasks, achieving promising results, especially for multi-view clustering. However, existing methods often lack effective feature selection mechanisms and rely on empirical hyperparameter selection. To address these issues, we…