mailitics

Category: cond-mat.dis-nn

  • Topological Exploration of High-Dimensional Empirical Risk Landscapes: general approach, and applications to phase retrieval

    Topological Exploration of High-Dimensional Empirical Risk Landscapes: general approach, and applications to phase retrieval arXiv:2602.17779v1 Announce Type: new Abstract: We consider the landscape of empirical risk minimization for high-dimensional Gaussian single-index models (generalized linear models). The objective is to recover an unknown signal $boldsymbol{theta}^star in mathbb{R}^d$ (where $d gg 1$) from a loss function $hat{R}(boldsymbol{theta})$…

  • Deep networks learn to parse uniform-depth context-free languages from local statistics

    Deep networks learn to parse uniform-depth context-free languages from local statistics arXiv:2602.06065v1 Announce Type: new Abstract: Understanding how the structure of language can be learned from sentences alone is a central question in both cognitive science and machine learning. Studies of the internal representations of Large Language Models (LLMs) support their ability to parse text…

  • High-Dimensional Limit of Stochastic Gradient Flow via Dynamical Mean-Field Theory

    High-Dimensional Limit of Stochastic Gradient Flow via Dynamical Mean-Field Theory arXiv:2602.06320v1 Announce Type: new Abstract: Modern machine learning models are typically trained via multi-pass stochastic gradient descent (SGD) with small batch sizes, and understanding their dynamics in high dimensions is of great interest. However, an analytical framework for describing the high-dimensional asymptotic behavior of multi-pass…

  • Parametric RDT approach to computational gap of symmetric binary perceptron

    Parametric RDT approach to computational gap of symmetric binary perceptron arXiv:2601.10628v1 Announce Type: new Abstract: We study potential presence of statistical-computational gaps (SCG) in symmetric binary perceptrons (SBP) via a parametric utilization of emph{fully lifted random duality theory} (fl-RDT) [96]. A structural change from decreasingly to arbitrarily ordered $c$-sequence (a key fl-RDT parametric component) is…

  • Perfect reconstruction of sparse signals using nonconvexity control and one-step RSB message passing

    Perfect reconstruction of sparse signals using nonconvexity control and one-step RSB message passing arXiv:2512.17426v1 Announce Type: new Abstract: We consider sparse signal reconstruction via minimization of the smoothly clipped absolute deviation (SCAD) penalty, and develop one-step replica-symmetry-breaking (1RSB) extensions of approximate message passing (AMP), termed 1RSB-AMP. Starting from the 1RSB formulation of belief propagation, we…

  • PCA recovery thresholds in low-rank matrix inference with sparse noise

    PCA recovery thresholds in low-rank matrix inference with sparse noise arXiv:2511.11927v1 Announce Type: new Abstract: We study the high-dimensional inference of a rank-one signal corrupted by sparse noise. The noise is modelled as the adjacency matrix of a weighted undirected graph with finite average connectivity in the large size limit. Using the replica method from…

  • Precise asymptotic analysis of Sobolev training for random feature models

    Precise asymptotic analysis of Sobolev training for random feature models arXiv:2511.03050v1 Announce Type: new Abstract: Gradient information is widely useful and available in applications, and is therefore natural to include in the training of neural networks. Yet little is known theoretically about the impact of Sobolev training — regression with both function and gradient data…

  • Graphical model for tensor factorization by sparse sampling

    Graphical model for tensor factorization by sparse sampling arXiv:2510.17886v1 Announce Type: new Abstract: We consider tensor factorizations based on sparse measurements of the tensor components. The measurements are designed in a way that the underlying graph of interactions is a random graph. The setup will be useful in cases where a substantial amount of data…

  • A universal compression theory: Lottery ticket hypothesis and superpolynomial scaling laws

    A universal compression theory: Lottery ticket hypothesis and superpolynomial scaling laws arXiv:2510.00504v1 Announce Type: new Abstract: When training large-scale models, the performance typically scales with the number of parameters and the dataset size according to a slow power law. A fundamental theoretical and practical question is whether comparable performance can be achieved with significantly smaller…

  • Proof of a perfect platonic representation hypothesis

    Proof of a perfect platonic representation hypothesis arXiv:2507.01098v1 Announce Type: cross Abstract: In this note, we elaborate on and explain in detail the proof given by Ziyin et al. (2025) of the “perfect” Platonic Representation Hypothesis (PRH) for the embedded deep linear network model (EDLN). We show that if trained with SGD, two EDLNs with…

  • Rare dense solutions clusters in asymmetric binary perceptrons — local entropy via fully lifted RDT

    Rare dense solutions clusters in asymmetric binary perceptrons — local entropy via fully lifted RDT arXiv:2506.19276v1 Announce Type: new Abstract: We study classical asymmetric binary perceptron (ABP) and associated emph{local entropy} (LE) as potential source of its algorithmic hardness. Isolation of emph{typical} ABP solutions in SAT phase seemingly suggests a universal algorithmic hardness. Paradoxically, efficient…

  • On the existence of consistent adversarial attacks in high-dimensional linear classification

    On the existence of consistent adversarial attacks in high-dimensional linear classification arXiv:2506.12454v1 Announce Type: new Abstract: What fundamentally distinguishes an adversarial attack from a misclassification due to limited model expressivity or finite data? In this work, we investigate this question in the setting of high-dimensional binary classification, where statistical effects due to limited data availability…

  • Improved Inference of Inverse Ising Problems under Missing Observations in Restricted Boltzmann Machines

    Improved Inference of Inverse Ising Problems under Missing Observations in Restricted Boltzmann Machines arXiv:2504.05643v1 Announce Type: new Abstract: Restricted Boltzmann machines (RBMs) are energy-based models analogous to the Ising model and are widely applied in statistical machine learning. The standard inverse Ising problem with a complete dataset requires computing both data and model expectations and…

  • Feature learning from non-Gaussian inputs: the case of Independent Component Analysis in high dimensions

    Feature learning from non-Gaussian inputs: the case of Independent Component Analysis in high dimensions arXiv:2503.23896v1 Announce Type: new Abstract: Deep neural networks learn structured features from complex, non-Gaussian inputs, but the mechanisms behind this process remain poorly understood. Our work is motivated by the observation that the first-layer filters learnt by deep convolutional neural networks…

  • Communities in the Kuramoto Model: Dynamics and Detection via Path Signatures

    Communities in the Kuramoto Model: Dynamics and Detection via Path Signatures arXiv:2503.17546v1 Announce Type: new Abstract: The behavior of multivariate dynamical processes is often governed by underlying structural connections that relate the components of the system. For example, brain activity which is often measured via time series is determined by an underlying structural graph, where…

  • Applications of Statistical Field Theory in Deep Learning

    Applications of Statistical Field Theory in Deep Learning arXiv:2502.18553v1 Announce Type: new Abstract: Deep learning algorithms have made incredible strides in the past decade yet due to the complexity of these algorithms, the science of deep learning remains in its early stages. Being an experimentally driven field, it is natural to seek a theory of…