mailitics

Tag: regularization

  • A Regularization-Sharpness Tradeoff for Linear Interpolators

    A Regularization-Sharpness Tradeoff for Linear Interpolators arXiv:2602.12680v1 Announce Type: new Abstract: The rule of thumb regarding the relationship between the bias-variance tradeoff and model size plays a key role in classical machine learning, but is now well-known to break down in the overparameterized setting as per the double descent curve. In particular, minimum-norm interpolating estimators…

  • Transcendental Regularization of Finite Mixtures:Theoretical Guarantees and Practical Limitations

    Transcendental Regularization of Finite Mixtures:Theoretical Guarantees and Practical Limitations arXiv:2602.03889v1 Announce Type: new Abstract: Finite mixture models are widely used for unsupervised learning, but maximum likelihood estimation via EM suffers from degeneracy as components collapse. We introduce transcendental regularization, a penalized likelihood framework with analytic barrier functions that prevent degeneracy while maintaining asymptotic efficiency. The…

  • On the Effect of Regularization on Nonparametric Mean-Variance Regression

    On the Effect of Regularization on Nonparametric Mean-Variance Regression arXiv:2511.22004v1 Announce Type: new Abstract: Uncertainty quantification is vital for decision-making and risk assessment in machine learning. Mean-variance regression models, which predict both a mean and residual noise for each data point, provide a simple approach to uncertainty quantification. However, overparameterized mean-variance models struggle with signal-to-noise…

  • Optimization and Regularization Under Arbitrary Objectives

    Optimization and Regularization Under Arbitrary Objectives arXiv:2511.19628v1 Announce Type: new Abstract: This study investigates the limitations of applying Markov Chain Monte Carlo (MCMC) methods to arbitrary objective functions, focusing on a two-block MCMC framework which alternates between Metropolis-Hastings and Gibbs sampling. While such approaches are often considered advantageous for enabling data-driven regularization, we show that…

  • Decreasing Entropic Regularization Averaged Gradient for Semi-Discrete Optimal Transport

    Decreasing Entropic Regularization Averaged Gradient for Semi-Discrete Optimal Transport arXiv:2510.27340v1 Announce Type: new Abstract: Adding entropic regularization to Optimal Transport (OT) problems has become a standard approach for designing efficient and scalable solvers. However, regularization introduces a bias from the true solution. To mitigate this bias while still benefiting from the acceleration provided by regularization,…

  • Learning Pareto manifolds in high dimensions: How can regularization help?

    Learning Pareto manifolds in high dimensions: How can regularization help? arXiv:2503.08849v1 Announce Type: new Abstract: Simultaneously addressing multiple objectives is becoming increasingly important in modern machine learning. At the same time, data is often high-dimensional and costly to label. For a single objective such as prediction risk, conventional regularization techniques are known to improve generalization…

  • Quantifying Overfitting along the Regularization Path for Two-Part-Code MDL in Supervised Classification

    Quantifying Overfitting along the Regularization Path for Two-Part-Code MDL in Supervised Classification arXiv:2503.02110v1 Announce Type: new Abstract: We provide a complete characterization of the entire regularization curve of a modified two-part-code Minimum Description Length (MDL) learning rule for binary classification, based on an arbitrary prior or description language. citet{GL} previously established the lack of asymptotic…

  • Fr’echet regression for multi-label feature selection with implicit regularization

    Fr’echet regression for multi-label feature selection with implicit regularization arXiv:2412.18247v1 Announce Type: new Abstract: Fr’echet regression extends linear regression to model complex responses in metric spaces, making it particularly relevant for multi-label regression, where each instance can have multiple associated labels. However, variable selection within this framework remains underexplored. In this paper, we pro pose…

  • Adversarially robust generalization theory via Jacobian regularization for deep neural networks

    Adversarially robust generalization theory via Jacobian regularization for deep neural networks arXiv:2412.12449v1 Announce Type: new Abstract: Powerful deep neural networks are vulnerable to adversarial attacks. To obtain adversarially robust models, researchers have separately developed adversarial training and Jacobian regularization techniques. There are abundant theoretical and empirical studies for adversarial training, but theoretical foundations for Jacobian…