mailitics

Tag: descent

  • The Machine Learning “Advent Calendar” Bonus 2: Gradient Descent Variants in Excel

    The Machine Learning “Advent Calendar” Bonus 2: Gradient Descent Variants in Excel Gradient Descent, Momentum, RMSProp, and Adam all aim for the same minimum. They do not change the destination, only the path. Each method adds a mechanism that fixes a limitation of the previous one, making the movement faster, more stable, or more adaptive.…

  • Enhanced Cyclic Coordinate Descent Methods for Elastic Net Penalized Linear Models

    Enhanced Cyclic Coordinate Descent Methods for Elastic Net Penalized Linear Models arXiv:2510.19999v1 Announce Type: new Abstract: We present a novel enhanced cyclic coordinate descent (ECCD) framework for solving generalized linear models with elastic net constraints that reduces training time in comparison to existing state-of-the-art methods. We redesign the CD method by performing a Taylor expansion…

  • Bayesian Double Descent

    Bayesian Double Descent arXiv:2507.07338v1 Announce Type: new Abstract: Double descent is a phenomenon of over-parameterized statistical models. Our goal is to view double descent from a Bayesian perspective. Over-parameterized models such as deep neural networks have an interesting re-descending property in their risk characteristics. This is a recent phenomenon in machine learning and has been…

  • Prototyping Gradient Descent in Machine Learning

    Prototyping Gradient Descent in Machine Learning Mathematical theorem and credit transaction prediction using Stochastic / Batch GD The post Prototyping Gradient Descent in Machine Learning appeared first on Towards Data Science. Kuriko Iwai Go to original source

  • A stochastic gradient descent algorithm with random search directions

    A stochastic gradient descent algorithm with random search directions arXiv:2503.19942v1 Announce Type: new Abstract: Stochastic coordinate descent algorithms are efficient methods in which each iterate is obtained by fixing most coordinates at their values from the current iteration, and approximately minimizing the objective with respect to the remaining coordinates. However, this approach is usually restricted…

  • Optimizing ML Training with Metagradient Descent

    Optimizing ML Training with Metagradient Descent arXiv:2503.13751v1 Announce Type: new Abstract: A major challenge in training large-scale machine learning models is configuring the training process to maximize model performance, i.e., finding the best training setup from a vast design space. In this work, we unlock a gradient-based approach to this problem. We first introduce an…