mailitics

Tag: optimization

  • Contextual Distributionally Robust Optimization with Causal and Continuous Structure: An Interpretable and Tractable Approach

    Contextual Distributionally Robust Optimization with Causal and Continuous Structure: An Interpretable and Tractable Approach arXiv:2601.11016v1 Announce Type: new Abstract: In this paper, we introduce a framework for contextual distributionally robust optimization (DRO) that considers the causal and continuous structure of the underlying distribution by developing interpretable and tractable decision rules that prescribe decisions using covariates.…

  • Automatic Prompt Optimization for Multimodal Vision Agents: A Self-Driving Car Example

    Automatic Prompt Optimization for Multimodal Vision Agents: A Self-Driving Car Example Walkthrough using open-source prompt optimization algorithms in Python to improve the accuracy of an autonomous vehicle car safety agent running on OpenAI’s GPT 5.2 The post Automatic Prompt Optimization for Multimodal Vision Agents: A Self-Driving Car Example appeared first on Towards Data Science. Vincent Koc Go to…

  • Agentic AI Swarm Optimization using Artificial Bee Colonization (ABC)

    Agentic AI Swarm Optimization using Artificial Bee Colonization (ABC) Using Agentic AI prompts with the Artificial Bee Colony algorithm to enhance unsupervised clustering and optimization workflows. The post Agentic AI Swarm Optimization using Artificial Bee Colonization (ABC) appeared first on Towards Data Science. Gal Arav Go to original source

  • The Interplay of Statistics and Noisy Optimization: Learning Linear Predictors with Random Data Weights

    The Interplay of Statistics and Noisy Optimization: Learning Linear Predictors with Random Data Weights arXiv:2512.10188v1 Announce Type: new Abstract: We analyze gradient descent with randomly weighted data points in a linear regression model, under a generic weighting distribution. This includes various forms of stochastic gradient descent, importance sampling, but also extends to weighting distributions with…

  • Bayesian Optimization for Function-Valued Responses under Min-Max Criteria

    Bayesian Optimization for Function-Valued Responses under Min-Max Criteria arXiv:2512.07868v1 Announce Type: cross Abstract: Bayesian optimization is widely used for optimizing expensive black box functions, but most existing approaches focus on scalar responses. In many scientific and engineering settings the response is functional, varying smoothly over an index such as time or wavelength, which makes classical…

  • Contextual Strongly Convex Simulation Optimization: Optimize then Predict with Inexact Solutions

    Contextual Strongly Convex Simulation Optimization: Optimize then Predict with Inexact Solutions arXiv:2512.06270v1 Announce Type: new Abstract: In this work, we study contextual strongly convex simulation optimization and adopt an “optimize then predict” (OTP) approach for real-time decision making. In the offline stage, simulation optimization is conducted across a set of covariates to approximate the optimal-solution…

  • SOCRATES: Simulation Optimization with Correlated Replicas and Adaptive Trajectory Evaluations

    SOCRATES: Simulation Optimization with Correlated Replicas and Adaptive Trajectory Evaluations arXiv:2511.00685v1 Announce Type: new Abstract: The field of simulation optimization (SO) encompasses various methods developed to optimize complex, expensive-to-sample stochastic systems. Established methods include, but are not limited to, ranking-and-selection for finite alternatives and surrogate-based methods for continuous domains, with broad applications in engineering and…

  • Generative Bayesian Optimization: Generative Models as Acquisition Functions

    Generative Bayesian Optimization: Generative Models as Acquisition Functions arXiv:2510.25240v1 Announce Type: new Abstract: We present a general strategy for turning generative models into candidate solution samplers for batch Bayesian optimization (BO). The use of generative models for BO enables large batch scaling as generative sampling, optimization of non-continuous design spaces, and high-dimensional and combinatorial design.…

  • Beating the Winner’s Curse via Inference-Aware Policy Optimization

    Beating the Winner’s Curse via Inference-Aware Policy Optimization arXiv:2510.18161v1 Announce Type: new Abstract: There has been a surge of recent interest in automatically learning policies to target treatment decisions based on rich individual covariates. A common approach is to train a machine learning model to predict counterfactual outcomes, and then select the policy that optimizes…

  • From Data to Rewards: a Bilevel Optimization Perspective on Maximum Likelihood Estimation

    From Data to Rewards: a Bilevel Optimization Perspective on Maximum Likelihood Estimation arXiv:2510.07624v1 Announce Type: new Abstract: Generative models form the backbone of modern machine learning, underpinning state-of-the-art systems in text, vision, and multimodal applications. While Maximum Likelihood Estimation has traditionally served as the dominant training paradigm, recent work have highlighted its limitations, particularly in…

  • Bilevel optimization for learning hyperparameters: Application to solving PDEs and inverse problems with Gaussian processes

    Bilevel optimization for learning hyperparameters: Application to solving PDEs and inverse problems with Gaussian processes arXiv:2510.05568v1 Announce Type: new Abstract: Methods for solving scientific computing and inference problems, such as kernel- and neural network-based approaches for partial differential equations (PDEs), inverse problems, and supervised learning tasks, depend crucially on the choice of hyperparameters. Specifically, the…

  • Quantile-Scaled Bayesian Optimization Using Rank-Only Feedback

    Quantile-Scaled Bayesian Optimization Using Rank-Only Feedback arXiv:2510.03277v1 Announce Type: new Abstract: Bayesian Optimization (BO) is widely used for optimizing expensive black-box functions, particularly in hyperparameter tuning. However, standard BO assumes access to precise objective values, which may be unavailable, noisy, or unreliable in real-world settings where only relative or rank-based feedback can be obtained. In…

  • Global Optimization of Stochastic Black-Box Functions with Arbitrary Noise Distributions using Wilson Score Kernel Density Estimation

    Global Optimization of Stochastic Black-Box Functions with Arbitrary Noise Distributions using Wilson Score Kernel Density Estimation arXiv:2509.09238v1 Announce Type: new Abstract: Many optimization problems in robotics involve the optimization of time-expensive black-box functions, such as those involving complex simulations or evaluation of real-world experiments. Furthermore, these functions are often stochastic as repeated experiments are subject…

  • A Dual Optimization View to Empirical Risk Minimization with f-Divergence Regularization

    A Dual Optimization View to Empirical Risk Minimization with f-Divergence Regularization arXiv:2508.03314v1 Announce Type: new Abstract: The dual formulation of empirical risk minimization with f-divergence regularization (ERM-fDR) is introduced. The solution of the dual optimization problem to the ERM-fDR is connected to the notion of normalization function introduced as an implicit function. This dual approach…

  • Efficient optimization of expensive black-box simulators via marginal means, with application to neutrino detector design

    Efficient optimization of expensive black-box simulators via marginal means, with application to neutrino detector design arXiv:2508.01834v1 Announce Type: new Abstract: With advances in scientific computing, computer experiments are increasingly used for optimizing complex systems. However, for modern applications, e.g., the optimization of nuclear physics detectors, each experiment run can require hundreds of CPU hours, making…

  • Bayesian preference elicitation for decision support in multiobjective optimization

    Bayesian preference elicitation for decision support in multiobjective optimization arXiv:2507.16999v1 Announce Type: new Abstract: We present a novel approach to help decision-makers efficiently identify preferred solutions from the Pareto set of a multi-objective optimization problem. Our method uses a Bayesian model to estimate the decision-maker’s utility function based on pairwise comparisons. Aided by this model,…

  • What Optimization Terminologies for Linear Programming Really Mean

    What Optimization Terminologies for Linear Programming Really Mean Understanding the duality of optimization problem, primal to dual conversion, and the optimality conditions for linear problems. The post What Optimization Terminologies for Linear Programming Really Mean appeared first on Towards Data Science. Himalaya Bir Shrestha Go to original source

  • Dynamic Inventory Optimization with Censored Demand

    Dynamic Inventory Optimization with Censored Demand A sequential decision framework with Bayesian learning The post Dynamic Inventory Optimization with Censored Demand appeared first on Towards Data Science. Mert Ersoz Go to original source

  • Zeroth-Order Optimization Finds Flat Minima

    Zeroth-Order Optimization Finds Flat Minima arXiv:2506.05454v1 Announce Type: cross Abstract: Zeroth-order methods are extensively used in machine learning applications where gradients are infeasible or expensive to compute, such as black-box attacks, reinforcement learning, and language model fine-tuning. Existing optimization theory focuses on convergence to an arbitrary stationary point, but less is known on the implicit…

  • Latent Guided Sampling for Combinatorial Optimization

    Latent Guided Sampling for Combinatorial Optimization arXiv:2506.03672v1 Announce Type: new Abstract: Combinatorial Optimization problems are widespread in domains such as logistics, manufacturing, and drug discovery, yet their NP-hard nature makes them computationally challenging. Recent Neural Combinatorial Optimization methods leverage deep learning to learn solution strategies, trained via Supervised or Reinforcement Learning (RL). While promising, these…

  • LLM Optimization: LoRA and QLoRA

    LLM Optimization: LoRA and QLoRA Scalable fine-tuning techniques for large language models The post LLM Optimization: LoRA and QLoRA appeared first on Towards Data Science. Vyacheslav Efimov Go to original source

  • Finite-Sample Convergence Bounds for Trust Region Policy Optimization in Mean-Field Games

    Finite-Sample Convergence Bounds for Trust Region Policy Optimization in Mean-Field Games arXiv:2505.22781v1 Announce Type: new Abstract: We introduce Mean-Field Trust Region Policy Optimization (MF-TRPO), a novel algorithm designed to compute approximate Nash equilibria for ergodic Mean-Field Games (MFG) in finite state-action spaces. Building on the well-established performance of TRPO in the reinforcement learning (RL) setting,…

  • Bayesian Optimization for Hyperparameter Tuning of Deep Learning Models

    Bayesian Optimization for Hyperparameter Tuning of Deep Learning Models Explore how Bayesian Optimization outperforms Grid Search in efficiency and performance over binary classification tasks. The post Bayesian Optimization for Hyperparameter Tuning of Deep Learning Models appeared first on Towards Data Science. Kuriko Iwai Go to original source

  • Gradient-based Sample Selection for Faster Bayesian Optimization

    Gradient-based Sample Selection for Faster Bayesian Optimization arXiv:2504.07742v1 Announce Type: new Abstract: Bayesian optimization (BO) is an effective technique for black-box optimization. However, its applicability is typically limited to moderate-budget problems due to the cubic complexity in computing the Gaussian process (GP) surrogate model. In large-budget scenarios, directly employing the standard GP model faces significant…

  • Batch Bayesian Optimization for High-Dimensional Experimental Design: Simulation and Visualization

    Batch Bayesian Optimization for High-Dimensional Experimental Design: Simulation and Visualization arXiv:2504.03943v1 Announce Type: new Abstract: Bayesian Optimization (BO) is increasingly used to guide experimental optimization tasks. To elucidate BO behavior in noisy and high-dimensional settings typical for materials science applications, we perform batch BO of two six-dimensional test functions: an Ackley function representing a needle-in-a-haystack…

  • Bayesian Optimization for Robust Identification of Ornstein-Uhlenbeck Model

    Bayesian Optimization for Robust Identification of Ornstein-Uhlenbeck Model arXiv:2503.06381v1 Announce Type: new Abstract: This paper deals with the identification of the stochastic Ornstein-Uhlenbeck (OU) process error model, which is characterized by an inverse time constant, and the unknown variances of the process and observation noises. Although the availability of the explicit expression of the log-likelihood…

  • Gradient-free stochastic optimization for additive models

    Gradient-free stochastic optimization for additive models arXiv:2503.02131v1 Announce Type: new Abstract: We address the problem of zero-order optimization from noisy observations for an objective function satisfying the Polyak-{L}ojasiewicz or the strong convexity condition. Additionally, we assume that the objective function has an additive structure and satisfies a higher-order smoothness property, characterized by the H”older family…

  • New Lower Bounds for Stochastic Non-Convex Optimization through Divergence Composition

    New Lower Bounds for Stochastic Non-Convex Optimization through Divergence Composition arXiv:2502.14060v1 Announce Type: new Abstract: We study fundamental limits of first-order stochastic optimization in a range of nonconvex settings, including L-smooth functions satisfying Quasar-Convexity (QC), Quadratic Growth (QG), and Restricted Secant Inequalities (RSI). While the convergence properties of standard algorithms are well-understood in deterministic regimes,…

  • Multi-Objective Bayesian Optimization for Networked Black-Box Systems: A Path to Greener Profits and Smarter Designs

    Multi-Objective Bayesian Optimization for Networked Black-Box Systems: A Path to Greener Profits and Smarter Designs arXiv:2502.14121v1 Announce Type: new Abstract: Designing modern industrial systems requires balancing several competing objectives, such as profitability, resilience, and sustainability, while accounting for complex interactions between technological, economic, and environmental factors. Multi-objective optimization (MOO) methods are commonly used to navigate…

  • Introduction to Minimum Cost Flow Optimization in Python

    Introduction to Minimum Cost Flow Optimization in Python Minimum cost flow optimization minimizes the cost of moving flow through a network of nodes and edges. Nodes include sources (supply) and sinks (demand), with different costs and capacity limits. The aim is to find the least costly way to move volume from sources to sinks while…

  • Exploring New Hyperparameter Dimensions with Laplace Approximated Bayesian Optimization

    Exploring New Hyperparameter Dimensions with Laplace Approximated Bayesian Optimization Is it better than grid search? Image by author from canva When I notice my model is overfitting, I often think, “It is time to regularize”. But how do I decide which regularization method to use (L1, L2) and what parameters to choose? Typically, I perform hyperparameter optimization…

  • Distributionally Robust Optimization via Iterative Algorithms in Continuous Probability Spaces

    Distributionally Robust Optimization via Iterative Algorithms in Continuous Probability Spaces arXiv:2412.20556v1 Announce Type: new Abstract: We consider a minimax problem motivated by distributionally robust optimization (DRO) when the worst-case distribution is continuous, leading to significant computational challenges due to the infinite-dimensional nature of the optimization problem. Recent research has explored learning the worst-case distribution using…

  • BOIDS: High-dimensional Bayesian Optimization via Incumbent-guided Direction Lines and Subspace Embeddings

    BOIDS: High-dimensional Bayesian Optimization via Incumbent-guided Direction Lines and Subspace Embeddings arXiv:2412.12918v1 Announce Type: new Abstract: When it comes to expensive black-box optimization problems, Bayesian Optimization (BO) is a well-known and powerful solution. Many real-world applications involve a large number of dimensions, hence scaling BO to high dimension is of much interest. However, state-of-the-art high-dimensional…

  • Optimization Can Learn Johnson Lindenstrauss Embeddings

    Optimization Can Learn Johnson Lindenstrauss Embeddings arXiv:2412.07242v1 Announce Type: new Abstract: Embeddings play a pivotal role across various disciplines, offering compact representations of complex data structures. Randomized methods like Johnson-Lindenstrauss (JL) provide state-of-the-art and essentially unimprovable theoretical guarantees for achieving such representations. These guarantees are worst-case and in particular, neither the analysis, nor the algorithm,…

  • Pathwise optimization for bridge-type estimators and its applications

    Pathwise optimization for bridge-type estimators and its applications arXiv:2412.04047v1 Announce Type: new Abstract: Sparse parametric models are of great interest in statistical learning and are often analyzed by means of regularized estimators. Pathwise methods allow to efficiently compute the full solution path for penalized estimators, for any possible value of the penalization parameter $lambda$. In…