mailitics

Category: aimldsaimlds

  • Apply Sphinx’s Functionality to Create Documentation for Your Next Data Science Project

    Apply Sphinx’s Functionality to Create Documentation for Your Next Data Science Project Three cases to use the Sphinx tool as a pro The post Apply Sphinx’s Functionality to Create Documentation for Your Next Data Science Project appeared first on Towards Data Science. Radmila Mandzhieva Go to original source

  • Temporal cross-validation impacts multivariate time series subsequence anomaly detection evaluation

    Temporal cross-validation impacts multivariate time series subsequence anomaly detection evaluation arXiv:2506.12183v1 Announce Type: new Abstract: Evaluating anomaly detection in multivariate time series (MTS) requires careful consideration of temporal dependencies, particularly when detecting subsequence anomalies common in fault detection scenarios. While time series cross-validation (TSCV) techniques aim to preserve temporal ordering during model evaluation, their impact…

  • Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory

    Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory arXiv:2506.12350v1 Announce Type: new Abstract: Despite its empirical success, Reinforcement Learning from Human Feedback (RLHF) has been shown to violate almost all the fundamental axioms in social choice theory — such as majority consistency, pairwise majority consistency, and Condorcet consistency. This raises…

  • A Transfer Learning Framework for Multilayer Networks via Model Averaging

    A Transfer Learning Framework for Multilayer Networks via Model Averaging arXiv:2506.12455v1 Announce Type: new Abstract: Link prediction in multilayer networks is a key challenge in applications such as recommendation systems and protein-protein interaction prediction. While many techniques have been developed, most rely on assumptions about shared structures and require access to raw auxiliary data, limiting…

  • On the existence of consistent adversarial attacks in high-dimensional linear classification

    On the existence of consistent adversarial attacks in high-dimensional linear classification arXiv:2506.12454v1 Announce Type: new Abstract: What fundamentally distinguishes an adversarial attack from a misclassification due to limited model expressivity or finite data? In this work, we investigate this question in the setting of high-dimensional binary classification, where statistical effects due to limited data availability…

  • Dependent Randomized Rounding for Budget Constrained Experimental Design

    Dependent Randomized Rounding for Budget Constrained Experimental Design arXiv:2506.12677v1 Announce Type: new Abstract: Policymakers in resource-constrained settings require experimental designs that satisfy strict budget limits while ensuring precise estimation of treatment effects. We propose a framework that applies a dependent randomized rounding procedure to convert assignment probabilities into binary treatment decisions. Our proposed solution preserves…

  • A Practical Starters’ Guide to Causal Structure Learning with Bayesian Methods in Python

    A Practical Starters’ Guide to Causal Structure Learning with Bayesian Methods in Python Learn Causal Structures and make inferences with Bayesian Methods: Python Tutorial The post A Practical Starters’ Guide to Causal Structure Learning with Bayesian Methods in Python appeared first on Towards Data Science. Erdogan Taskesen Go to original source

  • Grad-CAM from Scratch with PyTorch Hooks

    Grad-CAM from Scratch with PyTorch Hooks A hands-on look at an explainable AI (XAI) technique that helps reveal why a convolutional neural network (CNN) made a particular decision The post Grad-CAM from Scratch with PyTorch Hooks appeared first on Towards Data Science. Conor O’Sullivan Go to original source

  • Let’s Analyze OpenAI’s Claims About ChatGPT Energy Use

    Let’s Analyze OpenAI’s Claims About ChatGPT Energy Use ChatGPT uses an average of 0.34 Wh per query, according to a blog post by Sam Altman. Does that figure hold up? The post Let’s Analyze OpenAI’s Claims About ChatGPT Energy Use appeared first on Towards Data Science. Kasper Groes Albin Ludvigsen Go to original source

  • Regularisation: A Deep Dive into Theory, Implementation, and Practical Insights

    Regularisation: A Deep Dive into Theory, Implementation, and Practical Insights A detailed guide on controlling overfitting and increasing the stability of your models. The post Regularisation: A Deep Dive into Theory, Implementation, and Practical Insights appeared first on Towards Data Science. Sourav Mohile Go to original source

  • Build an AI Agent to Explore Your Data Catalog with Natural Language

    Build an AI Agent to Explore Your Data Catalog with Natural Language Leverage LLMs to query your Databricks Data Catalog The post Build an AI Agent to Explore Your Data Catalog with Natural Language appeared first on Towards Data Science. Fabiana Clemente Go to original source

  • A Framework for Non-Linear Attention via Modern Hopfield Networks

    A Framework for Non-Linear Attention via Modern Hopfield Networks arXiv:2506.11043v1 Announce Type: new Abstract: In this work we propose an energy functional along the lines of Modern Hopfield Networks (MNH), the stationary points of which correspond to the attention due to Vaswani et al. [12], thus unifying both frameworks. The minima of this landscape form…

  • Fast Bayesian Optimization of Function Networks with Partial Evaluations

    Fast Bayesian Optimization of Function Networks with Partial Evaluations arXiv:2506.11456v1 Announce Type: new Abstract: Bayesian optimization of function networks (BOFN) is a framework for optimizing expensive-to-evaluate objective functions structured as networks, where some nodes’ outputs serve as inputs for others. Many real-world applications, such as manufacturing and drug discovery, involve function networks with additional properties…

  • Collaborative Prediction: To Join or To Disjoin Datasets

    Collaborative Prediction: To Join or To Disjoin Datasets arXiv:2506.11271v1 Announce Type: new Abstract: With the recent rise of generative Artificial Intelligence (AI), the need of selecting high-quality dataset to improve machine learning models has garnered increasing attention. However, some part of this topic remains underexplored, even for simple prediction models. In this work, we study…

  • On the performance of multi-fidelity and reduced-dimensional neural emulators for inference of physiologic boundary conditions

    On the performance of multi-fidelity and reduced-dimensional neural emulators for inference of physiologic boundary conditions arXiv:2506.11683v1 Announce Type: new Abstract: Solving inverse problems in cardiovascular modeling is particularly challenging due to the high computational cost of running high-fidelity simulations. In this work, we focus on Bayesian parameter estimation and explore different methods to reduce the…

  • Using Deep Operators to Create Spatio-temporal Surrogates for Dynamical Systems under Uncertainty

    Using Deep Operators to Create Spatio-temporal Surrogates for Dynamical Systems under Uncertainty arXiv:2506.11761v1 Announce Type: new Abstract: Spatio-temporal data, which consists of responses or measurements gathered at different times and positions, is ubiquitous across diverse applications of civil infrastructure. While SciML methods have made significant progress in tackling the issue of response prediction for individual…

  • Weekly Entering & Transitioning – Thread 16 Jun, 2025 – 23 Jun, 2025

    Weekly Entering & Transitioning – Thread 16 Jun, 2025 – 23 Jun, 2025 Welcome to this week’s entering & transitioning thread! This thread is for any questions about getting started, studying, or transitioning into the data science field. Topics include: Learning resources (e.g. books, tutorials, videos) Traditional education (e.g. schools, degrees, electives) Alternative education (e.g.…

  • Don’t be the data scientist who’s in love with models, be the one who solves real problems

    Don’t be the data scientist who’s in love with models, be the one who solves real problems work at a company with around 100 data scientists, ML and data engineers. The most frustrating part of working with many data scientists and honestly, I see this on this sub all the time too, is how obsessed…

  • Books on applied data science for B2B marketing?

    Books on applied data science for B2B marketing? There’s this thread from 3 years ago: https://www.reddit.com/r/datascience/comments/ram75g/books_on_applied_data_science_for_b2b_marketing/ Unfortunately, it never got any book recommendations – I’m in pretty much the exact same position as the OP of the linked thread and am looking for resources that explain the best methods and provide practical how-tos for marketing…

  • “Data Annotation” spam

    “Data Annotation” spam Anyone else’s job search site just absolutely spammed by Data Annotation? If I look up Data, ML, AI, or anything similar in my area I get 2-3 pages of there job posting. submitted by /u/MahaloMerky [link] [comments] /u/MahaloMerky Go to original source

  • Significant humor

    Significant humor Saw this and found it hilarious , thought I’d share it here as this is one of the few places this joke might actually land. Datetime.now() + timedelta(days=4) submitted by /u/MamboAsher [link] [comments] /u/MamboAsher Go to original source

  • Stop Building AI Platforms

    Stop Building AI Platforms When small and medium companies achieve success in building Data and ML platforms, building AI platforms is now profoundly challenging The post Stop Building AI Platforms appeared first on Towards Data Science. Ming Gao Go to original source

  • What If I had AI in 2018: Rent the Runway Fulfillment Center Optimization

    What If I had AI in 2018: Rent the Runway Fulfillment Center Optimization An LLM in 2018 would not have trivialized a complex project, although it could have enhanced the final solution The post What If I had AI in 2018: Rent the Runway Fulfillment Center Optimization appeared first on Towards Data Science. Hugo Ducruc…

  • AI Is Not a Black Box (Relatively Speaking)

    AI Is Not a Black Box (Relatively Speaking) Compared to the opacity around human intelligence, AI is more transparent in some very tangible ways. The post AI Is Not a Black Box (Relatively Speaking) appeared first on Towards Data Science. Piotr (Peter) Mardziel Go to original source

  • How AI Agents “Talk” to Each Other

    How AI Agents “Talk” to Each Other Minimize chaos and maintain inter-agent harmony in your projects The post How AI Agents “Talk” to Each Other appeared first on Towards Data Science. TDS Editors Go to original source

  • Fundamental Limits of Learning High-dimensional Simplices in Noisy Regimes

    Fundamental Limits of Learning High-dimensional Simplices in Noisy Regimes arXiv:2506.10101v1 Announce Type: new Abstract: In this paper, we establish sample complexity bounds for learning high-dimensional simplices in $mathbb{R}^K$ from noisy data. Specifically, we consider $n$ i.i.d. samples uniformly drawn from an unknown simplex in $mathbb{R}^K$, each corrupted by additive Gaussian noise of unknown variance. We…

  • Momentum Multi-Marginal Schr”odinger Bridge Matching

    Momentum Multi-Marginal Schr”odinger Bridge Matching arXiv:2506.10168v1 Announce Type: new Abstract: Understanding complex systems by inferring trajectories from sparse sample snapshots is a fundamental challenge in a wide range of domains, e.g., single-cell biology, meteorology, and economics. Despite advancements in Bridge and Flow matching frameworks, current methodologies rely on pairwise interpolation between adjacent snapshots. This hinders…

  • Measuring Semantic Information Production in Generative Diffusion Models

    Measuring Semantic Information Production in Generative Diffusion Models arXiv:2506.10433v1 Announce Type: new Abstract: It is well known that semantic and structural features of the generated images emerge at different times during the reverse dynamics of diffusion, a phenomenon that has been connected to physical phase transitions in magnets and other materials. In this paper, we…

  • Distributionally-Constrained Adversaries in Online Learning

    Distributionally-Constrained Adversaries in Online Learning arXiv:2506.10293v1 Announce Type: new Abstract: There has been much recent interest in understanding the continuum from adversarial to stochastic settings in online learning, with various frameworks including smoothed settings proposed to bridge this gap. We consider the more general and flexible framework of distributionally constrained adversaries in which instances are…

  • Box-Constrained Softmax Function and Its Application for Post-Hoc Calibration

    Box-Constrained Softmax Function and Its Application for Post-Hoc Calibration arXiv:2506.10572v1 Announce Type: new Abstract: Controlling the output probabilities of softmax-based models is a common problem in modern machine learning. Although the $mathrm{Softmax}$ function provides soft control via its temperature parameter, it lacks the ability to enforce hard constraints, such as box constraints, on output probabilities,…

  • Connecting the Dots for Better Movie Recommendations

    Connecting the Dots for Better Movie Recommendations Connecting the Dots for Better Movie Recommendations: Lightweight graph RAG on Rotten Tomatoes movie reviews The post Connecting the Dots for Better Movie Recommendations appeared first on Towards Data Science. Brian Godsey Go to original source

  • Agentic AI 103: Building Multi-Agent Teams

    Agentic AI 103: Building Multi-Agent Teams Build multi-agent teams that can automate tasks and enhance productivity. The post Agentic AI 103: Building Multi-Agent Teams appeared first on Towards Data Science. Gustavo Santos Go to original source

  • Design Smarter Prompts and Boost Your LLM Output: Real Tricks from an AI Engineer’s Toolbox

    Design Smarter Prompts and Boost Your LLM Output: Real Tricks from an AI Engineer’s Toolbox Not just what you ask, but how you ask it. Practical techniques for prompt engineering that deliver The post Design Smarter Prompts and Boost Your LLM Output: Real Tricks from an AI Engineer’s Toolbox appeared first on Towards Data Science. Ugo Pradère…

  • User Authorisation in Streamlit With OIDC and Google

    User Authorisation in Streamlit With OIDC and Google Log in to a Streamlit app with a Google email account The post User Authorisation in Streamlit With OIDC and Google appeared first on Towards Data Science. Thomas Reid Go to original source

  • Know What You Don’t Know: Uncertainty Calibration of Process Reward Models

    Know What You Don’t Know: Uncertainty Calibration of Process Reward Models arXiv:2506.09338v1 Announce Type: new Abstract: Process reward models (PRMs) play a central role in guiding inference-time scaling algorithms for large language models (LLMs). However, we observe that even state-of-the-art PRMs can be poorly calibrated and often overestimate success probabilities. To address this, we present…

  • Attention-Bayesian Hybrid Approach to Modular Multiple Particle Tracking

    Attention-Bayesian Hybrid Approach to Modular Multiple Particle Tracking arXiv:2506.09441v1 Announce Type: new Abstract: Tracking multiple particles in noisy and cluttered scenes remains challenging due to a combinatorial explosion of trajectory hypotheses, which scales super-exponentially with the number of particles and frames. The transformer architecture has shown a significant improvement in robustness against this high combinatorial…

  • Evasion Attacks Against Bayesian Predictive Models

    Evasion Attacks Against Bayesian Predictive Models arXiv:2506.09640v1 Announce Type: new Abstract: There is an increasing interest in analyzing the behavior of machine learning systems against adversarial attacks. However, most of the research in adversarial machine learning has focused on studying weaknesses against evasion or poisoning attacks to predictive models in classical setups, with the susceptibility…

  • LLM-Powered CPI Prediction Inference with Online Text Time Series

    LLM-Powered CPI Prediction Inference with Online Text Time Series arXiv:2506.09516v1 Announce Type: new Abstract: Forecasting the Consumer Price Index (CPI) is an important yet challenging task in economics, where most existing approaches rely on low-frequency, survey-based data. With the recent advances of large language models (LLMs), there is growing potential to leverage high-frequency online text…

  • Scaling Laws for Uncertainty in Deep Learning

    Scaling Laws for Uncertainty in Deep Learning arXiv:2506.09648v1 Announce Type: new Abstract: Deep learning has recently revealed the existence of scaling laws, demonstrating that model performance follows predictable trends based on dataset and model sizes. Inspired by these findings and fascinating phenomena emerging in the over-parameterized regime, we examine a parallel direction: do similar scaling…

  • Exploring the Proportional Odds Model for Ordinal Logistic Regression

    Exploring the Proportional Odds Model for Ordinal Logistic Regression Understanding and Implementing Brant’s Tests in Ordinal Logistic Regression with Python The post Exploring the Proportional Odds Model for Ordinal Logistic Regression appeared first on Towards Data Science. JUNIOR JUMBONG Go to original source

  • Can AI Truly Develop a Memory That Adapts Like Ours?

    Can AI Truly Develop a Memory That Adapts Like Ours? Exploring Titans: A new architecture equipping LLMs with human-inspired memory that learns and updates itself during test-time. The post Can AI Truly Develop a Memory That Adapts Like Ours? appeared first on Towards Data Science. Moulik Gupta Go to original source

  • Model Context Protocol (MCP) Tutorial: Build Your First MCP Server in 6 Steps

    Model Context Protocol (MCP) Tutorial: Build Your First MCP Server in 6 Steps A beginner-friendly tutorial of MCP architecture, with the focus on MCP server components and applications, guiding through the process of building a custom MCP server that enables code-to-diagram. The post Model Context Protocol (MCP) Tutorial: Build Your First MCP Server in 6…

  • Mobile App Development with Python

    Mobile App Development with Python Build iOS & Android Apps with Kivy The post Mobile App Development with Python appeared first on Towards Data Science. Mauro Di Pietro Go to original source

  • Physics-Informed Teleconnection-Aware Transformer for Global Subseasonal-to-Seasonal Forecasting

    Physics-Informed Teleconnection-Aware Transformer for Global Subseasonal-to-Seasonal Forecasting arXiv:2506.08049v1 Announce Type: new Abstract: Subseasonal-to-seasonal (S2S) forecasting, which predicts climate conditions from several weeks to months in advance, presents significant challenges due to the chaotic dynamics of atmospheric systems and complex interactions across multiple scales. Current approaches often fail to explicitly model underlying physical processes and teleconnections…

  • Constrained Pareto Set Identification with Bandit Feedback

    Constrained Pareto Set Identification with Bandit Feedback arXiv:2506.08127v1 Announce Type: new Abstract: In this paper, we address the problem of identifying the Pareto Set under feasibility constraints in a multivariate bandit setting. Specifically, given a $K$-armed bandit with unknown means $mu_1, dots, mu_K in mathbb{R}^d$, the goal is to identify the set of arms whose…

  • WWAggr: A Window Wasserstein-based Aggregation for Ensemble Change Point Detection

    WWAggr: A Window Wasserstein-based Aggregation for Ensemble Change Point Detection arXiv:2506.08066v1 Announce Type: new Abstract: Change Point Detection (CPD) aims to identify moments of abrupt distribution shifts in data streams. Real-world high-dimensional CPD remains challenging due to data pattern complexity and violation of common assumptions. Resorting to standalone deep neural networks, the current state-of-the-art detectors…

  • Model-Free Kernel Conformal Depth Measures Algorithm for Uncertainty Quantification in Regression Models in Separable Hilbert Spaces

    Model-Free Kernel Conformal Depth Measures Algorithm for Uncertainty Quantification in Regression Models in Separable Hilbert Spaces arXiv:2506.08325v1 Announce Type: new Abstract: Depth measures are powerful tools for defining level sets in emerging, non–standard, and complex random objects such as high-dimensional multivariate data, functional data, and random graphs. Despite their favorable theoretical properties, the integration of…

  • Asymptotic Normality of Infinite Centered Random Forests -Application to Imbalanced Classification

    Asymptotic Normality of Infinite Centered Random Forests -Application to Imbalanced Classification arXiv:2506.08548v1 Announce Type: new Abstract: Many classification tasks involve imbalanced data, in which a class is largely underrepresented. Several techniques consists in creating a rebalanced dataset on which a classifier is trained. In this paper, we study theoretically such a procedure, when the classifier…

  • Audio Spectrogram Transformers Beyond the Lab

    Audio Spectrogram Transformers Beyond the Lab A recipe for building a portable soundscape monitoring app with AudioMoth, Raspberry Pi, and a decent dose of deep learning. The post Audio Spectrogram Transformers Beyond the Lab appeared first on Towards Data Science. Maciej Adamiak Go to original source

  • Automate Models Training: An MLOps Pipeline with Tekton and Buildpacks

    Automate Models Training: An MLOps Pipeline with Tekton and Buildpacks A step-by-step guide to containerizing and orchestrating an ML training workflow without the Dockerfile headache, using a lightweight GPT-2 example. The post Automate Models Training: An MLOps Pipeline with Tekton and Buildpacks appeared first on Towards Data Science. Sylvain Kalache Go to original source

  • 10,000x Faster Bayesian Inference: Multi-GPU SVI vs. Traditional MCMC

    10,000x Faster Bayesian Inference: Multi-GPU SVI vs. Traditional MCMC Using GPU acceleration to speed up Bayesian Inference from months to minutes… The post 10,000x Faster Bayesian Inference: Multi-GPU SVI vs. Traditional MCMC appeared first on Towards Data Science. Derek Tran Go to original source

  • Applications of Density Estimation to Legal Theory

    Applications of Density Estimation to Legal Theory A brief analysis using density estimation to compare the two-verdict and three-verdict systems. The post Applications of Density Estimation to Legal Theory appeared first on Towards Data Science. Jimin Kang Go to original source

  • Direct Fisher Score Estimation for Likelihood Maximization

    Direct Fisher Score Estimation for Likelihood Maximization arXiv:2506.06542v1 Announce Type: new Abstract: We study the problem of likelihood maximization when the likelihood function is intractable but model simulations are readily available. We propose a sequential, gradient-based optimization method that directly models the Fisher score based on a local score matching technique which uses simulations from…

  • On the Fundamental Impossibility of Hallucination Control in Large Language Models

    On the Fundamental Impossibility of Hallucination Control in Large Language Models arXiv:2506.06382v1 Announce Type: new Abstract: This paper explains textbf{why it is impossible to create large language models that do not hallucinate and what are the trade-offs we should be looking for}. It presents a formal textbf{impossibility theorem} demonstrating that no inference mechanism can simultaneously…

  • Robust Learnability of Sample-Compressible Distributions under Noisy or Adversarial Perturbations

    Robust Learnability of Sample-Compressible Distributions under Noisy or Adversarial Perturbations arXiv:2506.06613v1 Announce Type: new Abstract: Learning distribution families over $mathbb{R}^d$ is a fundamental problem in unsupervised learning and statistics. A central question in this setting is whether a given family of distributions possesses sufficient structure to be (at least) information-theoretically learnable and, if so, to…

  • Continuous Semi-Implicit Models

    Continuous Semi-Implicit Models arXiv:2506.06778v1 Announce Type: new Abstract: Semi-implicit distributions have shown great promise in variational inference and generative modeling. Hierarchical semi-implicit models, which stack multiple semi-implicit layers, enhance the expressiveness of semi-implicit distributions and can be used to accelerate diffusion models given pretrained score networks. However, their sequential training often suffers from slow convergence.…

  • The Currents of Conflict: Decomposing Conflict Trends with Gaussian Processes

    The Currents of Conflict: Decomposing Conflict Trends with Gaussian Processes arXiv:2506.06828v1 Announce Type: new Abstract: I present a novel approach to estimating the temporal and spatial patterns of violent conflict. I show how we can use highly temporally and spatially disaggregated data on conflict events in tandem with Gaussian processes to estimate temporospatial conflict trends.…

  • Mastering SQL Window Functions

    Mastering SQL Window Functions Understand how to use Window Functions to perform calculations without losing details The post Mastering SQL Window Functions appeared first on Towards Data Science. Eugenia Anello Go to original source

  • Exploratory Data Analysis: Gamma Spectroscopy in Python

    Exploratory Data Analysis: Gamma Spectroscopy in Python Let’s observe the matter on the atomic level The post Exploratory Data Analysis: Gamma Spectroscopy in Python appeared first on Towards Data Science. Dmitrii Eliuseev Go to original source

  • A Bird’s-Eye View of Linear Algebra: Measure of a Map — Determinants

    A Bird’s-Eye View of Linear Algebra: Measure of a Map — Determinants We roll up our sleeves and start to deal with matrices The post A Bird’s-Eye View of Linear Algebra: Measure of a Map — Determinants appeared first on Towards Data Science. Rohit Pandey Go to original source

  • How to Transition From Data Analyst to Data Scientist

    How to Transition From Data Analyst to Data Scientist Playbook on how data analysts can become data scientists The post How to Transition From Data Analyst to Data Scientist appeared first on Towards Data Science. Egor Howell Go to original source

  • Trying to Stay Sane in the Age of AI

    Trying to Stay Sane in the Age of AI A machine learning engineer’s quiet way of not losing her mind The post Trying to Stay Sane in the Age of AI appeared first on Towards Data Science. Amy Ma Go to original source

  • Nonlinear Causal Discovery through a Sequential Edge Orientation Approach

    Nonlinear Causal Discovery through a Sequential Edge Orientation Approach arXiv:2506.05590v1 Announce Type: new Abstract: Recent advances have established the identifiability of a directed acyclic graph (DAG) under additive noise models (ANMs), spurring the development of various causal discovery methods. However, most existing methods make restrictive model assumptions, rely heavily on general independence tests, or require…

  • Online Conformal Model Selection for Nonstationary Time Series

    Online Conformal Model Selection for Nonstationary Time Series arXiv:2506.05544v1 Announce Type: new Abstract: This paper introduces the MPS (Model Prediction Set), a novel framework for online model selection for nonstationary time series. Classical model selection methods, such as information criteria and cross-validation, rely heavily on the stationarity assumption and often fail in dynamic environments which…

  • Multilevel neural simulation-based inference

    Multilevel neural simulation-based inference arXiv:2506.06087v1 Announce Type: new Abstract: Neural simulation-based inference (SBI) is a popular set of methods for Bayesian inference when models are only available in the form of a simulator. These methods are widely used in the sciences and engineering, where writing down a likelihood can be significantly more challenging than constructing…

  • Adaptive stable distribution and Hurst exponent by method of moments moving estimator for nonstationary time series

    Adaptive stable distribution and Hurst exponent by method of moments moving estimator for nonstationary time series arXiv:2506.05354v1 Announce Type: cross Abstract: Nonstationarity of real-life time series requires model adaptation. In classical approaches like ARMA-ARCH there is assumed some arbitrarily chosen dependence type. To avoid their bias, we will focus on novel more agnostic approach: moving…

  • Zeroth-Order Optimization Finds Flat Minima

    Zeroth-Order Optimization Finds Flat Minima arXiv:2506.05454v1 Announce Type: cross Abstract: Zeroth-order methods are extensively used in machine learning applications where gradients are infeasible or expensive to compute, such as black-box attacks, reinforcement learning, and language model fine-tuning. Existing optimization theory focuses on convergence to an arbitrary stationary point, but less is known on the implicit…

  • Weekly Entering & Transitioning – Thread 09 Jun, 2025 – 16 Jun, 2025

    Weekly Entering & Transitioning – Thread 09 Jun, 2025 – 16 Jun, 2025 Welcome to this week’s entering & transitioning thread! This thread is for any questions about getting started, studying, or transitioning into the data science field. Topics include: Learning resources (e.g. books, tutorials, videos) Traditional education (e.g. schools, degrees, electives) Alternative education (e.g.…

  • PhD vs Masters prepared data scientist expectations.

    PhD vs Masters prepared data scientist expectations. Is there anything more that you expect from a data scientist with a PhD versus a data scientist with just a master’s degree, given the same level of experience? For the companies that I’ve worked with, most data science teams were mixes of folks with master’s degrees and…

  • What is your domain and what are the most important technical skills that help you stand out in your domain?

    What is your domain and what are the most important technical skills that help you stand out in your domain? Aside from soft skills and domain expertise, ofc those are a given. I’m manufacturing-adjacent (closer to product development and validation). Design of experiments has been my most useful data-related skill. I’m always being asked “We…

  • Data analyst vs. engineer? At non-profit

    Data analyst vs. engineer? At non-profit Hi all, I am the only Data Analyst at a medium-sized company related to shared transportation (adjacent to Lime Scooter/Bike). I’m pretty early in my career (grad from college 3 years ago). My role encompasses a LOT of responsibilities that aren’t traditionally under “data analyst”, the biggest of which…

  • You can now automate deep dives, with clear actionable recommendations based on data.

    You can now automate deep dives, with clear actionable recommendations based on data. submitted by /u/phicreative1997 [link] [comments] /u/phicreative1997 Go to original source

  • Not Everything Needs Automation: 5 Practical AI Agents That Deliver Enterprise Value

    Not Everything Needs Automation: 5 Practical AI Agents That Deliver Enterprise Value What actually works with AI agents inside enterprise organizations? The post Not Everything Needs Automation: 5 Practical AI Agents That Deliver Enterprise Value appeared first on Towards Data Science. Weiwei Hu Go to original source

  • Prescriptive Modeling Unpacked: A Complete Guide to Intervention With Bayesian Modeling.

    Prescriptive Modeling Unpacked: A Complete Guide to Intervention With Bayesian Modeling. Learn how to move beyond prediction and actively make intervention through prescriptive modeling. This in-depth guide walks you through Bayesian approaches to system intervention, with practical examples in predictive maintenance. The post Prescriptive Modeling Unpacked: A Complete Guide to Intervention With Bayesian Modeling. appeared…

  • How I Automated My Machine Learning Workflow with Just 10 Lines of Python

    How I Automated My Machine Learning Workflow with Just 10 Lines of Python Use LazyPredict and PyCaret to skip the grunt work and jump straight to performance. The post How I Automated My Machine Learning Workflow with Just 10 Lines of Python appeared first on Towards Data Science. Himanshu Sharma Go to original source

  • The Role of Luck in Sports: Can We Measure It?

    The Role of Luck in Sports: Can We Measure It? From last-minute goals to coin tosses: How much does randomness influence the outcomes of games? The post The Role of Luck in Sports: Can We Measure It? appeared first on Towards Data Science. Pol Marin Go to original source

  • Why AI Projects Fail

    Why AI Projects Fail No one agrees on the exact number, but estimates say anywhere from 50% to 80% of AI projects end in failure. The post Why AI Projects Fail appeared first on Towards Data Science. Ivo Bernardo Go to original source

  • On the Wasserstein Geodesic Principal Component Analysis of probability measures

    On the Wasserstein Geodesic Principal Component Analysis of probability measures arXiv:2506.04480v1 Announce Type: new Abstract: This paper focuses on Geodesic Principal Component Analysis (GPCA) on a collection of probability distributions using the Otto-Wasserstein geometry. The goal is to identify geodesic curves in the space of probability measures that best capture the modes of variation of…

  • Regret-Optimal Q-Learning with Low Cost for Single-Agent and Federated Reinforcement Learning

    Regret-Optimal Q-Learning with Low Cost for Single-Agent and Federated Reinforcement Learning arXiv:2506.04626v1 Announce Type: new Abstract: Motivated by real-world settings where data collection and policy deployment — whether for a single agent or across multiple agents — are costly, we study the problem of on-policy single-agent reinforcement learning (RL) and federated RL (FRL) with a…

  • Distributional encoding for Gaussian process regression with qualitative inputs

    Distributional encoding for Gaussian process regression with qualitative inputs arXiv:2506.04813v1 Announce Type: new Abstract: Gaussian Process (GP) regression is a popular and sample-efficient approach for many engineering applications, where observations are expensive to acquire, and is also a central ingredient of Bayesian optimization (BO), a highly prevailing method for the optimization of black-box functions. However,…

  • Learning Joint Interventional Effects from Single-Variable Interventions in Additive Models

    Learning Joint Interventional Effects from Single-Variable Interventions in Additive Models arXiv:2506.04945v1 Announce Type: new Abstract: Estimating causal effects of joint interventions on multiple variables is crucial in many domains, but obtaining data from such simultaneous interventions can be challenging. Our study explores how to learn joint interventional effects using only observational data and single-variable interventions.…

  • Nonlinear Causal Discovery for Grouped Data

    Nonlinear Causal Discovery for Grouped Data arXiv:2506.05120v1 Announce Type: new Abstract: Inferring cause-effect relationships from observational data has gained significant attention in recent years, but most methods are limited to scalar random variables. In many important domains, including neuroscience, psychology, social science, and industrial manufacturing, the causal units of interest are groups of variables rather…

  • SubSearch: Robust Estimation and Outlier Detection for Stochastic Block Models via Subgraph Search

    SubSearch: Robust Estimation and Outlier Detection for Stochastic Block Models via Subgraph Search arXiv:2506.03657v1 Announce Type: new Abstract: Community detection is a fundamental task in graph analysis, with methods often relying on fitting models like the Stochastic Block Model (SBM) to observed networks. While many algorithms can accurately estimate SBM parameters when the input graph…

  • Models of Heavy-Tailed Mechanistic Universality

    Models of Heavy-Tailed Mechanistic Universality arXiv:2506.03470v1 Announce Type: new Abstract: Recent theoretical and empirical successes in deep learning, including the celebrated neural scaling laws, are punctuated by the observation that many objects of interest tend to exhibit some form of heavy-tailed or power law behavior. In particular, the prevalence of heavy-tailed spectral densities in Jacobians,…

  • Position: There Is No Free Bayesian Uncertainty Quantification

    Position: There Is No Free Bayesian Uncertainty Quantification arXiv:2506.03670v1 Announce Type: new Abstract: Due to their intuitive appeal, Bayesian methods of modeling and uncertainty quantification have become popular in modern machine and deep learning. When providing a prior distribution over the parameter space, it is straightforward to obtain a distribution over the parameters that is…

  • Latent Guided Sampling for Combinatorial Optimization

    Latent Guided Sampling for Combinatorial Optimization arXiv:2506.03672v1 Announce Type: new Abstract: Combinatorial Optimization problems are widespread in domains such as logistics, manufacturing, and drug discovery, yet their NP-hard nature makes them computationally challenging. Recent Neural Combinatorial Optimization methods leverage deep learning to learn solution strategies, trained via Supervised or Reinforcement Learning (RL). While promising, these…

  • Infinitesimal Higher-Order Spectral Variations in Rectangular Real Random Matrices

    Infinitesimal Higher-Order Spectral Variations in Rectangular Real Random Matrices arXiv:2506.03764v1 Announce Type: new Abstract: We present a theoretical framework for deriving the general $n$-th order Fr’echet derivatives of singular values in real rectangular matrices, by leveraging reduced resolvent operators from Kato’s analytic perturbation theory for self-adjoint operators. Deriving closed-form expressions for higher-order derivatives of singular…

  • The Journey from Jupyter to Programmer: A Quick-Start Guide

    The Journey from Jupyter to Programmer: A Quick-Start Guide Explore the real benefits of ditching the notebook The post The Journey from Jupyter to Programmer: A Quick-Start Guide appeared first on Towards Data Science. Lucy Dickinson Go to original source

  • Building a Modern Dashboard with Python and Gradio

    Building a Modern Dashboard with Python and Gradio Data insights made simple The post Building a Modern Dashboard with Python and Gradio appeared first on Towards Data Science. Thomas Reid Go to original source

  • Assumption-free stability for ranking problems

    Assumption-free stability for ranking problems arXiv:2506.02257v1 Announce Type: new Abstract: In this work, we consider ranking problems among a finite set of candidates: for instance, selecting the top-$k$ items among a larger list of candidates or obtaining the full ranking of all items in the set. These problems are often unstable, in the sense that…

  • Enabling Probabilistic Learning on Manifolds through Double Diffusion Maps

    Enabling Probabilistic Learning on Manifolds through Double Diffusion Maps arXiv:2506.02254v1 Announce Type: new Abstract: We present a generative learning framework for probabilistic sampling based on an extension of the Probabilistic Learning on Manifolds (PLoM) approach, which is designed to generate statistically consistent realizations of a random vector in a finite-dimensional Euclidean space, informed by a…

  • MoCA: Multi-modal Cross-masked Autoencoder for Digital Health Measurements

    MoCA: Multi-modal Cross-masked Autoencoder for Digital Health Measurements arXiv:2506.02260v1 Announce Type: new Abstract: The growing prevalence of digital health technologies has led to the generation of complex multi-modal data, such as physical activity measurements simultaneously collected from various sensors of mobile and wearable devices. These data hold immense potential for advancing health studies, but current…

  • Large Stepsizes Accelerate Gradient Descent for Regularized Logistic Regression

    Large Stepsizes Accelerate Gradient Descent for Regularized Logistic Regression arXiv:2506.02336v1 Announce Type: new Abstract: We study gradient descent (GD) with a constant stepsize for $ell_2$-regularized logistic regression with linearly separable data. Classical theory suggests small stepsizes to ensure monotonic reduction of the optimization objective, achieving exponential convergence in $widetilde{mathcal{O}}(kappa)$ steps with $kappa$ being the condition…

  • Tensor State Space-based Dynamic Multilayer Network Modeling

    Tensor State Space-based Dynamic Multilayer Network Modeling arXiv:2506.02413v1 Announce Type: new Abstract: Understanding the complex interactions within dynamic multilayer networks is critical for advancements in various scientific domains. Existing models often fail to capture such networks’ temporal and cross-layer dynamics. This paper introduces a novel Tensor State Space Model for Dynamic Multilayer Networks (TSSDMN), utilizing…

  • Data Drift Is Not the Actual Problem: Your Monitoring Strategy Is

    Data Drift Is Not the Actual Problem: Your Monitoring Strategy Is Monitoring is easy; what to monitor is not. In the field of machine learning, data drift is just noise until you know what it means. The post Data Drift Is Not the Actual Problem: Your Monitoring Strategy Is appeared first on Towards Data Science.…

  • Reducing Time to Value for Data Science Projects: Part 2

    Reducing Time to Value for Data Science Projects: Part 2 Leveraging automation and parallelism to scale out experiments The post Reducing Time to Value for Data Science Projects: Part 2 appeared first on Towards Data Science. Kristopher McGlinchey Go to original source

  • Pairwise Cross-Variance Classification

    Pairwise Cross-Variance Classification Multi-class zero-shot embedding classification and error checking The post Pairwise Cross-Variance Classification appeared first on Towards Data Science. Doster Esh Go to original source

  • Landing your First Machine Learning Job: Startup vs Big Tech vs Academia

    Landing your First Machine Learning Job: Startup vs Big Tech vs Academia A practical guide to landing your first Machine Learning job across startups, big tech, and academia. The post Landing your First Machine Learning Job: Startup vs Big Tech vs Academia appeared first on Towards Data Science. Piero Paialunga Go to original source

  • Decision Trees Natively Handle Categorical Data

    Decision Trees Natively Handle Categorical Data But mean target encoding is their turbocharger The post Decision Trees Natively Handle Categorical Data appeared first on Towards Data Science. Vadim Arzamasov Go to original source

  • Minimax Rates for the Estimation of Eigenpairs of Weighted Laplace-Beltrami Operators on Manifolds

    Minimax Rates for the Estimation of Eigenpairs of Weighted Laplace-Beltrami Operators on Manifolds arXiv:2506.00171v1 Announce Type: new Abstract: We study the problem of estimating eigenpairs of elliptic differential operators from samples of a distribution $rho$ supported on a manifold $M$. The operators discussed in the paper are relevant in unsupervised learning and in particular are…