mailitics

Category: cs.LG

  • Asymptotics of Linear Regression with Linearly Dependent Data

    Asymptotics of Linear Regression with Linearly Dependent Data arXiv:2412.03702v1 Announce Type: new Abstract: In this paper we study the asymptotics of linear regression in settings where the covariates exhibit a linear dependency structure, departing from the standard assumption of independence. We model the covariates using stochastic processes with spatio-temporal covariance and analyze the performance of…

  • Community Detection with Heterogeneous Block Covariance Model

    Community Detection with Heterogeneous Block Covariance Model arXiv:2412.03780v1 Announce Type: new Abstract: Community detection is the task of clustering objects based on their pairwise relationships. Most of the model-based community detection methods, such as the stochastic block model and its variants, are designed for networks with binary (yes/no) edges. In many practical scenarios, edges often…

  • Learning Networks from Wide-Sense Stationary Stochastic Processes

    Learning Networks from Wide-Sense Stationary Stochastic Processes arXiv:2412.03768v1 Announce Type: new Abstract: Complex networked systems driven by latent inputs are common in fields like neuroscience, finance, and engineering. A key inference problem here is to learn edge connectivity from node outputs (potentials). We focus on systems governed by steady-state linear conservation laws: $X_t = {L^{ast}}Y_{t}$,…

  • How well behaved is finite dimensional Diffusion Maps?

    How well behaved is finite dimensional Diffusion Maps? arXiv:2412.03992v1 Announce Type: new Abstract: Under a set of assumptions on a family of submanifolds $subset {mathbb R}^D$, we derive a series of geometric properties that remain valid after finite-dimensional and almost isometric Diffusion Maps (DM), including almost uniform density, finite polynomial approximation and local reach. Leveraging…

  • Pathwise optimization for bridge-type estimators and its applications

    Pathwise optimization for bridge-type estimators and its applications arXiv:2412.04047v1 Announce Type: new Abstract: Sparse parametric models are of great interest in statistical learning and are often analyzed by means of regularized estimators. Pathwise methods allow to efficiently compute the full solution path for penalized estimators, for any possible value of the penalization parameter $lambda$. In…

  • Universal Rates of Empirical Risk Minimization

    Universal Rates of Empirical Risk Minimization arXiv:2412.02810v1 Announce Type: new Abstract: The well-known empirical risk minimization (ERM) principle is the basis of many widely used machine learning algorithms, and plays an essential role in the classical PAC theory. A common description of a learning algorithm’s performance is its so-called “learning curve”, that is, the decay…

  • An Information-Theoretic Analysis of Thompson Sampling for Logistic Bandits

    An Information-Theoretic Analysis of Thompson Sampling for Logistic Bandits arXiv:2412.02861v1 Announce Type: new Abstract: We study the performance of the Thompson Sampling algorithm for logistic bandit problems, where the agent receives binary rewards with probabilities determined by a logistic function $exp(beta langle a, theta rangle)/(1+exp(beta langle a, theta rangle))$. We focus on the setting where…

  • Preference-based Pure Exploration

    Preference-based Pure Exploration arXiv:2412.02988v1 Announce Type: new Abstract: We study the preference-based pure exploration problem for bandits with vector-valued rewards. The rewards are ordered using a (given) preference cone $mathcal{C}$ and our the goal is to identify the set of Pareto optimal arms. First, to quantify the impact of preferences, we derive a novel lower…

  • Generalized Diffusion Model with Adjusted Offset Noise

    Generalized Diffusion Model with Adjusted Offset Noise arXiv:2412.03134v1 Announce Type: new Abstract: Diffusion models have become fundamental tools for modeling data distributions in machine learning and have applications in image generation, drug discovery, and audio synthesis. Despite their success, these models face challenges when generating data with extreme brightness values, as evidenced by limitations in…

  • Nonparametric Filtering, Estimation and Classification using Neural Jump ODEs

    Nonparametric Filtering, Estimation and Classification using Neural Jump ODEs arXiv:2412.03271v1 Announce Type: new Abstract: Neural Jump ODEs model the conditional expectation between observations by neural ODEs and jump at arrival of new observations. They have demonstrated effectiveness for fully data-driven online forecasting in settings with irregular and partial observations, operating under weak regularity assumptions. This…

  • MEP-Net: Generating Solutions to Scientific Problems with Limited Knowledge by Maximum Entropy Principle

    MEP-Net: Generating Solutions to Scientific Problems with Limited Knowledge by Maximum Entropy Principle arXiv:2412.02090v1 Announce Type: new Abstract: Maximum entropy principle (MEP) offers an effective and unbiased approach to inferring unknown probability distributions when faced with incomplete information, while neural networks provide the flexibility to learn complex distributions from data. This paper proposes a novel…

  • Selective Reviews of Bandit Problems in AI via a Statistical View

    Selective Reviews of Bandit Problems in AI via a Statistical View arXiv:2412.02251v1 Announce Type: new Abstract: Reinforcement Learning (RL) is a widely researched area in artificial intelligence that focuses on teaching agents decision-making through interactions with their environment. A key subset includes stochastic multi-armed bandit (MAB) and continuum-armed bandit (SCAB) problems, which model sequential decision-making…

  • Deep Matrix Factorization with Adaptive Weights for Multi-View Clustering

    Deep Matrix Factorization with Adaptive Weights for Multi-View Clustering arXiv:2412.02292v1 Announce Type: new Abstract: Recently, deep matrix factorization has been established as a powerful model for unsupervised tasks, achieving promising results, especially for multi-view clustering. However, existing methods often lack effective feature selection mechanisms and rely on empirical hyperparameter selection. To address these issues, we…

  • Composition of Experts: A Modular Compound AI System Leveraging Large Language Models

    Composition of Experts: A Modular Compound AI System Leveraging Large Language Models arXiv:2412.01868v1 Announce Type: cross Abstract: Large Language Models (LLMs) have achieved remarkable advancements, but their monolithic nature presents challenges in terms of scalability, cost, and customization. This paper introduces the Composition of Experts (CoE), a modular compound AI system leveraging multiple expert LLMs.…

  • Nonlinearity and Uncertainty Informed Moment-Matching Gaussian Mixture Splitting

    Nonlinearity and Uncertainty Informed Moment-Matching Gaussian Mixture Splitting arXiv:2412.00343v1 Announce Type: new Abstract: Many problems in navigation and tracking require increasingly accurate characterizations of the evolution of uncertainty in nonlinear systems. Nonlinear uncertainty propagation approaches based on Gaussian mixture density approximations offer distinct advantages over sampling based methods in their computational cost and continuous representation.…

  • Optimal Particle-based Approximation of Discrete Distributions (OPAD)

    Optimal Particle-based Approximation of Discrete Distributions (OPAD) arXiv:2412.00545v1 Announce Type: new Abstract: Particle-based methods include a variety of techniques, such as Markov Chain Monte Carlo (MCMC) and Sequential Monte Carlo (SMC), for approximating a probabilistic target distribution with a set of weighted particles. In this paper, we prove that for any set of particles, there…

  • Explicit and data-Efficient Encoding via Gradient Flow

    Explicit and data-Efficient Encoding via Gradient Flow arXiv:2412.00864v1 Announce Type: new Abstract: The autoencoder model typically uses an encoder to map data to a lower dimensional latent space and a decoder to reconstruct it. However, relying on an encoder for inversion can lead to suboptimal representations, particularly limiting in physical sciences where precision is key.…

  • A Note on Estimation Error Bound and Grouping Effect of Transfer Elastic Net

    A Note on Estimation Error Bound and Grouping Effect of Transfer Elastic Net arXiv:2412.01010v1 Announce Type: new Abstract: The Transfer Elastic Net is an estimation method for linear regression models that combines $ell_1$ and $ell_2$ norm penalties to facilitate knowledge transfer. In this study, we derive a non-asymptotic $ell_2$ norm estimation error bound for the…

  • Energy-Based Modelling for Discrete and Mixed Data via Heat Equations on Structured Spaces

    Energy-Based Modelling for Discrete and Mixed Data via Heat Equations on Structured Spaces arXiv:2412.01019v1 Announce Type: new Abstract: Energy-based models (EBMs) offer a flexible framework for probabilistic modelling across various data domains. However, training EBMs on data in discrete or mixed state spaces poses significant challenges due to the lack of robust and fast sampling…

  • The Return of Pseudosciences in Artificial Intelligence: Have Machine Learning and Deep Learning Forgotten Lessons from Statistics and History?

    The Return of Pseudosciences in Artificial Intelligence: Have Machine Learning and Deep Learning Forgotten Lessons from Statistics and History? arXiv:2411.18656v1 Announce Type: new Abstract: In today’s world, AI programs powered by Machine Learning are ubiquitous, and have achieved seemingly exceptional performance across a broad range of tasks, from medical diagnosis and credit rating in banking,…

  • Graph Max Shift: A Hill-Climbing Method for Graph Clustering

    Graph Max Shift: A Hill-Climbing Method for Graph Clustering arXiv:2411.18794v1 Announce Type: new Abstract: We present a method for graph clustering that is analogous with gradient ascent methods previously proposed for clustering points in space. We show that, when applied to a random geometric graph with data iid from some density with Morse regularity, the…

  • Intrinsic Wrapped Gaussian Process Regression Modeling for Manifold-valued Response Variable

    Intrinsic Wrapped Gaussian Process Regression Modeling for Manifold-valued Response Variable arXiv:2411.18989v1 Announce Type: new Abstract: In this paper, we propose a novel intrinsic wrapped Gaussian process regression model for response variable measured on Riemannian manifold. We apply the parallel transport operator to define an intrinsic covariance structure addressing a critical aspect of constructing a well…

  • ABROCA Distributions For Algorithmic Bias Assessment: Considerations Around Interpretation

    ABROCA Distributions For Algorithmic Bias Assessment: Considerations Around Interpretation arXiv:2411.19090v1 Announce Type: new Abstract: Algorithmic bias continues to be a key concern of learning analytics. We study the statistical properties of the Absolute Between-ROC Area (ABROCA) metric. This fairness measure quantifies group-level differences in classifier performance through the absolute difference in ROC curves. ABROCA is…

  • Contrastive representations of high-dimensional, structured treatments

    Contrastive representations of high-dimensional, structured treatments arXiv:2411.19245v1 Announce Type: new Abstract: Estimating causal effects is vital for decision making. In standard causal effect estimation, treatments are usually binary- or continuous-valued. However, in many important real-world settings, treatments can be structured, high-dimensional objects, such as text, video, or audio. This provides a challenge to traditional causal…

  • On the ERM Principle in Meta-Learning

    On the ERM Principle in Meta-Learning arXiv:2411.17898v1 Announce Type: new Abstract: Classic supervised learning involves algorithms trained on $n$ labeled examples to produce a hypothesis $h in mathcal{H}$ aimed at performing well on unseen examples. Meta-learning extends this by training across $n$ tasks, with $m$ examples per task, producing a hypothesis class $mathcal{H}$ within some…

  • Isometry pursuit

    Isometry pursuit arXiv:2411.18502v1 Announce Type: new Abstract: Isometry pursuit is a convex algorithm for identifying orthonormal column-submatrices of wide matrices. It consists of a novel normalization method followed by multitask basis pursuit. Applied to Jacobians of putative coordinate functions, it helps identity isometric embeddings from within interpretable dictionaries. We provide theoretical and experimental results justifying…

  • Functional relevance based on the continuous Shapley value

    Functional relevance based on the continuous Shapley value arXiv:2411.18575v1 Announce Type: new Abstract: The presence of Artificial Intelligence (AI) in our society is increasing, which brings with it the need to understand the behaviour of AI mechanisms, including machine learning predictive algorithms fed with tabular data, text, or images, among other types of data. This…

  • When Is Heterogeneity Actionable for Personalization?

    When Is Heterogeneity Actionable for Personalization? arXiv:2411.16552v1 Announce Type: cross Abstract: Targeting and personalization policies can be used to improve outcomes beyond the uniform policy that assigns the best performing treatment in an A/B test to everyone. Personalization relies on the presence of heterogeneity of treatment effects, yet, as we show in this paper, heterogeneity…

  • Conformalised Conditional Normalising Flows for Joint Prediction Regions in time series

    Conformalised Conditional Normalising Flows for Joint Prediction Regions in time series arXiv:2411.17042v1 Announce Type: new Abstract: Conformal Prediction offers a powerful framework for quantifying uncertainty in machine learning models, enabling the construction of prediction sets with finite-sample validity guarantees. While easily adaptable to non-probabilistic models, applying conformal prediction to probabilistic generative models, such as Normalising…

  • Fast, Precise Thompson Sampling for Bayesian Optimization

    Fast, Precise Thompson Sampling for Bayesian Optimization arXiv:2411.17071v1 Announce Type: new Abstract: Thompson sampling (TS) has optimal regret and excellent empirical performance in multi-armed bandit problems. Yet, in Bayesian optimization, TS underperforms popular acquisition functions (e.g., EI, UCB). TS samples arms according to the probability that they are optimal. A recent algorithm, P-Star Sampler (PSS),…

  • Spatio-Temporal Conformal Prediction for Power Outage Data

    Spatio-Temporal Conformal Prediction for Power Outage Data arXiv:2411.17099v1 Announce Type: new Abstract: In recent years, increasingly unpredictable and severe global weather patterns have frequently caused long-lasting power outages. Building resilience, the ability to withstand, adapt to, and recover from major disruptions, has become crucial for the power industry. To enable rapid recovery, accurately predicting future…

  • Training a neural netwok for data reduction and better generalization

    Training a neural netwok for data reduction and better generalization arXiv:2411.17180v1 Announce Type: new Abstract: The motivation for sparse learners is to compress the inputs (features) by selecting only the ones needed for good generalization. Linear models with LASSO-type regularization achieve this by setting the weights of irrelevant features to zero, effectively identifying and ignoring…

  • A Generalized Unified Skew-Normal Process with Neural Bayes Inference

    A Generalized Unified Skew-Normal Process with Neural Bayes Inference arXiv:2411.17400v1 Announce Type: new Abstract: In recent decades, statisticians have been increasingly encountering spatial data that exhibit non-Gaussian behaviors such as asymmetry and heavy-tailedness. As a result, the assumptions of symmetry and fixed tail weight in Gaussian processes have become restrictive and may fail to capture…