mailitics

Tag: kernel

  • Random Features for Operator-Valued Kernels: Bridging Kernel Methods and Neural Operators

    Random Features for Operator-Valued Kernels: Bridging Kernel Methods and Neural Operators arXiv:2603.00971v1 Announce Type: new Abstract: In this work, we investigate the generalization properties of random feature methods. Our analysis extends prior results for Tikhonov regularization to a broad class of spectral regularization techniques and further generalizes the setting to operator-valued kernels. This unified framework…

  • A Kernel Approach for Semi-implicit Variational Inference

    A Kernel Approach for Semi-implicit Variational Inference arXiv:2601.12023v1 Announce Type: new Abstract: Semi-implicit variational inference (SIVI) enhances the expressiveness of variational families through hierarchical semi-implicit distributions, but the intractability of their densities makes standard ELBO-based optimization biased. Recent score-matching approaches to SIVI (SIVI-SM) address this issue via a minimax formulation, at the expense of an…

  • The Machine Learning “Advent Calendar” Day 16: Kernel Trick in Excel

    The Machine Learning “Advent Calendar” Day 16: Kernel Trick in Excel Kernel SVM often feels abstract, with kernels, dual formulations, and support vectors. In this article, we take a different path. Starting from Kernel Density Estimation, we build Kernel SVM step by step as a sum of local bells, weighted and selected by hinge loss,…

  • Error Analysis of Generalized Langevin Equations with Approximated Memory Kernels

    Error Analysis of Generalized Langevin Equations with Approximated Memory Kernels arXiv:2512.10256v1 Announce Type: new Abstract: We analyze prediction error in stochastic dynamical systems with memory, focusing on generalized Langevin equations (GLEs) formulated as stochastic Volterra equations. We establish that, under a strongly convex potential, trajectory discrepancies decay at a rate determined by the decay of…

  • Sparse Multiple Kernel Learning: Alternating Best Response and Semidefinite Relaxations

    Sparse Multiple Kernel Learning: Alternating Best Response and Semidefinite Relaxations arXiv:2511.21890v1 Announce Type: new Abstract: We study Sparse Multiple Kernel Learning (SMKL), which is the problem of selecting a sparse convex combination of prespecified kernels for support vector binary classification. Unlike prevailing l1 regularized approaches that approximate a sparsifying penalty, we formulate the problem by…

  • Quantum Fourier Transform Based Kernel for Solar Irrandiance Forecasting

    Quantum Fourier Transform Based Kernel for Solar Irrandiance Forecasting arXiv:2511.17698v1 Announce Type: new Abstract: This study proposes a Quantum Fourier Transform (QFT)-enhanced quantum kernel for short-term time-series forecasting. Each signal is windowed, amplitude-encoded, transformed by a QFT, then passed through a protective rotation layer to avoid the QFT/QFT adjoint cancellation; the resulting kernel is used…

  • Learning Triton One Kernel at a Time: Softmax

    Learning Triton One Kernel at a Time: Softmax All you need to know about a fast, readable and PyTorch-ready softmax kernel The post Learning Triton One Kernel at a Time: Softmax appeared first on Towards Data Science. Ryan Pégoud Go to original source

  • Atlas Gaussian processes on restricted domains and point clouds

    Atlas Gaussian processes on restricted domains and point clouds arXiv:2511.15822v1 Announce Type: new Abstract: In real-world applications, data often reside in restricted domains with unknown boundaries, or as high-dimensional point clouds lying on a lower-dimensional, nontrivial, unknown manifold. Traditional Gaussian Processes (GPs) struggle to capture the underlying geometry in such settings. Some existing methods assume…

  • A general technique for approximating high-dimensional empirical kernel matrices

    A general technique for approximating high-dimensional empirical kernel matrices arXiv:2511.03892v1 Announce Type: new Abstract: We present simple, user-friendly bounds for the expected operator norm of a random kernel matrix under general conditions on the kernel function $k(cdot,cdot)$. Our approach uses decoupling results for U-statistics and the non-commutative Khintchine inequality to obtain upper and lower bounds…

  • $L_1$-norm Regularized Indefinite Kernel Logistic Regression

    $L_1$-norm Regularized Indefinite Kernel Logistic Regression arXiv:2510.26043v1 Announce Type: new Abstract: Kernel logistic regression (KLR) is a powerful classification method widely applied across diverse domains. In many real-world scenarios, indefinite kernels capture more domain-specific structural information than positive definite kernels. This paper proposes a novel $L_1$-norm regularized indefinite kernel logistic regression (RIKLR) model, which extends…

  • Kernel Learning with Adversarial Features: Numerical Efficiency and Adaptive Regularization

    Kernel Learning with Adversarial Features: Numerical Efficiency and Adaptive Regularization arXiv:2510.20883v1 Announce Type: new Abstract: Adversarial training has emerged as a key technique to enhance model robustness against adversarial input perturbations. Many of the existing methods rely on computationally expensive min-max problems that limit their application in practice. We propose a novel formulation of adversarial…

  • Signature Kernel Scoring Rule as Spatio-Temporal Diagnostic for Probabilistic Forecasting

    Signature Kernel Scoring Rule as Spatio-Temporal Diagnostic for Probabilistic Forecasting arXiv:2510.19110v1 Announce Type: new Abstract: Modern weather forecasting has increasingly transitioned from numerical weather prediction (NWP) to data-driven machine learning forecasting techniques. While these new models produce probabilistic forecasts to quantify uncertainty, their training and evaluation may remain hindered by conventional scoring rules, primarily MSE,…

  • Learning Triton One Kernel at a Time: Matrix Multiplication

    Learning Triton One Kernel at a Time: Matrix Multiplication Tiled GEMM, GPU memory, coalescing, and much more! The post Learning Triton One Kernel at a Time: Matrix Multiplication appeared first on Towards Data Science. Ryan Pégoud Go to original source

  • Kernel Treatment Effects with Adaptively Collected Data

    Kernel Treatment Effects with Adaptively Collected Data arXiv:2510.10245v1 Announce Type: new Abstract: Adaptive experiments improve efficiency by adjusting treatment assignments based on past outcomes, but this adaptivity breaks the i.i.d. assumptions that underpins classical asymptotics. At the same time, many questions of interest are distributional, extending beyond average effects. Kernel treatment effects (KTE) provide a…

  • A Representer Theorem for Hawkes Processes via Penalized Least Squares Minimization

    A Representer Theorem for Hawkes Processes via Penalized Least Squares Minimization arXiv:2510.08916v1 Announce Type: new Abstract: The representer theorem is a cornerstone of kernel methods, which aim to estimate latent functions in reproducing kernel Hilbert spaces (RKHSs) in a nonparametric manner. Its significance lies in converting inherently infinite-dimensional optimization problems into finite-dimensional ones over dual…

  • Learning Multi-Index Models with Hyper-Kernel Ridge Regression

    Learning Multi-Index Models with Hyper-Kernel Ridge Regression arXiv:2510.02532v1 Announce Type: new Abstract: Deep neural networks excel in high-dimensional problems, outperforming models such as kernel methods, which suffer from the curse of dimensionality. However, the theoretical foundations of this success remain poorly understood. We follow the idea that the compositional structure of the learning task is…

  • Learning Triton One Kernel At a Time: Vector Addition

    Learning Triton One Kernel At a Time: Vector Addition The basics of GPU programming, optimisation, and your first Triton kernel The post Learning Triton One Kernel At a Time: Vector Addition appeared first on Towards Data Science. Ryan Pégoud Go to original source

  • Kernel VICReg for Self-Supervised Learning in Reproducing Kernel Hilbert Space

    Kernel VICReg for Self-Supervised Learning in Reproducing Kernel Hilbert Space arXiv:2509.07289v1 Announce Type: new Abstract: Self-supervised learning (SSL) has emerged as a powerful paradigm for representation learning by optimizing geometric objectives–such as invariance to augmentations, variance preservation, and feature decorrelation–without requiring labels. However, most existing methods operate in Euclidean space, limiting their ability to capture…

  • Fast kernel methods: Sobolev, physics-informed, and additive models

    Fast kernel methods: Sobolev, physics-informed, and additive models arXiv:2509.02649v1 Announce Type: new Abstract: Kernel methods are powerful tools in statistical learning, but their cubic complexity in the sample size n limits their use on large-scale datasets. In this work, we introduce a scalable framework for kernel regression with O(n log n) complexity, fully leveraging GPU…

  • Interpretable Kernels

    Interpretable Kernels arXiv:2508.15932v1 Announce Type: new Abstract: The use of kernels for nonlinear prediction is widespread in machine learning. They have been popularized in support vector machines and used in kernel ridge regression, amongst others. Kernel methods share three aspects. First, instead of the original matrix of predictor variables or features, each observation is mapped…

  • Uniform convergence for Gaussian kernel ridge regression

    Uniform convergence for Gaussian kernel ridge regression arXiv:2508.11274v1 Announce Type: new Abstract: This paper establishes the first polynomial convergence rates for Gaussian kernel ridge regression (KRR) with a fixed hyperparameter in both the uniform and the $L^{2}$-norm. The uniform convergence result closes a gap in the theoretical understanding of KRR with the Gaussian kernel, where…

  • Structured Kernel Regression VAE: A Computationally Efficient Surrogate for GP-VAEs in ICA

    Structured Kernel Regression VAE: A Computationally Efficient Surrogate for GP-VAEs in ICA arXiv:2508.09721v1 Announce Type: new Abstract: The interpretability of generative models is considered a key factor in demonstrating their effectiveness and controllability. The generated data are believed to be determined by latent variables that are not directly observable. Therefore, disentangling, decoupling, decomposing, causal inference,…

  • From Local Interactions to Global Operators: Scalable Gaussian Process Operator for Physical Systems

    From Local Interactions to Global Operators: Scalable Gaussian Process Operator for Physical Systems arXiv:2506.15906v1 Announce Type: new Abstract: Operator learning offers a powerful paradigm for solving parametric partial differential equations (PDEs), but scaling probabilistic neural operators such as the recently proposed Gaussian Processes Operators (GPOs) to high-dimensional, data-intensive regimes remains a significant challenge. In this…

  • Kernel Quantile Embeddings and Associated Probability Metrics

    Kernel Quantile Embeddings and Associated Probability Metrics arXiv:2505.20433v1 Announce Type: new Abstract: Embedding probability distributions into reproducing kernel Hilbert spaces (RKHS) has enabled powerful nonparametric methods such as the maximum mean discrepancy (MMD), a statistical distance with strong theoretical and computational properties. At its core, the MMD relies on kernel mean embeddings to represent distributions…

  • Kernel Density Machines

    Kernel Density Machines arXiv:2504.21419v1 Announce Type: new Abstract: We introduce kernel density machines (KDM), a novel density ratio estimator in a reproducing kernel Hilbert space setting. KDM applies to general probability measures on countably generated measurable spaces without restrictive assumptions on continuity, or the existence of a Lebesgue density. For computational efficiency, we incorporate a…

  • Sobolev norm inconsistency of kernel interpolation

    Sobolev norm inconsistency of kernel interpolation arXiv:2504.20617v1 Announce Type: new Abstract: We study the consistency of minimum-norm interpolation in reproducing kernel Hilbert spaces corresponding to bounded kernels. Our main result give lower bounds for the generalization error of the kernel interpolation measured in a continuous scale of norms that interpolate between $L^2$ and the hypothesis…

  • A Dictionary of Closed-Form Kernel Mean Embeddings

    A Dictionary of Closed-Form Kernel Mean Embeddings arXiv:2504.18830v1 Announce Type: new Abstract: Kernel mean embeddings — integrals of a kernel with respect to a probability distribution — are essential in Bayesian quadrature, but also widely used in other computational tools for numerical integration or for statistical inference based on the maximum mean discrepancy. These methods…

  • On the minimax optimality of Flow Matching through the connection to kernel density estimation

    On the minimax optimality of Flow Matching through the connection to kernel density estimation arXiv:2504.13336v1 Announce Type: new Abstract: Flow Matching has recently gained attention in generative modeling as a simple and flexible alternative to diffusion models, the current state of the art. While existing statistical guarantees adapt tools from the analysis of diffusion models,…

  • On the Convergence of Irregular Sampling in Reproducing Kernel Hilbert Spaces

    On the Convergence of Irregular Sampling in Reproducing Kernel Hilbert Spaces arXiv:2504.13623v1 Announce Type: new Abstract: We analyse the convergence of sampling algorithms for functions in reproducing kernel Hilbert spaces (RKHS). To this end, we discuss approximation properties of kernel regression under minimalistic assumptions on both the kernel and the input data. We first prove…

  • Smoothed Distance Kernels for MMDs and Applications in Wasserstein Gradient Flows

    Smoothed Distance Kernels for MMDs and Applications in Wasserstein Gradient Flows arXiv:2504.07820v1 Announce Type: new Abstract: Negative distance kernels $K(x,y) := – |x-y|$ were used in the definition of maximum mean discrepancies (MMDs) in statistics and lead to favorable numerical results in various applications. In particular, so-called slicing techniques for handling high-dimensional kernel summations profit…

  • Bayesian Kernel Regression for Functional Data

    Bayesian Kernel Regression for Functional Data arXiv:2503.13676v1 Announce Type: new Abstract: In supervised learning, the output variable to be predicted is often represented as a function, such as a spectrum or probability distribution. Despite its importance, functional output regression remains relatively unexplored. In this study, we propose a novel functional output regression model based on…

  • ROCK: A variational formulation for occupation kernel methods in Reproducing Kernel Hilbert Spaces

    ROCK: A variational formulation for occupation kernel methods in Reproducing Kernel Hilbert Spaces arXiv:2503.13791v1 Announce Type: new Abstract: We present a Representer Theorem result for a large class of weak formulation problems. We provide examples of applications of our formulation both in traditional machine learning and numerical methods as well as in new and emerging…

  • A Practical Introduction to Kernel Discrepancies: MMD, HSIC & KSD

    A Practical Introduction to Kernel Discrepancies: MMD, HSIC & KSD arXiv:2503.04820v1 Announce Type: new Abstract: This article provides a practical introduction to kernel discrepancies, focusing on the Maximum Mean Discrepancy (MMD), the Hilbert-Schmidt Independence Criterion (HSIC), and the Kernel Stein Discrepancy (KSD). Various estimators for these discrepancies are presented, including the commonly-used V-statistics and U-statistics,…

  • Multi-View Oriented GPLVM: Expressiveness and Efficiency

    Multi-View Oriented GPLVM: Expressiveness and Efficiency arXiv:2502.08253v1 Announce Type: new Abstract: The multi-view Gaussian process latent variable model (MV-GPLVM) aims to learn a unified representation from multi-view data but is hindered by challenges such as limited kernel expressiveness and low computational efficiency. To overcome these issues, we first introduce a new duality between the spectral…

  • Diagonal Over-parameterization in Reproducing Kernel Hilbert Spaces as an Adaptive Feature Model: Generalization and Adaptivity

    Diagonal Over-parameterization in Reproducing Kernel Hilbert Spaces as an Adaptive Feature Model: Generalization and Adaptivity arXiv:2501.08679v1 Announce Type: cross Abstract: This paper introduces a diagonal adaptive kernel model that dynamically learns kernel eigenvalues and output coefficients simultaneously during training. Unlike fixed-kernel methods tied to the neural tangent kernel theory, the diagonal adaptive kernel model adapts…

  • Unified Native Spaces in Kernel Methods

    Unified Native Spaces in Kernel Methods arXiv:2501.01825v1 Announce Type: new Abstract: There exists a plethora of parametric models for positive definite kernels, and their use is ubiquitous in disciplines as diverse as statistics, machine learning, numerical analysis, and approximation theory. Usually, the kernel parameters index certain features of an associated process. Amongst those features, smoothness…