mailitics

Tag: feature

  • Scaling Feature Engineering Pipelines with Feast and Ray

    Scaling Feature Engineering Pipelines with Feast and Ray Utilizing feature stores like Feast and distributed compute frameworks like Ray in production machine learning systems The post Scaling Feature Engineering Pipelines with Feast and Ray appeared first on Towards Data Science. Kenneth Leung Go to original source

  • ROOFS: RObust biOmarker Feature Selection

    ROOFS: RObust biOmarker Feature Selection arXiv:2601.05151v1 Announce Type: new Abstract: Feature selection (FS) is essential for biomarker discovery and in the analysis of biomedical datasets. However, challenges such as high-dimensional feature space, low sample size, multicollinearity, and missing values make FS non-trivial. Moreover, FS performances vary across datasets and predictive tasks. We propose roofs, a…

  • Feature Detection, Part 3: Harris Corner Detection

    Feature Detection, Part 3: Harris Corner Detection Finding the most informative points in images The post Feature Detection, Part 3: Harris Corner Detection appeared first on Towards Data Science. Vyacheslav Efimov Go to original source

  • Unsupervised Feature Selection via Robust Autoencoder and Adaptive Graph Learning

    Unsupervised Feature Selection via Robust Autoencoder and Adaptive Graph Learning arXiv:2512.18720v1 Announce Type: new Abstract: Effective feature selection is essential for high-dimensional data analysis and machine learning. Unsupervised feature selection (UFS) aims to simultaneously cluster data and identify the most discriminative features. Most existing UFS methods linearly project features into a pseudo-label space for clustering,…

  • Provable FDR Control for Deep Feature Selection: Deep MLPs and Beyond

    Provable FDR Control for Deep Feature Selection: Deep MLPs and Beyond arXiv:2512.04696v1 Announce Type: new Abstract: We develop a flexible feature selection framework based on deep neural networks that approximately controls the false discovery rate (FDR), a measure of Type-I error. The method applies to architectures whose first layer is fully connected. From the second…

  • Shap or LGBM gain for feature selection?

    Shap or LGBM gain for feature selection? Which one do you use during recursive feature elimination or forward/backward selection? I’ve always used gain and only used shap for analytics on model predictions, but came across some shap values recommendations. Bonus question: have you used “null importance” / permutation method? Fitting models with shuffled targets to…

  • When Features Beat Noise: A Feature Selection Technique Through Noise-Based Hypothesis Testing

    When Features Beat Noise: A Feature Selection Technique Through Noise-Based Hypothesis Testing arXiv:2511.20851v1 Announce Type: new Abstract: Feature selection has remained a daunting challenge in machine learning and artificial intelligence, where increasingly complex, high-dimensional datasets demand principled strategies for isolating the most informative predictors. Despite widespread adoption, many established techniques suffer from notable limitations; some…

  • Feature Detection, Part 1: Image Derivatives, Gradients, and Sobel Operator

    Feature Detection, Part 1: Image Derivatives, Gradients, and Sobel Operator Applying calculus fundamentals to computer vision for edge detection The post Feature Detection, Part 1: Image Derivatives, Gradients, and Sobel Operator appeared first on Towards Data Science. Vyacheslav Efimov Go to original source

  • Sparse minimum Redundancy Maximum Relevance for feature selection

    Sparse minimum Redundancy Maximum Relevance for feature selection arXiv:2508.18901v1 Announce Type: new Abstract: We propose a feature screening method that integrates both feature-feature and feature-target relationships. Inactive features are identified via a penalized minimum Redundancy Maximum Relevance (mRMR) procedure, which is the continuous version of the classic mRMR penalized by a non-convex regularizer, and where…

  • Decorrelated feature importance from local sample weighting

    Decorrelated feature importance from local sample weighting arXiv:2508.06337v1 Announce Type: new Abstract: Feature importance (FI) statistics provide a prominent and valuable method of insight into the decision process of machine learning (ML) models, but their effectiveness has well-known limitations when correlation is present among the features in the training data. In this case, the FI…

  • Subgrid BoostCNN: Efficient Boosting of Convolutional Networks via Gradient-Guided Feature Selection

    Subgrid BoostCNN: Efficient Boosting of Convolutional Networks via Gradient-Guided Feature Selection arXiv:2507.22842v1 Announce Type: new Abstract: Convolutional Neural Networks (CNNs) have achieved remarkable success across a wide range of machine learning tasks by leveraging hierarchical feature learning through deep architectures. However, the large number of layers and millions of parameters often make CNNs computationally expensive…

  • GOLFS: Feature Selection via Combining Both Global and Local Information for High Dimensional Clustering

    GOLFS: Feature Selection via Combining Both Global and Local Information for High Dimensional Clustering arXiv:2507.10956v1 Announce Type: new Abstract: It is important to identify the discriminative features for high dimensional clustering. However, due to the lack of cluster labels, the regularization methods developed for supervised feature selection can not be directly applied. To learn the…

  • Disentangled Feature Importance

    Disentangled Feature Importance arXiv:2507.00260v1 Announce Type: new Abstract: Feature importance quantification faces a fundamental challenge: when predictors are correlated, standard methods systematically underestimate their contributions. We prove that major existing approaches target identical population functionals under squared-error loss, revealing why they share this correlation-induced bias. To address this limitation, we introduce emph{Disentangled Feature Importance (DFI)},…

  • AICO: Feature Significance Tests for Supervised Learning

    AICO: Feature Significance Tests for Supervised Learning arXiv:2506.23396v1 Announce Type: new Abstract: The opacity of many supervised learning algorithms remains a key challenge, hindering scientific discovery and limiting broader deployment — particularly in high-stakes domains. This paper develops model- and distribution-agnostic significance tests to assess the influence of input features in any regression or classification…

  • Random feature approximation for general spectral methods

    Random feature approximation for general spectral methods arXiv:2506.16283v1 Announce Type: new Abstract: Random feature approximation is arguably one of the most widely used techniques for kernel methods in large-scale learning algorithms. In this work, we analyze the generalization properties of random feature methods, extending previous results for Tikhonov regularization to a broad class of spectral…

  • Feature Representation Transferring to Lightweight Models via Perception Coherence

    Feature Representation Transferring to Lightweight Models via Perception Coherence arXiv:2505.06595v1 Announce Type: new Abstract: In this paper, we propose a method for transferring feature representation to lightweight student models from larger teacher models. We mathematically define a new notion called textit{perception coherence}. Based on this notion, we propose a loss function, which takes into account…

  • Explained: How Does L1 Regularization Perform Feature Selection?

    Explained: How Does L1 Regularization Perform Feature Selection? Feature Selection is the process of selecting an optimal subset of features from a given set of features; an optimal feature subset is the one which maximizes the performance of the model on the given task. Feature selection can be a manual or rather explicit process when…

  • Clustering Items through Bandit Feedback: Finding the Right Feature out of Many

    Clustering Items through Bandit Feedback: Finding the Right Feature out of Many arXiv:2503.11209v1 Announce Type: new Abstract: We study the problem of clustering a set of items based on bandit feedback. Each of the $n$ items is characterized by a feature vector, with a possibly large dimension $d$. The items are partitioned into two unknown…

  • Formulation of Feature Circuits with Sparse Autoencoders in LLM

    Formulation of Feature Circuits with Sparse Autoencoders in LLM Large Language models (LLMs) have witnessed impressive progress and these large models can do a variety of tasks, from generating human-like text to answering questions. However, understanding how these models work still remains challenging, especially due a phenomenon called superposition where features are mixed into one…

  • A Differentiable Rank-Based Objective For Better Feature Learning

    A Differentiable Rank-Based Objective For Better Feature Learning arXiv:2502.09445v1 Announce Type: new Abstract: In this paper, we leverage existing statistical methods to better understand feature learning from data. We tackle this by modifying the model-free variable selection method, Feature Ordering by Conditional Independence (FOCI), which is introduced in cite{azadkia2021simple}. While FOCI is based on a…

  • ML Feature Management: A Practical Evolution Guide

    ML Feature Management: A Practical Evolution Guide In the world of machine learning, we obsess over model architectures, training pipelines, and hyper-parameter tuning, yet often overlook a fundamental aspect: how our features live and breathe throughout their lifecycle. From in-memory calculations that vanish after each prediction to the challenge of reproducing exact feature values months…

  • Random Feature Representation Boosting

    Random Feature Representation Boosting arXiv:2501.18283v1 Announce Type: new Abstract: We introduce Random Feature Representation Boosting (RFRBoost), a novel method for constructing deep residual random feature neural networks (RFNNs) using boosting theory. RFRBoost uses random features at each layer to learn the functional gradient of the network representation, enhancing performance while preserving the convex optimization benefits…

  • Explaining Categorical Feature Interactions Using Graph Covariance and LLMs

    Explaining Categorical Feature Interactions Using Graph Covariance and LLMs arXiv:2501.14932v1 Announce Type: new Abstract: Modern datasets often consist of numerous samples with abundant features and associated timestamps. Analyzing such datasets to uncover underlying events typically requires complex statistical methods and substantial domain expertise. A notable example, and the primary data focus of this paper, is…

  • Statistical Inference for Sequential Feature Selection after Domain Adaptation

    Statistical Inference for Sequential Feature Selection after Domain Adaptation arXiv:2501.09933v1 Announce Type: new Abstract: In high-dimensional regression, feature selection methods, such as sequential feature selection (SeqFS), are commonly used to identify relevant features. When data is limited, domain adaptation (DA) becomes crucial for transferring knowledge from a related source domain to a target domain, improving…

  • Superposition: What Makes it Difficult to Explain Neural Network

    Superposition: What Makes it Difficult to Explain Neural Network When there are more features than model dimensions Introduction It would be ideal if the world of neural network represented a one-to-one relationship: each neuron activates on one and only one feature. In such a world, interpreting the model would be straightforward: this neuron fires for…

  • Feature creation out of two features.

    Feature creation out of two features. I have been working on a project that tried to identify interactions in variables. What is a good way to capture these interactions by creating features? What are good mathematical expressions to capture interaction beyond multiplication and division? Do note i have nulls and i cannot change it. submitted…