Tag: experts
-
Horseshoe Mixtures-of-Experts (HS-MoE)
Horseshoe Mixtures-of-Experts (HS-MoE) arXiv:2601.09043v1 Announce Type: new Abstract: Horseshoe mixtures-of-experts (HS-MoE) models provide a Bayesian framework for sparse expert selection in mixture-of-experts architectures. We combine the horseshoe prior’s adaptive global-local shrinkage with input-dependent gating, yielding data-adaptive sparsity in expert usage. Our primary methodological contribution is a particle learning algorithm for sequential inference, in which the…
-
DriftMoE: A Mixture of Experts Approach to Handle Concept Drifts
DriftMoE: A Mixture of Experts Approach to Handle Concept Drifts arXiv:2507.18464v1 Announce Type: new Abstract: Learning from non-stationary data streams subject to concept drift requires models that can adapt on-the-fly while remaining resource-efficient. Existing adaptive ensemble methods often rely on coarse-grained adaptation mechanisms or simple voting schemes that fail to optimally leverage specialized knowledge. This…
-
From Fuzzy to Precise: How a Morphological Feature Extractor Enhances AI’s Recognition Capabilities
From Fuzzy to Precise: How a Morphological Feature Extractor Enhances AI’s Recognition Capabilities Introduction: Can AI really distinguish dog breeds like human experts? One day while taking a walk, I saw a fluffy white puppy and wondered, Is that a Bichon Frise or a Maltese? No matter how closely I looked, they seemed almost identical.…
-
From Fuzzy to Precise: How a Morphological Feature Extractor Enhances AI’s Recognition Capabilities
From Fuzzy to Precise: How a Morphological Feature Extractor Enhances AI’s Recognition Capabilities Introduction: Can AI really distinguish dog breeds like human experts? One day while taking a walk, I saw a fluffy white puppy and wondered, Is that a Bichon Frise or a Maltese? No matter how closely I looked, they seemed almost identical.…
-
Convergence Rates for Softmax Gating Mixture of Experts
Convergence Rates for Softmax Gating Mixture of Experts arXiv:2503.03213v1 Announce Type: new Abstract: Mixture of experts (MoE) has recently emerged as an effective framework to advance the efficiency and scalability of machine learning models by softly dividing complex tasks among multiple specialized sub-models termed experts. Central to the success of MoE is an adaptive softmax…
-
Composition of Experts: A Modular Compound AI System Leveraging Large Language Models
Composition of Experts: A Modular Compound AI System Leveraging Large Language Models arXiv:2412.01868v1 Announce Type: cross Abstract: Large Language Models (LLMs) have achieved remarkable advancements, but their monolithic nature presents challenges in terms of scalability, cost, and customization. This paper introduces the Composition of Experts (CoE), a modular compound AI system leveraging multiple expert LLMs.…