mailitics

Tag: preference

  • Fisher Random Walk: Automatic Debiasing Contextual Preference Inference for Large Language Model Evaluation

    Fisher Random Walk: Automatic Debiasing Contextual Preference Inference for Large Language Model Evaluation arXiv:2509.05852v1 Announce Type: new Abstract: Motivated by the need for rigorous and scalable evaluation of large language models, we study contextual preference inference for pairwise comparison functionals of context-dependent preference score functions across domains. Focusing on the contextual Bradley-Terry-Luce model, we develop…

  • Preference Models assume Proportional Hazards of Utilities

    Preference Models assume Proportional Hazards of Utilities arXiv:2508.13189v1 Announce Type: new Abstract: Approaches for estimating preferences from human annotated data typically involves inducing a distribution over a ranked list of choices such as the Plackett-Luce model. Indeed, modern AI alignment tools such as Reward Modelling and Direct Preference Optimization are based on the statistical assumptions…

  • Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory

    Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory arXiv:2506.12350v1 Announce Type: new Abstract: Despite its empirical success, Reinforcement Learning from Human Feedback (RLHF) has been shown to violate almost all the fundamental axioms in social choice theory — such as majority consistency, pairwise majority consistency, and Condorcet consistency. This raises…

  • Combinatorial Reinforcement Learning with Preference Feedback

    Combinatorial Reinforcement Learning with Preference Feedback arXiv:2502.10158v1 Announce Type: new Abstract: In this paper, we consider combinatorial reinforcement learning with preference feedback, where a learning agent sequentially offers an action–an assortment of multiple items to–a user, whose preference feedback follows a multinomial logistic (MNL) model. This framework allows us to model real-world scenarios, particularly those…

  • Preference-based Pure Exploration

    Preference-based Pure Exploration arXiv:2412.02988v1 Announce Type: new Abstract: We study the preference-based pure exploration problem for bandits with vector-valued rewards. The rewards are ordered using a (given) preference cone $mathcal{C}$ and our the goal is to identify the set of Pareto optimal arms. First, to quantify the impact of preferences, we derive a novel lower…