Re-Centering Humans in LLM Personalization

Re-Centering Humans in LLM Personalization, from the University of Illinois Urbana-Champaign and Carnegie Mellon University, argues that synthetic personas and LLM judges systematically overstate how well models personalize for real people. The paper decomposes personalization into three stages: extracting stable user attributes from conversation history, matching those attributes to a new prompt, and generating a response that beats a generic baseline. Using 550 real conversations from WildChat and 5,949 attribute-validity judgments, 11,919 attribute–prompt relevance judgments, and 1,101 response-preference judgments, the authors show that real conversations are harder than synthetic data: extracted attributes from human dialogue contain many more uncertain or rejected cases, with overgeneralization accounting for 53.9% of failures. In relevance matching, humans mark only about 20% of attributes as relevant, while LLM judges mark 40–60%, with average human–LLM agreement dropping to κ≈0.30. Lightweight interventions help: a trained RoBERTa verifier reaches 0.726 F1 for attribute verification, and a Qwen3-4B model optimized with GRPO reaches 0.641 F1 for relevance selection. But the third stage remains difficult: humans judge 54.6% of personalized responses as no better than generic, and LLM judges show only modest alignment with human ratings, with Spearman correlation peaking around 0.376. The core message is that personalization quality must be grounded in human judgments, not just synthetic benchmarks or surface-level attribute mentions.