NTH

PReMind: When Personal Memory Meets Visual Hazards

AuthorsYeongtak Oh, Donghyeok Lee, Joonhee Oh, Jaewon Jeong, Sungroh Yoon

AffiliationsDepartment of Electrical and Computer Engineering, Seoul National University · Interdisciplinary Program in AI, Seoul National University

November 21, 2026 2 min read
Watch on YouTube
The one-line take

PReMind tests and trains visual assistants to connect what they see with the personal context needed to give safer advice.

Key results

1024
PReMind-Bench test requests

Total requests in the main test set.

140
Interdependency cases

Cases requiring a hazard response informed by a personal fact.

12.9%
Base Qwen3.5-4B joint success

Joint success on the interdependency set before PReMind training.

28.6%
PReMind-4B joint success

Joint success on the interdependency set after training.

34.4%
Base acute-hazard adequacy

Adequacy on acute hazards before PReMind training.

56.9%
PReMind acute-hazard adequacy

Adequacy on acute hazards after PReMind training.

What the paper found

PReMind targets a failure in personalized visual assistants: safe advice can depend on both what an image shows and what the assistant remembers about the specific person. The paper introduces PReMind-Bench, with 1024 test requests and 140 interdependency cases that require combining a visible hazard with a person-specific fact. It separately scores memory retrieval, identity grounding, abstention, hazard response, and joint success. PReMind trains Qwen3.5-4B with two-phase, group-relative DAPO reinforcement learning, mixing on-demand memory calls with forced-call episodes so the model learns both when to retrieve and how to use retrieved facts. Phase II adds difficulty-based curriculum, replay of memory-use failures, and credit for scene-specific hazard actions. On the interdependency set, joint success rises from 12.9% for the base to 28.6% for PReMind; on acute hazards, response adequacy increases from 34.4% to 56.9%. The gains persist under forced-call evaluation and a Claude Sonnet 5.5 cross-check. Evaluation uses OpenAI’s GPT-6-Luna as the primary judge, and compares against systems including Anthropic’s Claude Opus 5.5. The results show that jointly training personalization and situational safety helps, while the model still over-flags some safe requests and does not establish reduced harm on interdependency cases.

Original abstract

Personalized vision-language assistants use visual observations and personal memories to provide advice tailored to the user. When the observed scene contains a physical hazard, personal context, such as allergies, medical conditions, or mobility constraints, can change what help is appropriate. Responding appropriately therefore requires recognizing the visible hazard and applying the relevant person's information. We introduce PReMind-Bench, a diagnostic benchmark that evaluates this combination across diverse hazards and personal-memory conditions. It separately measures memory access, grounding, abstention, and hazard response, and assesses joint success on interdependency cases. Existing models often satisfy one requirement while failing the other, and training only for safety or personalization leaves a gap in joint success. We therefore propose PReMind, a two-phase reinforcement learning framework that combines on-demand and forced-call interactions (memory always retrieved) to jointly train grounded memory use and hazard response. At 4B, PReMind raises joint success from 12.9% to 28.6% over its base on the interdependency set and improves hazard adequacy and correct abstention on the main test. Joint-success gains persist under forced-call evaluation and under a second LLM judge across three Phase II seeds; averaged over these seeds, joint success also improves on unseen hazard groups. These findings highlight the need to evaluate and train personal assistants to connect visual evidence with the relevant person's context.

Read the original paper

More in AI Benchmarks

Browse all 60 papers →