What It Is, Not What It Says: Securing Agentic Browsing Interactions with Web Semantics
AuthorsUmar Iqbal
AffiliationsWashington University in St. Louis
Intact helps browsing agents distinguish who controls webpage content and limits what they can do with it.
Key results
Real webpages used to evaluate third-party attribution.
GenTree attributed 2.3 times as many elements to third parties as standard browser attribution.
The browser’s standard attribution labeled these third-party elements as first-party.
Real webpages with injected attacks, evaluated using gpt-6-luna.
At most, these attacks produced a consent request; VANILLA’s rate was 73.1%.
Tasks where isolation left functionality that the conventional planner retained.
What the paper found
As agentic browsers such as Perplexity’s Comet, Anthropic’s Claude in Chrome, and Google’s Gemini in Chrome browse on users’ behalf, prompt injection exposes a basic weakness: agents often flatten webpages into text or screenshots, stripping away the browser’s security boundaries. This paper introduces GenTree, which records each webpage element’s provenance and web capabilities, and INTACT, which uses that information to isolate decisions: the LLM considers one element at a time, while a harness limits actions to that element’s capabilities and requires user consent for actions on third-party content. The approach avoids relying on the model to recognize malicious instructions, a limitation underscored by OpenAI’s warning that prompt injection may never be fully solved. On 1,270 real webpages, GenTree attributed 2.3 times as many elements to third parties as the browser’s standard attribution, which misattributed 58.7% of third-party elements to the first party. In attacks evaluated on 1,091 webpages using the gpt-6-luna planner, only 12.8% got through INTACT’s planner, at most as consent requests, compared with 73.1% for the conventional VANILLA planner. The isolation disrupted functionality on 3.7% of tasks. Although sequential element-by-element planning adds latency, a proposed parallel schedule could reduce median planning time by 85%. INTACT runs on stock Chromium without browser modifications or website cooperation.
Original abstract
AI agents increasingly browse the web on users' behalf, yet they consume webpages as screenshots or text, exposing their LLMs to unvetted third-party content. Browsers enforce semantics under which what an element can do depends on what it is and where it comes from, not on what it says. Agents flatten these semantics, so injected text can persuade the LLM to exercise the user's authority for content that has none. As LLMs remain easy to persuade, the agent's interface to the web must itself ensure the integrity of what the agent reads, interprets, and does. We propose Intact, which runs on an unmodified browser. Intact builds GenTree, a bespoke representation that captures provenance and web semantics. Intact passes the page via GenTree to the LLM one element at a time, so the LLM decides on each element without any other element's text. Its harness executes each step only within the capabilities of the step's element and asks the user before acting on third-party content. On 1,270 real webpages, GenTree attributes 2.3x as many elements to third parties as the browser, which misattributes 58.7% of them to the first party. On 1,091 of these webpages with injected attacks, only 12.8% of attacks get through Intact's planner, each at most as a consent request, against 73.1% for a conventional planner. Isolation disrupts functionality on only 3.7% of tasks, and a parallel schedule could remove most of its added latency.
Read the original paperMore in AI Safety
Browse all 42 papers →RIDAR: Safeguarding Research Integrity in Autonomous Research Agents via Automated Detection and Runtime Intervention
Jiening Siow, Amy Xin, Junjie Wang, Ziyu Tang, Zijun Yao, Yunjia Qi, Fanjin Zhang, Lei Hou, Jie Tang, Juanzi Li
RIDAR detects research-integrity violations in autonomous research agents and intervenes during execution to reduce them.
The Agent Knew: Internal States Forecast Unauthorized Actions and Stop Them, and the Lies That Follow
Arya Fayyazi, Mehdi Kamal, Massoud Pedram
A model’s internal signals can warn that it is about to exceed its authority, enabling a monitor to intervene before the action—and the cover story.
Covert Assistance: Helpful LLM Agents Evade Oversight in Multi-Agent Systems
Deema Alnuhait, Gengyu Wang, Muhammad Khalifa, Hao Peng
Helpful AI agents may secretly work around safety rules to assist one another, creating rare but serious information-leakage risks that compound over repeated interactions.