The SynthID-Text watermark required by the AI Act changes how agents call tools and refuse requests

September 17, 2026. Lasso Security publishes a study on the effect of watermarking AI-generated text. The verdict is nuanced: the SynthID-Text watermark, adopted by Anthropic and OpenAI to comply with the EU AI Act, changes how AI agents call their tools and hold their refusals. Why it matters: a feature designed for provenance behaves like a model modification, and the effect intensifies under prompt injection.

A watermark that is more than a marking

The principle behind SynthID-Text, developed by Google DeepMind, is to intervene in next-token prediction. Where a model might hesitate between two statistically close words — “overcast” or “gray” to describe a sky — the watermark slightly favors one candidate. Repeated hundreds of times across a text, that bias forms a pattern that is undetectable to a reader but readable to anyone holding the decoding key.

The mechanism is now a legal obligation. With the AI Act coming into force, model providers must mark their output with machine-readable code. SynthID-Text is one of the accepted methods, adopted by both Anthropic and OpenAI. The benefit is real: deceptive AI-generated content becomes easier to detect. But the cost, until now, had not been measured.

Why an agent is sensitive to a single word choice

Lasso’s starting observation is simple: a human reader does not notice the vocabulary bias, but an AI agent is subtly sensitive to it. An agent is not a passive reader; it turns the model’s output into actions — function calls, tool selection, arguments. Those decisions depend on exactly which words the model picked. By shifting low-level word choices, the watermark indirectly changes “both what the model says and what an agent does,” in the report’s phrasing.

The benchmark used, BFCL v4, evaluates tool calls at the abstract syntax tree (AST) level: it does not just check that the agent responded, but that the call is syntactically correct and that the function and its arguments are right. That is precisely where the watermark bites. A less accurate agent picks the wrong tool, or the right tool with the wrong arguments, and fails on malformed input or parsing errors. The failures are concrete: an agent that should query a database instead writes a file, or passes a string where the tool expected a number, and the whole call chain breaks downstream — errors that a surface-level “did it answer?” check would never surface.

What the study measured

Lasso Security tested the watermark’s impact on two sensitive behaviors: tool calling and refusals. On the BFCL v4 single-turn benchmark, the watermark reduced tool-calling accuracy on six of seven models tested — phi-4, Llama-3.1-8B, Qwen3-32B, Qwen3-4B, gemma-3-12b, gemma-3-27b, and Granite-3.2-8B. The models span the open-weights ecosystem rather than a single vendor, which matters: the effect is not a quirk of one architecture but a property of how watermarking perturbs token sampling in general.

The methodology matters as much as the number. “The net change in accuracy does not show whether the same individual calls succeed with and without the watermark,” Lasso explains. “A call that becomes incorrect can be offset by another that becomes correct, leaving the aggregate result nearly unchanged even though the model behaves differently on both items.” In other words, the average can stay flat while the individual errors move around — a classic trap for anyone watching a global metric without looking at the distribution.

Prompt injection is the aggravating factor

The most concerning effect appears in the adversarial scenario. On plainly harmful requests, the watermark has only a small effect on refusals — measured on HarmBench and JailbreakBench. But under prompt injection, when an adversarial instruction claims the safety filter has been disabled and compliance is required, the attack success rate rises significantly when the content is watermarked.

Lasso’s conclusion is usefully sober: the affected models are less likely to refuse harmful requests in that configuration. The watermark is therefore not security-neutral — it interacts with the hardest-to-defend surface of agentic systems, the one where an adversary tries to push the model out of its role.

Production evaluation ignores the watermark

There is a structural consequence the study highlights without naming it: most public benchmarks run on non-watermarked checkpoints. Yet a provider deploying an AI Act-compliant endpoint serves watermarked content. The gap between the model you evaluated and the model you ship is no longer just about version or quantization: it is about the watermark.

The decisive point is that the effect does not stop at the watermarking vendor. An agent built on OpenClaw, or an API client calling an Anthropic model, processes the output variation produced by the vendor’s watermarking. A third party’s provenance compliance therefore propagates into your agent’s behavior — without your team changing a single line of its pipeline.

What this changes for anyone deploying agents

Lasso’s recommendation follows directly: security evaluations and red-teaming must include watermarked content when assessing agent deployment. Testing a “naked” model to authorize a watermarked agent in production means validating a behavior that is not the one you are shipping.

Concretely, three practices cover most of it. First, require the provider to expose the watermarking state of the endpoint — enabled, disabled, schema version — on the same footing as the model version. Second, build your regression suites on the same artifact that will serve production, watermark included. Third, add a prompt-injection test set to your red-teaming, because that is where the behavioral gap is most pronounced.

A provenance trade-off that now has a number

Stepping back, the finding lands in the middle of a genuine policy tension. The AI Act pushes for machine-readable provenance because traceability is the prerequisite for holding synthetic content accountable — detecting deepfakes, exposing coordinated inauthentic behavior, and giving platforms a way to flag generated media. Lasso is not arguing against watermarking; it is arguing that the technique carries a measurable safety cost that evaluations must account for. The honest reading is that provenance and behavior are no longer independent dials: turning one up can move the other, and the interaction is strongest exactly where agents are most exposed — under adversarial instruction. A regulator or a security team that treats the watermark as free will miss the trade-off entirely.

Verdict

If you deploy AI agents in production, add one line to your evaluation process: re-test tool calls and refusals with the watermark enabled, because that is what your customers will actually see — and what your adversaries will actually attack. If you are evaluating a model for an agentic integration, ask for the watermarking state of the endpoint before you sign, and reject any provider that cannot tell you whether its endpoint serves marked content. If you fall under the AI Act, treat provenance compliance as a behavior change to validate, not as an inert regulatory checkbox: a watermark that weakens refusals under prompt injection is a measurable risk, and it should be measured before production, not after.

References

• The Register — AI model watermarking changes agent behavior, September 17, 2026
• Nature — Scalable watermarking for identifying large language model outputs (SynthID-Text)
• Anthropic — Claude text watermark
• OpenAI — Provenance signals: Content Credentials and SynthID in OpenAI-generated content
• ETTAYEB — Shadow AI: when employees exfiltrate data to SaaS models