Understanding the Impact of LLM Watermarking on AI Agent Behavior

Watermarking techniques like SynthID-Text, now being deployed at the model level, inject a token-selection bias during generation to enable machine-detectable provenance. Because tournament-based watermarking changes sampled tokens under a fixed key (even in its non-distortionary configuration), it can alter both what a model says and what an agent does: safety refusals are generated token-by-token and agent actions (tool calls, arguments) are driven by the same tokens. To measure this, paired experiments compared watermarked and unwatermarked generations using SynthID’s non-distortionary setup (30 tournament layers, n-gram length 5, sampling table 216, context 1,024) across temperatures (tool-calling at T=0.001, 0.7, 1.0; refusals at T=0.001, 0.7). Tool-calling was evaluated on BFCL v4 single-turn AST tasks and refusals on HarmBench plus JailbreakBench controls, with identical seeds and batch ordering for each pair.

Results show measurable sampling drift: watermarking reduced tool-call accuracy for six of seven models and produced substantial paired disagreement (“churn”) far larger than net accuracy changes (e.g., phi‑4 at T=1.0 had 16.8% churn but only a 2.87‑point net accuracy loss), averaging 6.5% churn across combinations. Error modes varied by model - wrong arguments, wrong tool, or malformed output - so similar aggregate scores can mask different risks. Watermarking also weakened refusals under a fixed prompt‑injection technique. The findings imply provenance watermarking can meaningfully affect agent safety and behavior; developers should test multiple keys, temperatures, and downstream agent actions rather than relying on aggregate metrics alone.