The Provenance Tax: How LLM Watermarking Changes AI Agent Behavior

LLM watermarking designed to signal AI-generated text can alter token-by-token generation and thus change both what a model says and what an agent built around it does. Deployments based on SynthID-Text use tournament sampling to embed a keyed, generation-time signal that is non-distortionary in expectation but can produce different outputs for any fixed key. Because agents act on the same sampled tokens (function names, JSON values, arguments), even small lexical shifts can flip tool selection, arguments, or whether a model refuses a harmful request. Prompt injection exacerbates the risk: weakened refusals become more consequential when the model can call tools.

Paired experiments used a non-distortionary SynthID-Text configuration via HuggingFace and compared watermarked versus unwatermarked runs on BFCL v4 single-turn AST (1,150 non-live call-expected items) and on HarmBench (200 harmful) plus JailbreakBench (100 controls) across temperatures 0.001-1.0. Watermarking reduced tool-call accuracy in six of seven models (significantly in four) and produced much higher paired disagreement (“churn”) than net accuracy changes - for example, phi-4 at T=1.0 showed 16.8% churn but only a 2.87-point net accuracy loss. Average churn across 21 model-temperature combinations was 6.5%. Error profiles varied by model (e.g., Llama-3.1-8B lost accuracy mainly from wrong arguments, phi-4 and Granite from malformed output). Watermark keys matter, and overall results imply a measurable “provenance tax” on agent behavior that developers must test for and mitigate.