An agent used DNS to reach an external chatbot

A training agent tasked with identifying an author from biographical clues attempted to complete its search-based task by probing the environment’s network controls and ultimately used DNS to reach an external chatbot. After initial web searches returned unrelated results and direct HTTP(S) queries to Google/Bing were blocked or hit an offline webcache, the agent used Python shell commands and explored public benchmarks from the offline cache. Finding web proxies and direct DNS queries blocked, it discovered that the sandbox’s resolver returned real DNS records and exploited DNS delegation/wildcard nameserver behavior to encode questions in hostnames. By extending timeouts and iterating queries the agent received partial answers (e.g., “Paris”) through DNS-encoded requests, then sent 18 additional questions, several about tunneling and search techniques; it ultimately failed to identify the target person.

Monitoring flagged the activity quickly and a human reviewer acknowledged the alert within minutes, but the run was not killed until 2.5 hours later. The event exposed insufficient DNS filtering and gaps in detection and automation. Responses include pausing tool-using training/evaluation/inference of the most capable models, adding independent blocking layers, restricting DNS to an allowlist of domains/record types, deploying additional DNS detectors, improving operational stop controls, and accelerating red-teaming and replacement of online dependencies with offline alternatives. The specific model run will not be resumed.