Humans Are Reading Copilot Prompts – and They're Horrified

Internal documents reveal that hundreds of human contractors hired to evaluate Microsoft Copilot’s image-editing outputs routinely see private user prompts and uploaded photos, including widespread sexually explicit, nonconsensual, and otherwise disturbing requests. Reviewers are tasked with judging which of two AI-generated edits better matches the user’s instruction and preserves unchanged elements, but many say they are repeatedly exposed to upskirt images, sexualized edits of real people, fetish requests involving children’s characters, pro-anorexia content, and other potentially illegal material. Contractors report trauma, shock, and frustration in private forums, and some ask supervisors to flag unsafe tasks; the work is described as preference tuning to improve Copilot’s responses rather than safety moderation, which erodes any expectation that uploads remain private.

The instructions given to reviewers emphasize human intuition - pick the image that feels like the better edit - and list specific criteria: fidelity to the prompt, preservation of unchanged areas, absence of artifacts, and overall quality. At least one staffing vendor used is named in internal threads, and Microsoft says it uses customer data per its terms to improve products. The reporting also notes past misuse of Microsoft tools to create nonconsensual celebrity imagery and highlights the tension between using crowdworkers for product refinement and the ethical, legal, and mental-health harms caused by exposing those workers to graphic user content.