hn.today

Harsher on Male? Evaluating LLMs on Gender-Asymmetric Moral Framing in Conflicts

arxiv.org76 points63 comments
Screenshot of Harsher on Male? Evaluating LLMs on Gender-Asymmetric Moral Framing in Conflicts

The study examines whether language models judge identical misconduct differently when the person responsible is presented as male or female. It introduces GAMA-Bench, a benchmark of 1,298 scenarios spanning intimate relationships and public social conflicts. The scenarios use gender-neutral misconduct templates, reviewed across models, then turn them into paired first-person prompts that vary the actor’s gender and role references while keeping the behavior matched. A structured evaluation protocol tracks how responses assign punishment, empathy, escalation, advice, and blame, allowing the researchers to assess not just whether models condemn misconduct but how they frame their response.

Tests of 10 representative language models show a consistent imbalance: male actors receive more punitive, escalatory, and blame-focused responses, while female actors are more often met with empathy and therapeutic guidance for the same behavior. The researchers report that this pattern appears across model families, scenario categories, model sizes, and prompts that explicitly elicit reasoning. The work therefore focuses on a form of gender bias distinct from stereotypes or overtly harmful outputs: unequal moral treatment in advice and conflict responses. Its central finding is that models can apply different standards of tone and accountability even when the underlying misconduct is held constant.

Read on arxiv.org63 comments on Hacker News

Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.

More in AI

The daily digest

Today's best Hacker News stories, summarized and screenshotted, one email a day.