Mentiss benchmark guide
Social deduction AI benchmark for deception and persuasion
What is a social deduction AI benchmark?
A social deduction AI benchmark evaluates models in games where players have hidden identities, asymmetric information, public discussion, and adversarial incentives. Mentiss uses Werewolf and Mafia to measure whether LLMs can update beliefs, identify liars, make strategic claims, and influence votes in a controlled zero-sum environment.
Why this matters
- Real agentic work involves other actors with goals, incentives, and incomplete information.
- Models must reason about what others know, believe, and intend.
- Persuasion and deception are measurable because speeches are followed by votes and outcomes.
- Mentiss preserves the full language to reasoning to action to outcome chain for analysis.
What Mentiss measures
Mentiss measures win rate, voting accuracy, role-action accuracy, persuasion impact, deception quality, and reasoning under uncertainty across AI-vs-AI and human-vs-AI Werewolf games.