Mercor
Behavioral Health Expert, AI Safety and Model Evaluation
$45–$70 / hourRemote · US
A leading AI lab is seeking behavioral health experts to help evaluate and improve how its AI models handle sensitive, everyday conversations.
Help a leading AI lab ensure its models respond to people with care, balance and sound judgment.
1. Overview
A leading AI lab is seeking behavioral health experts to help evaluate and improve how its AI models handle sensitive, everyday conversations. People increasingly turn to AI for support with relationships, family dynamics, emotional wellbeing, personal beliefs and difficult life decisions. These conversations rarely involve crisis content, but they are exactly where a model's judgment matters most: whether it stays balanced, avoids taking sides, resists telling people what they want to hear, respects a person's beliefs without endorsing or dismissing them, and understands the limits of its role. You'll review these interactions, assess whether the model's responses are appropriate, neutral and safe, and help the lab's researchers define what good looks like. If you bring professional experience in mental health, counseling, social work or behavioral science, and you can articulate clearly why one response serves a person better than another, this role is for you. This is a part-time commitment of at least 20 hours per week, with the option to increase to up to 40 hours per week.
This is a W-2 employment position with Cincinnatus LLC, with the opportunity to be placed at a leading AI lab as part of their extended workforce.
2. Key Responsibilities
- Evaluate conversations between users and AI models across topics such as relationship and family advice, emotional wellbeing, spiritual and metaphysical questions, and unconventional or unfounded beliefs, and assess whether responses are neutral, appropriate and safe.
- Identify patterns of concern, including excessive agreement, taking sides, reinforcing distorted or unfounded beliefs, moralizing, or overstepping into clinical or directive advice, and document them with clear written rationale.
- Develop rubrics, guidelines and reference responses that define a balanced, supportive and appropriately bounded reply, grounded in established practice from counseling and behavioral health.