AI Workflow Evaluator - AI Trainer

Mercor Β· Remote, US
Work typeRemote Β· Contract
Pay$50 – $190/hr
PostedNov 20, 2026
LevelMid level
In short Mercor connects elite creative and technical talent with leading AI research labs.
LLM

πŸ“‹ About the job

Job typeContract
Pay$50–$190/hour
LocationRemote

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .

Position: MCP & Plug In Connectors Expert

Commitment: 20+ hours/week

Role Responsibilities

  • Evaluate AI systems on complex personal workflows, including personal health , travel , and activity planning .
  • Design realistic prompts for complex personal-life tasks using Google Drive , Expedia , Notion , and other plugins/connectors.
  • Execute tasks while recording your screen and using personal plugins/connectors.
  • Write clear explanations of AI successes and failures to improve AI model performance .
  • Judge AI outputs for practicality, personalization, and reasoning.
  • Identify where models miss context, overreach, or fail to use tools correctly.

Qualifications

Must-Have

  • US-based only.
  • Strong MCP experience and plugin/connector usage.
  • Experience using LLM plugins/connectors like Google Drive , Expedia , and Notion .
  • Heavy personal usage of LLM products.
  • Active LLM account with 6+ months of history.
  • Willingness to sign a data-share consent form via DocuSign .
  • Experience using AI for multi-step planning, research, and decision-making.
  • Strong written judgment and attention to detail.
  • Ability to evaluate AI outputs and write against rubrics.

Preferred

  • Extensive rubric experience with 100+ hours on prior projects.

Compensation & Legal

  • Hourly contractor .
  • Experts added to the project will begin in a trial period.

Application Process (Takes 20–30 mins to complete)

  • Upload resume
  • AI interview based on your resume
  • Submit form

Resources & Support

  • For details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcome
  • For any help or support, reach out to: [email protected]

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

#hiringmercor

Interested in this role?Apply in one click β€” no JobQ account needed.
Apply now β†’
AI Workflow Evaluator - AI TrainerMercor
Apply now