Data Scientist, AI & Machine Learning (PhD Student)

Cobalt · Remote, US
Work typeRemote · Contract
Pay · estimate~$120k – $160kMarket rate · not set by employer
PostedNov 17, 2026
LevelMid level
In short Cobalt is hiring current Masters and PhD students to review reinforcement learning and data science tasks and environments used to test whether frontier AI models can carry out…
SQLPYTHONDOCKERPYTORCHMACHINE LEARNING

📋 About the job

Job typeFlexible Contract (Remote)

Cobalt is hiring current Masters and PhD students to review reinforcement learning and data science tasks and environments used to test whether frontier AI models can carry out real research, analysis, and engineering work. Your job is to make sure each one is correct, fair, and actually measures what it claims to measure before it is accepted. If you enjoy finding the flaw in an experiment, the leak in a dataset, the bug in a grader, or the loophole in a reward function, this role is built around that skill.

What you will do

  • Review tasks end to end, including the task description, environment code, datasets, dependencies, reference solution, and automated tests or graders, and confirm that each part is correct and consistent with the others.
  • Run reference solutions and graders yourself to confirm that correct solutions pass, incorrect solutions fail, and results are reproducible. Because model training and reinforcement learning results vary between runs and random seeds, you will check that tests are deterministic or use tolerances that are clearly justified.
  • Audit the data behind each task, checking for leakage between training and test sets, mislabeled or inconsistent records, and evaluation metrics that do not match the stated goal.
  • Look for ways an agent could pass a task without doing the intended work, such as exploiting a reward signal, reading the answer from the environment or the data, or satisfying a weak test, and document each issue so the task author can fix it.
  • Review how a frontier AI model attempted each task and judge whether its failure reflects a real gap in capability, or instead reflects ambiguous wording, a hidden assumption, a broken environment, flawed data, or an unfair grader.
  • Write clear, specific feedback to task authors and work with them through revisions until the task meets the acceptance standard.
  • Track recurring problems across tasks and help improve the review checklist and quality standards over time.

Example tasks you might review include implementing proximal policy optimization from a paper’s description alone, finding the bug in a flawed reward function or data pipeline, building a model on a messy tabular dataset without leaking information from the test set, reproducing a reported result and explaining a discrepancy, and running a sound statistical comparison between two models or agents across multiple random seeds.

Who we are looking for

  • Curren…
📄 Read the full description → The rest of this listing is on the hiring partner’s site.
Interested in this role?Apply in one click — no JobQ account needed.
Apply now →
Data Scientist, AI & Machine Learning (PhD Stude...Cobalt
Apply now