← All jobs

AI Evaluation Engineer

Posted 2d ago

About the role

Expedock is hiring a benchmark and evaluation engineer for a client that builds benchmarks for AI analysts working on production-scale enterprise data. The role stands up realistic multi-system data environments, writes benchmark tasks with golden SQL, and extends the grading harness, including LLM judges. It is full-time, remote in the Philippines, on a US-shift schedule.

What you'll do

  • Design and stand up multi-system enterprise data environments, including warehouses, lakes, APIs, and databases.
  • Build calibrated benchmark datasets with realistic noise, drift, and synthetic personal data.
  • Write and verify benchmark tasks, golden SQL, and rubric-scored reasoning keys, and keep leakage out.
  • Extend the grading harness, including deterministic checks and LLM judges, and keep a regression-tracked leaderboard.

Requirements

  • Expert production SQL and data modeling, including CTEs, window functions, query plans, and star or snowflake schemas.
  • Strong Python (pandas, NumPy, or Polars) and hands-on experience loading a major warehouse such as Snowflake, BigQuery, Redshift, or Postgres.
  • Experience designing evaluation sets or benchmark tasks, including golden SQL, rubric or LLM-judge calibration, and regression tracking.
  • Able to build realistic multi-system data setups with large or messy synthetic data.
  • Full-time, remote, on a US-shift schedule.

About Expedock

Expedock is a tech-enabled workforce platform that places offshore talent with client teams.

Visit Expedock →

AI Alerts shares third-party job opportunities for informational purposes only. We are not the employer and are not involved in the hiring process. Always verify the company and role through official channels before applying, and never pay to apply, train, onboard, process documents, or secure a job offer. Legitimate employers do not ask applicants for money. Read our Terms to learn more.