Senior Machine Learning Engineer, Agentic Evals and Self-Improvement (Nova)

• Build evaluation systems for models and agents: benchmarks, LLM judges, simulations, trajectory analysis, and regression tests.

• Create a loop that takes failures from diagnosis to fix to measured improvement.

• Turn evaluation results into automatic optimization of prompts, context, tools, rubrics, and agent harnesses.

• Build synthetic data pipelines for evaluation and post-training, with attention to quality, privacy, and reproducibility.

• Apply recent research (LLM-as-judge, reward modeling, test-time search, post-training) to production.

• Mentor engineers on the team and help set the technical direction for Nova.

• 5+ years of hands-on MLE experience putting complex ML systems into production at scale, such as production ML pipelines or large-scale experimentation platforms.

• Master's or PhD in Computer Science, Machine Learning, AI, or a related field.

• You keep up with the latest research and focus on applying it: shipping reliable changes that measurably improve the product experience for customers.

• Strong Python skills with an emphasis on readable, modular, and maintainable code, plus experience with modern ML frameworks.

• Deep experience evaluating LLMs or agents, including multi-turn behavior, tool use, and reasoning.

• Experience building automated evaluation, such as LLM judges, reward models, or benchmark infrastructure.

• Experience with at least one of: prompt or context optimization, post-training, preference optimization, or reinforcement learning.

• Strong communication and cross-team collaboration.

• Experience with LLM APIs from multiple providers (Anthropic, OpenAI, Google), including tool calling and prompt caching.

• Experience with automatic prompt optimization and error analysis

Please be aware that Iterable, Inc. ("Iterable") and our official professional recruiting agencies and platforms do not:

• Send job offers from free email services like Gmail, Yahoo mail, Hotmail, etc.

• Request money, fees, or payment of any kind from prospective candidates to apply to Iterable, for employment, or for the recruitment process (e.g. for home office supplies, or training, etc.).

• Request or require personal documents like bank account details, tax forms, or credit card information as part of the recruitment process prior to the candidate signing an engagement letter or an employment contract with Iterable.

You may see all job vacancies on our official Iterable channels:

• Official Iterable website, Careers page: https://iterable.com/careers/

• Official LinkedIn Jobs page: https://www.linkedin.com/company/iterable/jobs/

Iterable is not affiliated in any way to these impostors and we hereby confirm that such individuals/entities are not authorized, encouraged, or sponsored to act on behalf of Iterable. Such job opportunities are entirely fake and not valid. Therefore, please disregard any written or oral request for a job offer or an interview that you believe is or might be fraudulent or suspicious and immediately reach out to us via email at