AI/ML Research Engineer, LLM Post-Training & Evaluation

About the role

Innodata is expanding its team of technical experts in LLM training, post-training, and evaluation systems. This role is ideal for someone with hands-on experience fine-tuning and evaluating large language models (and ideally multimodal models).

Responsibilities

• Design and implement the pipelines and tooling that connect data, evaluation, and post-training.
• Help customers and internal teams move from evaluation findings to measurable model improvements.
• Build fine-tuning workflows (e.g., supervised fine-tuning and preference-based optimization).
• Integrate evaluation harnesses into model development loops.
• Improve experiment reliability and throughput.
• Support advanced evaluation scenarios such as long-context, cross-modal, and dynamic multi-turn interactions.
• Contribute to Innodata’s internal R&D efforts, including benchmark datasets, evaluation frameworks, and reusable infrastructure for model assessment and post-training experimentation.
• Lead or co-lead technically complex ML engineering projects from initial customer discussions through implementation and delivery.
• Implement and optimize evaluation systems for LLMs and multimodal models, including offline benchmarks and task-specific test harnesses.
• Integrate human-in-the-loop and AI-augmented evaluation signals into model development workflows.
• Build robust infrastructure and tooling for reproducible experimentation, metrics logging, and regression monitoring.
• Diagnose model behavior and pipeline failures, including data issues, training instability, metric inconsistencies, and evaluation drift.
• Collaborate with Language Data Scientists and Applied Research Scientists to translate evaluation frameworks into executable systems.
• Work closely with customer technical stakeholders to understand goals, constraints, and success criteria; propose and implement technically sound solutions.
• Contribute to internal research and platform development, including benchmark frameworks, evaluation tooling, and post-training workflow improvements.
• Contribute to best practices and standards for LLM training, evaluation, and quality assurance across projects.
• Mentor junior engineers and contribute to technical design reviews, documentation, and engineering rigor across the team.

Qualifications

• BS/MS/PhD in Computer Science, Machine Learning, AI, Applied Mathematics, or a related quantitative technical field (MS/PhD preferred)
• 2-3 years of relevant industry or research engineering experience in ML/AI systems
• Hands-on experience with LLM training / fine-tuning / post-training, including at least one of:
• supervised fine-tuning (SFT)
• preference optimization (e.g., DPO or related methods)
• RLHF / RLAIF-style workflows
• task- or domain-adaptation of foundation models
• Strong programming skills in Python and experience building production-quality ML code
• Experience with modern ML frameworks (e.g., PyTorch, JAX, TensorFlow) and model libraries/tooling (e.g., Hugging Face ecosystem, vLLM, distributed training stacks)
• Experience designing and implementing evaluation pipelines for LLM/ML systems, including metrics computation, dataset handling, and experiment comparisons
• Strong understanding of data pipelines and ML systems engineering, including reproducibility, observability, and debugging
• Experience with large-scale distributed ML systems and performance optimization for training/evaluation workloads (GPU/accelerator environments preferred)
• Experience with large-scale data processing and workflow orchestration in support of model training/evaluation
• Able to collaborate directly with technical stakeholders including research scientists, ML engineers, data engineers, and customer technical leads
• Strong written and verbal communication skills, including the ability to explain complex technical tradeoffs to both technical and non-technical audiences

Pay

The expected salary range for this position is $80,000 – $175,000 USD per year, based on experience, skills, and qualifications.