Research Engineer (RE) Curator
TransPerfect · New York, United States · 1 mo ago
RemoteRemoteAdministrativeOther
Main Responsibilities
- Design, curate, and implement complex AI evaluation benchmarks and dataset frameworks to test advanced LLM capabilities and limitations.
- Conduct experimental research, red teaming, and systematic testing to identify model failure modes, edge cases, and reasoning gaps.
- Utilize Python, Jupyter/Colab environments, and modern IDEs to write scalable code, process datasets, and automate evaluation pipelines.
- Collaborate with AI research and engineering teams to translate complex subject-matter concepts into actionable model benchmarks.
- Document research findings, track codebase and dataset changes via Git, and deliver precise sourcing and evaluation reports.
Role Requirements
- Education: MSc or PhD in a STEM discipline (Computer Science, AI/ML, Statistics, Mathematics, Physics, Computational Sciences, or related field).
- Experience: Proven background as a Research Engineer, Applied Scientist, ML Engineer, Research Scientist, or AI Researcher.
- Programming & Tools: Strong hands-on Python programming skills, with daily fluency in Git, modern IDEs, and Jupyter/Colab environments.
- AI/ML Technical Expertise: Core experience in Machine Learning, LLMs, data analysis, and structured experimental research.
- Specialized Background (Preferred): Direct experience in AI evaluation, benchmark curation/development, red teaming, or model testing frameworks.
- Mindset: Strong research mindset paired with exceptional analytical and problem-solving capabilities.