Python Insfrastructure Engineer - Model Evaluation

What You'll Do

• Design, build, and optimize high-performance Python systems supporting AI data pipelines and model evaluation workflows
• Develop full-stack tooling and backend services for large-scale data annotation, validation, and quality control
• Build and maintain evaluation harnesses that integrate with ML inference frameworks
• Improve reliability, performance, and safety across existing Python codebases
• Instrument systems with observability and metrics collection to monitor reliability and model performance
• Identify bottlenecks and edge cases in data and system behavior, and implement scalable fixes
• Collaborate with data, research, and engineering teams to support model training and evaluation workflows
• Participate in synchronous design reviews to iterate on architecture and implementation decisions

Who You Are

• Native or fluent English speaker with clear written and verbal communication skills
• Full-stack developer with a strong systems programming background
• 3–5+ years of professional experience writing production-grade Python
• Experienced building evaluation harnesses for ML models and integrating with inference frameworks
• Solid background in observability, metrics collection, and monitoring for production systems
• Self-motivated and reliable — able to commit 20–40 hours per week

Nice to Have

• Prior experience with data annotation, data quality, or evaluation systems
• Familiarity with AI/ML workflows, model training, or benchmarking pipelines
• Experience with distributed systems or developer tooling
• Background in MLOps or AI infrastructure