MLOps Engineer

About the role

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a MLOps Engineer based in the United States. This role is focused on building and operating high-performance machine learning inference platforms that support large-scale, production AI systems.

Responsibilities

• Design, build, and operate scalable model serving platforms for LLMs, vision models, and recommendation systems.
• Optimize inference performance using techniques such as batching, caching, speculative decoding, and request routing strategies.
• Implement multi-tenant serving architectures with rate limiting, QoS policies, and traffic management controls.
• Develop autoscaling and capacity planning systems to balance latency, cost, and throughput across workloads.
• Improve GPU utilization and memory efficiency for high-performance inference workloads.
• Integrate model serving systems with APIs, identity services, and observability platforms.
• Build and enhance observability frameworks covering latency, GPU metrics, error tracking, and system health.
• Support deployment pipelines including canary releases, shadow testing, and rollback mechanisms.
• Participate in incident response for production AI services and drive long-term reliability improvements.
• Collaborate with ML and product teams to support model releases and production rollouts.
• Implement security and abuse prevention controls at the serving layer.
• Document system behavior, operational procedures, and performance tuning best practices.

Requirements

• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
• 6+ years of experience in distributed systems, infrastructure engineering, or ML platform development.
• Strong proficiency in Python and a systems programming language such as Go, Rust, or C++.
• Hands-on experience with LLM inference frameworks such as vLLM, TensorRT-LLM, or similar.
• Strong understanding of GPU architecture, memory management, and performance optimization.
• Experience with Kubernetes, cloud platforms, and autoscaling infrastructure.
• Strong knowledge of observability tools including metrics, logging, and distributed tracing systems.
• Solid understanding of performance engineering, capacity planning, and distributed system design.
• Strong communication and incident response skills in production environments.
• Experience with AI model serving at scale, multi-region systems, or FinOps optimization is a plus.

Benefits

Competitive salary range of $100,000 - $150,000 annually.
100% remote position within the United States.
Full-time W2 employment with long-term stability.
Opportunity to work on cutting-edge AI inference and LLM serving systems.
Exposure to advanced GPU optimization and large-scale distributed AI infrastructure.
Career growth through ownership of production AI platforms and architecture decisions.
Inclusive and equal opportunity workplace culture.