Senior Principal AI Engineer

For our client, a leader in the Manufacturing & Automotive space, we are seeking a Senior Principal AI Engineer to lead technical initiatives focused on scalable systems, platform reliability, and meaningful business transformation. This role involves working across engineering, product, operations, and business stakeholders to translate complex requirements into practical technology solutions, influencing architecture, execution quality, and long-term technology capabilities.

Location: Remote - US based candidates only, no visa sponsorship available

Responsibilities

• Design and operate distributed training systems for various neural network models across GPU clusters
• Optimize execution of multi-node, multi-GPU setups to enhance throughput
• Diagnose and resolve bottlenecks in compute, memory, and networking
• Enhance training stability and fault tolerance for large scale deployments
• Collaborate with research and applied ML teams to implement robust training pipelines
• Build and optimize orchestration frameworks for GPU clusters
• Minimize networking bottlenecks to improve overall training efficiency

Qualifications

• 5-7 years of experience with distributed systems or ML systems
• Proven ability to manage large-scale workloads on GPU clusters
• Experience with PyTorch distributed training in production environments
• In-depth knowledge of parallelism strategies including data, tensor, and pipeline parallelism
• Understanding of GPU communication and networking at a low level
• Familiarity with orchestration tools like Slurm, Kubernetes, Ray, RunAI
• Competence with memory optimization techniques such as activation checkpointing and ZeRO strategies

Pay

$150,000 – $180,000 annually

Benefits

• Opportunity to work on cutting-edge AI technology
• Direct impact on large model training and research efficiency
• Collaborative work environment with access to ML experts
• Focus on developing scalable, reliable AI infrastructure
• Potential for professional growth in a rapidly evolving field