Senior Staff Deployment Automation Engineer

About the role

As a Senior Staff/Principal Deployment Automation Engineer for the Compute Team, you will be responsible for deployment and testing automation of large-scale, multi-node GPU clusters. You will own the CI/CD infrastructure, including both deployment and integration testing, for a rapidly scaling fleet of virtualized GPU and CPU hosts across our AI Cloud. Your role is critical in ensuring the stability of the low-level infrastructure and enabling teams across our Cloud Infrastructure organization to quickly and reliably release, test, and deploy their artifacts across our datacenters.

San Francisco, Sunnyvale, Bellevue (Onsite)

Responsibilities

• Completely own deployment and integration testing automation for all bare-metal, on-premise systems across Crusoe’s AI Cloud Stack.
• Build CI/CD platforms that enable developers to quickly test, iterate, and deploy critical, low-level systems and applications.
• Design and execute large-scale validation tests across multi-node virtualized clusters to ensure linear scaling and stability of GPU workloads.
• Maintain and scale bare-metal Linux configurations using a mix of custom and off-the-shelf tooling such as Gitlab, Ansible, AWX, osquery, etc.
• Create control applications to coordinate canary deployments on live production systems, run Blue/Green testing, and perform automatic rollback where necessary.
• Develop and maintain automation frameworks in Python or Go to dynamically provision, configure, and stress-test multi-node virtualized environments.
• Create automated test suites leveraging tools like fio, stress-ng, and iperf to ensure performance and multi-tenant isolation of CPU and GPU hosts.

Requirements

• 12+ years of experience demonstrated ability to competently and independently perform responsibilities.
• Bachelor’s or Master’s degree in Computer Science, Electrical Engineering, or a related technical field.
• Experience building and deploying automated integration testing for an AI Cloud Environment, ranging from low-level Linux Systems up to Distributed Control Planes.
• Working knowledge of the modern infrastructure stack, including Kubernetes, Docker, Terraform, and Postgres.
• Intimate knowledge of CI/CD pipelines and Gitlab Tooling to enable stable infrastructure releases across multiple datacenters.
• Previous experience with at least 1-2 configuration management systems, including Ansible, Puppet, Chef, or SaltStack.
• Advanced proficiency in Python and/or Bash for automating complex cluster-wide test scenarios.
• Knowledge of Linux kernel internals, specifically PCIe topology, VFIO, and memory management (HugePages, IOMMU).
• Familiarity with NVIDIA (CUDA/NCCL) and/or AMD (ROCm/RCCL) stacks in a multi-node context.
• Strong understanding of RDMA, RoCE, and InfiniBand protocols and their implementation in virtualized systems.

Bonus Points

• Experience with MNNVL (Multi-Node NVLink) or specialized AI fabric architectures.
• Familiarity with hardware-level debugging tools and performance profilers (e.g., NVIDIA Nsight, AMD Omniperf).
• Knowledge of containerized orchestration for GPUs (e.g., Kubernetes with specialized device plugins).

Benefits

• Competitive compensation and equity packages
• Restricted Stock Units
• Paid time off, paid holidays & leave of absence programs
• Comprehensive health, dental & vision insurance
• Employer contributions to HSA account
• Paid parental leave
• Paid life insurance, short-term and long-term disability
• Professional development & tuition reimbursement
• Mental health & wellness support
• Commuter benefits (parking & transit)
• Cell phone stipend
• 401(k) Retirement plan with company match up to 4% of salary
• Volunteer time off
• Global travel insurance & emergency assistance
• Daily meals allowance
• Additional perks & programs specific to location

Pay

Compensation will be paid in the range of up to $250,000 - $300,000 + Bonus. Restricted Stock Units are included in all offers. Compensation to be determined by the applicant's knowledge, education, and abilities, as well as internal equity and alignment with market data.