GPU Fabric Engineer

Key Responsibilities

• Validate and troubleshoot InfiniBand and RoCE fabrics during GPU cluster bring-up and expansion
• Tune fabric performance parameters for distributed AI workloads (NCCL, MPI, collective operations)
• Monitor and manage fabrics using NVIDIA UFM (Unified Fabric Manager) for health, topology, and performance visibility
• Diagnose and resolve fabric-level issues including link errors, congestion, packet loss, and path asymmetry
• Optimize RDMA transport settings, PFC/ECN behavior, and lossless queue configuration for GPU traffic
• Validate fabric performance benchmarks and ensure line-rate throughput for AI workloads
• Collaborate with GPU Engineers to correlate fabric health with workload performance
• Collaborate with networking teams on fabric provisioning, configuration, and remediation
• Respond to fabric alerts and degradation events across production GPU clusters
• Document fabric troubleshooting procedures, tuning parameters, and validation runbooks

Qualifications

• 3–7 years of experience in network engineering, HPC fabric, or GPU infrastructure
• Hands-on experience with InfiniBand and/or RoCE fabrics in GPU cluster environments
• Experience with NVIDIA UFM for fabric management, monitoring, and diagnostics
• Strong understanding of RDMA transport, lossless Ethernet design, and congestion management (PFC, ECN, DCQCN)
• Experience with GPU cluster networking and distributed communication libraries (NCCL, MPI)
• Familiarity with GPU platforms and their interconnect requirements (NVIDIA NVLink, NVSwitch, ConnectX)
• Experience with fabric diagnostic tools (ibstat, ibqueryerrors, perfquery, etc.)
• Proficiency in Python or Bash for scripting and validation
• Basic understanding of Linux systems and server hardware
• Strong troubleshooting and analytical skills across network and system layers

Pay

$125,000 - $135,000 Final compensation will vary depending on years of experience, background/skill set, location, and applicable laws.