Systems Administrator Lead (Second Shift)
HPC Infrastructure Operations
Administer, maintain, and troubleshoot Linux-based HPC and AI compute environments, including large-scale GPU clusters built on NVIDIA HGX, DGX, or equivalent accelerated computing platforms.
Diagnose hardware, OS, driver, firmware, networking, and storage-related failures, and perform root-cause analysis to develop corrective actions for recurring issues.
Serve as a senior escalation point for complex incidents affecting production compute infrastructure, troubleshooting degraded or unstable compute/GPU nodes to restore service.
Support infrastructure across heterogeneous environments - bare-metal, virtualized, containerized, and cloud-hosted - and maintain operational documentation, runbooks, and shift-handoff notes.
GPU and Accelerated Computing Systems
Install, configure, validate, and troubleshoot NVIDIA GPU drivers, CUDA components, firmware, and supporting system software using tools such as Nvidia Smi, DCGM, and NVIDIA Fabric Manager.
Investigate GPU Xid errors, NVLink/NVSwitch faults, PCIe errors, ECC events, GPU resets, and thermal or power-related issues.
Diagnose communication and performance issues involving NCCL, GPUDirect RDMA, PCIe topology, NUMA placement, and multi-GPU systems, validating GPU-to-GPU, GPU-to-network, and GPU-to-storage communication after repairs.
Coordinate replacement and RMA activities for GPUs, GPU trays, baseboards, NVSwitch components, system boards, and NICs.
Linux Systems Administration
Administer enterprise Linux distributions across the production fleet, troubleshooting boot failures, kernel issues, systemd services, filesystems, and OS performance.
Analyze system and kernel logs (journalctl, dmesg, lspci, dmidecode, ipmitool, ethtool, ss, sar, perf) and manage kernel modules, device drivers, DKMS packages, and OS updates.
Support Linux networking, storage mounts, authentication, and security controls, and investigate OS crashes, kernel panics, and out-of-memory conditions.
Support enterprise server platforms from vendors such as Dell, HPE, ASUS, Supermicro, and NVIDIA, troubleshooting processors, memory, PCIe devices, NICs, storage controllers, and other hardware components.
Manage BIOS, BMC, CPLD, NIC, GPU, switch, and drive firmware, and perform remote console troubleshooting, power cycling, and log bundle generation.
Partner with on-site Data Center technicians on component replacement, cabling validation, break-fix activity, and post-repair testing.
High-Performance Networking
Troubleshoot Ethernet, InfiniBand, and RoCE connectivity within HPC and AI environments, including link state, VLAN/bonding issues, MTU mismatches, packet loss, and RDMA/GPUDirect RDMA connectivity.
Collaborate with Deployment Engineering to troubleshoot fabric instability, link degradation, congestion, and node-level connectivity issues.
Storage and Filesystem Support
Troubleshoot server connectivity to parallel, distributed, and enterprise storage platforms such as NFS, Lustre, BeeGFS, GPFS/Spectrum Scale, and Ceph, diagnosing mount failures, throughput degradation, and capacity issues.
Validate storage network paths and coordinate with storage and networking teams during service-impacting events.
Incident, Problem, and Change Management
Lead or support incident response for critical infrastructure events, participating in technical bridges and providing clear status updates during outages.
Create detailed timelines, root-cause analyses, and corrective-action plans, maintaining accurate records in Jira Service Management or similar ITSM platforms.
Leadership and Collaboration
Mentor junior and mid-level Systems Administrators and HPC Engineers, providing technical guidance during complex troubleshooting and infrastructure changes.
Promote consistent troubleshooting methods, documentation standards, and operational discipline across shift handoffs.