AI Infrastructure Engineer

We need to output HTML fragment with sections: About the role (position summary), Responsibilities (key responsibilities), Requirements (qualifications), maybe Preferred Qualifications as separate? The instruction: Use

for section headings (e.g. About the role, Responsibilities, Requirements, Qualifications, Skills, Benefits, Pay, Schedule). We should include sections only if posting has real content. We have About the role (position summary). Responsibilities (Key Responsibilities). Requirements (Qualifications). Benefits (Compensation & Benefits). Pay (Compensation). Schedule? Not present. Skills? Could be part of qualifications but not separate. We can include Skills if we want but not required. We'll include sections: About the role, Responsibilities, Requirements, Benefits, Pay. Ensure we don't repeat job title. Use

• for any list of duties/requirements/benefits. Use

  for prose. We need to preserve all substantive information, not invent or summarize away detail. Must not include boilerplate like EEO etc. Remove navigation etc. The provided text includes sections: About Amaara Networks (company description) maybe we can include as part of About the role? The instruction: Use

  for section headings (e.g. About the role, Responsibilities, Requirements, Qualifications, Skills, Benefits, Pay, Schedule). The company description "About Amaara Networks" could be considered part of About the role? But we could include a short paragraph about the company under About the role or maybe separate? The instruction says do NOT repeat the job title as a heading (the page already shows it) and do NOT put Job ID / requisition numbers as headings — fold minor metadata into a short

  or drop it. It doesn't forbid a company description heading, but they gave examples of headings. Probably we should include About the role (position summary) and maybe also include a brief About Amaara Networks as a paragraph within that section? Or we could add a separate section "About the company" but not in list. Safer to keep only the sections they listed. We'll put company description in About the role paragraph. We need to preserve all substantive information: So we need to include details from Position Summary, Key Responsibilities (with sub-bullets), Qualifications, Preferred Qualifications maybe as part of Requirements? Could separate Preferred Qualifications as part of Requirements but maybe better to include under Requirements as a subsection? But we must use

  • for any list. We can have nested lists? Probably fine. We need to avoid filler like "commensurate with experience". In Compensation & Benefits we have "Compensation: Negotiable, depending on experience". That's somewhat filler but it's actual content. Should we include Pay section? The posting says "Compensation & Benefits" with bullet list: Compensation: Negotiable, depending on experience; 401(k); Healthcare reimbursement; Paid holidays; Flexible time off. That's substantive enough to include Pay and Benefits sections. However instruction: never create Pay/Schedule/Benefits sections unless the posting actually states them. It does state compensation and benefits, so we can include Pay and Benefits sections. But we need to avoid filler like "Negotiable, depending on experience" - that's not placeholder like "not specified". It's actual statement, so okay. We need to separate Pay and Benefits? Could combine but they said prefer few substantive sections over many thin ones. Could have a Benefits section that includes compensation and benefits. But they gave separate headings for Pay and Benefits. Might be better to have a Benefits section that includes compensation details. However they said "never create Pay/Schedule/Benefits sections unless the posting actually states them." It does state compensation and benefits, so we could have both Pay and Benefits. But we could also combine into Benefits and mention compensation there. To be safe, we can have a Benefits section that includes compensation details. But they might expect separate Pay and Benefits. However they said prefer a few substantive sections over many thin ones. So maybe combine into Benefits. Let's decide: We'll have sections: About the role, Responsibilities, Requirements, Benefits. In Benefits we include compensation and benefits details. We must not include Schedule as not present. Now we need to format. We'll start with

    About the role

    then

    containing position summary and company description. Then

    Responsibilities

    then
    • each major bullet; within each li we can have further
      • for sub-items. Then

        Requirements

        then
        • for qualifications (mandatory) and maybe separate
          for Preferred Qualifications? Could embed as separate
          inside li? Better to have two lists: one for Required Qualifications, another for Preferred Qualifications. But heading only Requirements; we could have a

          stating "Required:" then list, then another

          stating "Preferred:" then list. That's acceptable. Then

          Benefits

          then
          • for each benefit item, including compensation statement. Make sure to preserve all details: need to include all responsibilities sub-bullets exactly as given. Let's extract responsibilities: Key Responsibilities Infrastructure Troubleshooting & Escalation: Own Tier 2 and Tier 3 infrastructure escalations from investigation through resolution. Perform root-cause analysis on server and infrastructure failures using BMC, IPMI logs, kernel messages, MCE, EDAC records, thermal and power telemetry, and vendor diagnostics. Identify recurring hardware and infrastructure issues and develop permanent corrective actions. Participate in post-incident reviews and translate findings into monitoring improvements, runbooks, and operational standards. Work directly with hardware vendors on diagnostics, RMAs, firmware issues, and field-service activities. Communicate clearly with customers during incidents and scheduled maintenance windows. Coordinate remote hands and field technicians during work performed at remote sites. Participate in an infrastructure on-call rotation. GPU Infrastructure: Deploy, administer, and troubleshoot high-performance GPU compute systems. Support NVIDIA H100, H200, B200, B300 SXM and PCIe platforms, AMD Instinct, or comparable accelerator platforms. Use tools such as nvidia-smi, DCGM, and DCGM diagnostics to troubleshoot GPU health and performance issues. Analyze XID errors, ECC events, NVLink, NVSwitch health, thermal conditions, and power-related throttling. Install and maintain GPU drivers, CUDA toolkits, Fabric Manager, and container runtimes. Manage software and driver compatibility across the GPU fleet. Perform GPU stress testing, burn-in, NCCL testing, bandwidth testing, and all-reduce benchmarking. Investigate performance issues including straggler nodes, reduced collective bandwidth, PCIe degradation, NUMA configuration, and topology problems. High-Performance Networking: Configure and support InfiniBand and RoCEv2 high-performance AI networking environments. Configure switches, manage firmware, and validate network topology. Support InfiniBand subnet management, partitioning, routing, and topology validation. Use tools such as UFM, OFED, ibdiagnet, ibnetdiscover, and perfquery to troubleshoot network issues. Diagnose link flaps, symbol errors, degraded optics, cabling issues, and port-level failures. Configure and troubleshoot RoCE environments, including PFC, ECN, lossless queues, and congestion control. Administer RDMA, OFED stacks, NIC, HCA firmware, bonding, VLANs, and management networks. Support enterprise networking, including L2, L3 networking, VLANs, LACP, MLAG, VLT, firewalls, NAT, VPN, DHCP, and DNS. Validate network performance following deployments and infrastructure changes. Maintain accurate network topology, port maps, and infrastructure documentation. Linux Systems Administration: Administer Linux systems including RHEL, Rocky Linux, and Ubuntu. Manage provisioning, patching, kernels, drivers, storage, networking, users, and access. Operate bare-metal provisioning platforms such as MAAS, Foreman, Ironic, or similar technologies. Troubleshoot PXE, commissioning, enrollment, deployment, and image-related issues. Configure cloud-init, netplan, bonded networking, and storage layouts. Tune Linux hosts for GPU and HPC workloads, including NUMA, CPU pinning, IRQ affinity, and I/O configuration. Monitor hosts, GPUs, networking, and environmental conditions. Develop monitoring, dashboards, and alerts to identify issues before they affect customers. Write Python and Bash scripts to automate diagnostics, reporting, and repetitive operational tasks. Data Center Deployment & Physical Infrastructure: Rack, cable, configure, and commission high-density GPU servers, storage, and networking equipment. Lead the technical execution of data center deployment windows. Install and terminate copper, fiber, DAC, and AOC cabling for high-speed 100G–800G connections. Plan and verify rack-level power distribution, A/B redundancy, circuit sizing, and load balancing. Contribute to rack and cabinet layouts, including U-space, airflow, weight, cable management, and power requirements. Maintain accurate DCIM and asset records, including rack elevations, cable runs, serial numbers, port maps, firmware inventories, and lifecycle status. Develop and maintain installation standards, validation checklists, and operational runbooks. Now qualifications (mandatory): Qualifications 4–7 years of experience combining data center operations, infrastructure engineering, and Linux systems administration. Strong production Linux administration experience with RHEL, Rocky Linux, or Ubuntu. Experience managing Linux kernels, drivers, storage, networking, and shell-level diagnostics. Hands-on experience operating a bare-metal provisioning platform. Experience with PXE, node commissioning, deployment, and provisioning troubleshooting. Experience using Redfish and IPMI for hardware management, power control, boot configuration, and firmware operations. Demonstrated Tier 2, Tier 3 escalation and incident-resolution experience. Strong server hardware troubleshooting and root-cause analysis skills. Experience with BMC, IPMI logs, system event logs, and Linux kernel diagnostics. Experience with high-performance networking, including InfiniBand, RoCE, or high-speed Ethernet. Working knowledge of production L2, L3 networking. Experience supporting GPU or accelerated-computing systems. Understanding of data center fundamentals including power, cooling, airflow, cabling, and rack layouts. Scripting experience with Python or Bash. Familiarity with configuration management tools such as Ansible. Strong technical writing and documentation skills. Ability to communicate directly with customers during infrastructure incidents. Ability to coordinate remote technicians and field-service personnel. Willingness to participate in an on-call rotation and occasional after-hours maintenance windows. Preferred Qualifications (list): Preferred Qualifications Experience supporting large-scale, multi-rack GPU clusters. Experience with NVIDIA SXM systems, NVLink, NVSwitch, or rail-optimized network architectures. Deep InfiniBand experience, including UFM, subnet management, ibdiagnet, adaptive routing, or SHARP. Experience with Slurm, Kubernetes GPU Operators, Run ai, or similar workload schedulers. Experience with high-performance storage such as Lustre, GPFS, WEKA, VAST, or NFS over RDMA. Experience with liquid-cooled or high-density data center infrastructure. Experience with Prometheus, Grafana, DCGM Exporter, Elasticsearch, or similar observability platforms. RHCSA, RHCE, NVIDIA networking, InfiniBand certifications, CompTIA Server+, Network+, or Uptime Institute certifications. Experience working in a growth-stage environment where processes and standards are being developed. Working Conditions & Physical Requirements (we could include as part of Benefits? Not exactly benefits. Could be a separate section but not in list of headings. Might be okay to include as a paragraph under Requirements? But it's not a qualification. Could be omitted? The instruction says preserve ALL substantive information. So we need to include it somewhere. Could add a section "Working Conditions" but not in the given list. However they said use

            for section headings (e.g. About the role, Responsibilities, Requirements, Qualifications, Skills, Benefits, Pay, Schedule). They gave examples but not exhaustive; we could add other headings if needed. But