Senior Staff Network Engineer, Operations
About the role
Crusoe Cloud is seeking a Senior Staff Network Operations Engineer to own production reliability across our global network, including edge, backbone, data center fabric, and GPU cluster interconnects. You will drive incident response, root cause analysis, and the operational excellence initiatives that keep our hyperscale AI infrastructure healthy at scale.
Responsibilities
Own Production Reliability: Serve as the senior technical owner for uptime of Crusoe's global edge, backbone, data center, and GPU cluster network, directly affecting the availability of AI workloads running on hundreds of thousands of GPUs.
Lead Incident Response: Own end-to-end response for high-severity network events, including rapid mitigation, stakeholder communication, and postmortem documentation that prevents recurrence.
Drive Root Cause Analysis: Lead RCAs for production incidents, identify systemic issues, author remediation plans, and track them to closure.
Define SLIs and SLOs: Partner with Architecture and Site Reliability to define network reliability metrics and service level objectives, backed by real-time dashboards and alerting.
Set Operational Standards: Author and maintain runbooks, escalation playbooks, and SOPs used by the broader operations team.
Improve Observability: Drive continuous improvement of Crusoe's network monitoring stack including streaming telemetry, SNMP, NetFlow, and tools such as Kentik, Grafana, Prometheus, and ThousandEyes.
Build Operational Automation: Write Python-based auto-remediation tooling that reduces toil and accelerates mean time to resolution for known failure modes.
Mentor and Multiply: Provide technical guidance to Staff and Senior engineers. Drive post-incident learning and build a culture of operational excellence across the team.
Requirements
12+ years of production network engineering experience with a demonstrated focus on large-scale operations, incident response, and reliability in hyperscale or internet-scale environments.
Observability and Monitoring: Hands-on experience with streaming telemetry, SNMP, NetFlow, sFlow, and tools such as Kentik, Grafana, Prometheus, ThousandEyes, and Arbor.
GPU Cluster and RDMA Networking: Hands-on experience operating RDMA/RoCE (v1 and v2) lossless fabrics for GPU and HPC workloads, including PFC, ECN, and DCQCN tuning.
Skills
Protocol Fluency: Expert hands-on knowledge of BGP, EVPN-VXLAN, IS-IS, OSPF, MPLS, QoS, and TCP/IP across production DC fabric environments at scale.
Hardware Platform Depth: Expert knowledge of Arista (EOS), Juniper (Junos), and NVIDIA/Mellanox platforms in leaf-spine CLOS architectures across multi-vendor environments.
Operational Automation: Proficiency in Python for auto-remediation scripts, diagnostic tooling, and operational workflows that reduce toil and accelerate incident resolution.
SLI and SLO Ownership: Experience defining and owning network reliability metrics and service level objectives in partnership with engineering and product leadership.
Benefits
Competitive compensation
Restricted Stock Units
Paid time off & paid holidays
Comprehensive health, dental & vision insurance
Employer contributions to HSA account
Paid parental leave
Paid life insurance, short-term and long-term disability
Mental health & wellness support
Commuter benefits (parking & transit)
Cell phone stipend
401(k) Retirement plan with company match up to 4% of salary
Volunteer time off
Compensation
Compensation will be paid in the range of $225,000 - $275,000 + Bonus. Restricted Stock Units are included in all offers. Compensation to be determined by the applicant’s education, experience, knowledge, skills, and abilities, as well as internal equity and alignment with market data.
Equal Opportunity Employer
Crusoe is an Equal Opportunity Employer. Employment decisions are made without regard to race, color, religion, disability, genetic information, pregnancy, citizenship, marital status, sex/gender, sexual preference/ orientation, gender identity, age, veteran status, national origin, or any other status protected by law or regulation.