Senior/Staff Backend Engineer - Distributed System

About Us

At Zettabyte, we’re on a mission to make AI compute ubiquitous, seamless, and limitless. We’re building a cloud where AI just works—anywhere, anytime. “AI Power. Everywhere.” Be part of the team designing the infrastructure for the AI-first world.

Why this role exists

We need a Backend Engineer to build the systems that orchestrate GPU clusters for AI workloads. You'll create APIs that handle GPU allocation, memory management, compute scheduling, and multi-tenant isolation—challenges unique to AI infrastructure that go far beyond typical backend engineering.

What you’ll do

• Design APIs that abstract complex GPU operations into simple developer experiences
• Build scheduling algorithms that maximize GPU utilization while ensuring SLA compliance
• Develop resource management systems for GPU lifecycle—provisioning, allocation, scheduling, and release
• Create usage tracking and billing systems for GPU-hours, memory usage, and compute utilization
• Implement monitoring for GPU-specific metrics, health checks, and automatic failure recovery
• Create multi-tenancy systems with resource isolation, quota management, and fair scheduling
• Optimize cold starts for model serving and implement efficient model loading strategies
• Collaborate with frontend engineers to expose complex infrastructure through intuitive interfaces
• Leverage AI-assisted coding tools (GitHub Copilot, Claude Code, Cursor IDE, etc.) to boost productivity and code quality

You’ll thrive here if you:

• Have 5+ years backend engineering experience with distributed systems
• Are proficient in Go, Python, or similar backend languages
• Have experience with resource scheduling, orchestration, and API design (REST, GraphQL, gRPC)
• Understand hardware constraints and system optimization
• Have Linux systems knowledge and containerization experience (Docker, Kubernetes)
• Are comfortable working with expensive resources where efficiency directly impacts costs
• Are excited about solving novel problems in AI infrastructure (not just another CRUD app)
• Have a startup mindset—comfortable with ambiguity and rapid iteration

Bonus Qualifications

• Have GPU or HPC cluster management experience
• Understand ML/AI workload patterns and requirements
• Have experience with high-value resource allocation systems
• Have background in performance optimization for compute-intensive workloads
• Have familiarity with GPU virtualization and sharing technologies
• Have experience building billing or metering systems