Member of Technical Staff, Platform Infrastructure
Edison Scientific · San Francisco, CA · Yesterday
On-siteEngineering$200k–$350k/yrFull-time
Responsibilities
- Arcitect, implement, and operate Kubernetes clusters that support thousands of concurrent, persistent resources (agents, jobs, services) with high availability and efficient resource utilization.
- Design and develop custom resource definitions (CRDs) and Kubernetes operators to model and manage domain-specific workloads such as AI agent lifecycles, research pipelines, and long-running compute tasks.
- Drive the strategy for cluster scaling, node pool management, autoscaling policies, and resource quota frameworks to handle rapid workload growth.
- Build and maintain infrastructure-as-code (Terraform, Pulumi, or similar) for reproducible, version-controlled environment management.
- Design and implement robust scheduling, placement, and affinity strategies to optimize cost, performance, and fault tolerance for heterogeneous workloads (CPU, GPU, memory-intensive).
- Establish and uphold best practices around observability, monitoring, alerting, and incident response for infrastructure systems (Prometheus, Grafana, Datadog, or similar).
- Own storage and networking strategy within Kubernetes — including persistent volume management, CSI drivers, service mesh, network policies, and ingress architecture.
- Troubleshoot complex, cross-system infrastructure issues and guide others through effective debugging and remediation in distributed environments.
- Collaborate closely with backend, ML, and research teams to understand workload requirements and translate them into reliable infrastructure patterns.
Qualifications
- Typically, 5+ years of professional infrastructure or platform engineering experience, with deep hands-on Kubernetes expertise in production environments.
- Experience designing and implementing custom resource definitions (CRDs) and Kubernetes operators (using frameworks such as Kubebuilder, Operator SDK, or controller-runtime).
- Track record of operating and scaling Kubernetes clusters supporting thousands of persistent or long-lived resources (stateful workloads, persistent pods, long-running jobs).
- Deep understanding of Kubernetes internals — API server, etcd, scheduler, controller manager, kubelet — and how they behave at scale.
- Expertise with cloud infrastructure (AWS EKS, GCP GKE, or Azure AKS) and associated networking, storage, and IAM primitives.
- Proficiency in at least one systems or backend language for operator development and infrastructure tooling.
- Hands-on experience with infrastructure-as-code tools (Terraform, Pulumi, or Crossplane) and GitOps workflows.
- Strong working knowledge of container networking (CNI plugins, service mesh, network policies), storage (CSI, persistent volumes, StatefulSets), and security (RBAC, Pod Security Standards, secrets management).
- Ability to operate autonomously, make sound technical judgments, and drive projects from concept through production.
- Bonus points for Experience with data-intensive platforms, scientific computing, or ML/AI infrastructure.
- Prior experience in startups or small teams with significant architectural ownership and ambiguity.
- Experience scaling systems, teams, or platforms through periods of rapid growth.