Infrastructure Software Engineer

About the role

Building cutting-edge model-specific ASICs requires crafting custom infrastructure and toolchains to support ultra-fast, reliable, and scalable development across the stack - from simulation to silicon. We build this infrastructure as software - and we engineer it with the same best practices we apply to our products. We use the same rigor, design discipline, and quality standards and testing as we do to our ASIC, software, and platform.

Key Responsibilities

• Design and build the orchestration layers that drive our hybrid high-performance clusters—enabling simulation, synthesis, and continuous integration of AI ASICs at unprecedented scale.
• Develop and maintain a fully programmable infrastructure control plane to ensure reproducibility, auditability, and rapid iteration across the entire stack.
• Create tools and abstractions that empower engineers to harness massive parallelism without worrying about the underlying complexity.
• Prototype and execute workload orchestration and migration strategies between on-premise and cloud environments, balancing performance, storage availability and replication, uptime, and cost across heterogeneous hardware and compute backends.
• Implement real-time telemetry, tracing systems that surface insights from millions of metrics, enabling proactive debugging and system optimization.
• Build a full observability stack that includes dashboards, alerting, automated responses, and a synthetic testing framework to proactively test infrastructure performance and reliability for various application and data flows, ensuring we remain proactive against issues impacting development and productivity workflows.

Representative Projects

• Design and deploy a fully automated, scalable hybrid HPC cluster, combining bare-metal servers and switches with cloud instances, provisioned through MaaS and orchestrated via SLURM and Kubernetes, optimized for mixed EDA workloads and parallel CI pipelines.
• Develop a real-time observability system for ASIC toolchain jobs and distributed builds, integrating Prometheus, Grafana, and VictoriaMetrics with streaming telemetry, tracing, and alerting to detect performance regressions before they hit silicon.
• Architect and implement a programmable infrastructure-as-code control plane, using Terraform, Ansible, and Puppet, to version, audit, and redeploy every layer of Etched's development stack with deterministic reproducibility.
• Create a zero-downtime interactive development environment that provisions and connects Jupyter and VS Code sessions to GPUs and high-memory nodes via a secure zero-trust network, abstracting away cluster state and machine failures.
• Prototype and evaluate dynamic workload migration strategies between on-premise and cloud environments to optimize for latency, reliability, and cost across simulation and synthesis pipelines.
• Design a synthetic testing and fault injection framework to validate the behavior of infrastructure under high-load, degraded hardware, and intermittent network partitions - before they happen in production.

Qualifications

• Strong software engineering mindset, quality instincts, and deep understanding of systems.
• Experience in infrastructure engineering, systems programming, or backend software development - ideally in environments where performance, scale, or hardware interaction mattered.
• Expert-level knowledge of Linux, virtualization, containerization, and CI/CD pipelines, with a deep understanding of how to debug, optimize, and scale complex systems.
• Familiarity with Infrastructure as Code tools like OpenTofu, Ansible, or Puppet, and enjoyment of designing declarative, reproducible infrastructure systems.
• Understanding and use of PromQL and other telemetry/query languages and experience with LLM to extract insight from real-time metrics, and knowledge of how to architect and tune observability stacks.
• Experience monitoring, provisioning, and debugging bare-metal servers, network hardware, and high-performance storage systems in rack-scale environments.
• Comfortable in profiling and optimizing compute environments for single-threaded latency, memory-bound workloads, or I/O throughput, especially in the context of simulation or CI performance.
• Proficiency building or operating telemetry systems at scale using Prometheus, Grafana, Loki, VictoriaMetrics, and tools for distributed tracing, log aggregation, and real-time alerting across heterogeneous mediums (SMS, email, push alerts, etc.).

Skills

• Python, Go, Rust, and C++.
• Linux, virtualization, containerization, and CI/CD pipelines.
• Infrastructure as Code tools like OpenTofu, Ansible, or Puppet.
• Prometheus, Grafana, VictoriaMetrics, and distributed tracing, log aggregation, and real-time alerting tools.
• Bazel build system.
• ASIC development flows, especially those involving Synopsys, Cadence, and Verilator.
• Experience architecting systems with AWS, GCP, or Azure, including hybrid on-prem/cloud deployments, workload migration strategies, and cloud-native orchestration tooling.
• Experience monitoring, provisioning, and debugging bare-metal servers, network hardware, and high-performance storage systems in rack-scale environments.
• Proficiency building or operating telemetry systems at scale.

Benefits

• Medical, dental, and vision packages with generous premium coverage.
• $500 per month credit for waiving medical benefits.
• Housing subsidy of $2k per month for those living within walking distance of the office.
• Relocation support for those moving to San Jose (Santana Row).
• Variety of wellness benefits covering fitness, mental health, and more.
• Daily lunch + dinner in our office.
• Unlimited compute budget subject to ROI justification.