Software Engineer, Infrastructure

About the role

We're looking for a Senior Software Engineer (Infrastructure) to own the systems behind CVI, our real-time conversational product. Your responsibilities will include:

• Ownership of CVI's inference deployments.
• GPU infrastructure serving live conversations across multiple providers and regions.
• Joining as an early senior member of a growing infra team, working on projects like tuning the newest GPU generations and cutting cold-start and model load times.
• Expanding our GPU footprint.
• Broadening our provider and region coverage, setting up clusters on EKS, and building the routing, scheduling, and throughput needed for fast weight loading.
• Pushing our uptime bar higher, ensuring security and SOC2 compliance.
• Fixing issues as you find them, reworking our deploy pipeline for faster and more efficient shipping.

Requirements

• Hands-on GPU inference experience, including deployment and optimization.
• Kubernetes and EKS expertise, including routing and scheduling.
• Deep AWS experience, including spinning up new services and turning them into simple, repeatable processes.
• A senior track record of ownership, setting technical direction and carrying ambiguous work over the finish line.

Nice to have

• Experience with GCP.
• Experience with video streaming infrastructure.
• Experience with training infrastructure or Large Language Model (LLM) serving.
• Experience with SOC2 or security compliance.