Principal Site Reliability Engineer

Responsibilities

• System Reliability and Operations (SRE Focus)
• Architect, build, and maintain scalable, highly available, and reliable cloud infrastructure in AWS leveraging modern container orchestration technologies.
• Platform Design and Management
• Serve as the reliability and cost optimization expert for high-volume, data-intensive workloads. Focus on optimizing and ensuring the stability of distributed data processing engines, specifically Apache Spark and related ecosystems (e.g., EMR, Databricks, Glue).
• Data Pipeline Reliability
• Establish comprehensive observability practices by defining SLIs/SLOs, implementing advanced monitoring, alerting, and logging solutions to quickly identify and resolve system anomalies.
• Observability and Monitoring
• Drive automation across all operational aspects, including infrastructure provisioning (Terraform), scaling, deployment, and incident response, minimizing toil and manual effort.
• Incident Management
• Lead and participate in the incident response lifecycle, performing thorough post-mortems to derive actionable insights and implement preventative measures to improve system resilience.
• AIOps
• Define and champion the strategic roadmap for AI/ML integration within SRE, establishing organizational best practices for AIOps, automated incident remediation, Toil Reduction via LLMs, and Automated Root Cause Analysis (RCA) and the governance of LLM-driven tooling to enhance system observability and resilience.
• Developer Experience and Productivity (DevEx Focus)
• Platform Strategy
• Design, implement, and champion self-service tools, internal developer portals, and services that empower engineering teams to manage their infrastructure and deployments independently and efficiently.
• AI Developer Tools
• Lead the standardization of AI developer assistants by architecting and maintaining global 'steering files' and context-configuration standards, ensuring AI-generated code aligns with our specific patterns, security protocols, and architectural guardrails.
• CI/CD Optimization
• Own and continuously improve the CI/CD pipelines, reducing build times, streamlining deployment workflows, and integrating best practices for testing, security (Shift Left), and code quality. Maintain and improve our container orchestration and deployment tools, leveraging Kubernetes, Helm, and ArgoCD to create seamless developer workflows.
• KPIs
• Develop, implement, and maintain a set of key performance indicators (KPIs) to measure and improve the developer experience across all of Engineering.
• Mentorship and Documentation
• Guide and mentor senior engineers, promoting SRE/DevEx principles. Develop clear, comprehensive documentation and tutorials to ensure seamless adoption of new tools and platforms.
• Cost and Efficiency
• Strategically identify and implement opportunities for cloud cost optimization and resource efficiency without compromising reliability or performance.

Qualifications and Education Requirements

• Bachelor's degree in Computer Science, Engineering, or a related technical field, or equivalent practical experience.
• 10+ years of relevant experience in software engineering, cloud architecture, and/or Site Reliability Engineering, with at least 3 years in a leadership or lead contributor role.
• Deep expertise of AWS, including EKS, ECR, RDS, SQS/SNS, VPC, MWAA and S3.
• Strong proficiency in Infrastructure as Code (IaC) tools (e.g., Terraform, CloudFormation).
• Specialized experience in optimizing large-scale data platforms, specifically with Apache Spark. Proven ability to profile, troubleshoot, and tune Spark jobs for performance, cost, and reliability.
• 5+ years of experience with Kubernetes and containerization in general, including associated tools (kubectl, Helm, ArgoCD).
• Strong knowledge of AWS cost optimization.
• TCP/IP networking, including routing and AWS security groups.
• Excellent knowledge of CI/CD concepts and experience developing associated pipelines in CircleCI.
• Proficient in high-level scripting languages, including shell scripting, Python, and/or JavaScript.
• Experience with OTel and monitoring tools such as Splunk or DataDog. Experience with native AI observability tools is a plus.
• Experience with evaluating and rolling out GenAI tools for improving developer efficiency.
• Excellent communication, collaboration, and stakeholder management skills, with proven experience driving technical initiatives across multiple teams.
• Experience with researching and selecting new/modern developer toolsets and assisting teams in adopting them including vendor assessments, security assessments and procurement process.
• Experience in Ad-Tech or “BIG Data” processing organization is highly preferred.

Pay

Target cash compensation range: $163,620 - $212,710 USD Annually