Senior Site Reliability Engineer

Responsibilities

• Designs solutions to visualize key production support metrics enabling Operational Readiness and Site Reliability Engineer teams to identify scenarios requiring intervention
• Develops software solutions and/or improved processes to address work identified as ‘toil’ by collaborating with key partners to identify, track and remediate processes to free time allocated to reliability
• PARTNERS WITH DEVELOPMENT AND INFRASTRUCTURE TEAMS TO CREATE ERROR BUDGET POLICIES PRIORITIZING RELIABILITY STORIES THAT FALL BELOW SERVICE LEVEL OBJECTIVE (SLO) THRESHOLDS AND SUGGESTS CODE OPTIMIZATIONS, ADDITIONAL INSTRUMENTATION AND/OR LOGGING STRUCTURES TO GAIN SERVICE RELIABILITY VISIBILITY
• IDENTIFIES AND PLANS FOR CAPACITY BOTTLENECKS, VULNERABILITIES AND OPPORTUNITIES FOR RELIABILITY IMPROVEMENT, SUCH AS LOW LEVEL ERROR RATES AND 'NOISE', AND REDUCES MANUAL SUPPORT EFFORT AND/OR IMPROVES SYSTEM RELIABILITY
• ENGAGES AS A SUBJECT MATTER EXPERT IN INCIDENT TRIAGE EFFORTS, FAILURE SCENARIO MODELLING AND WORKS WITH THE PROBLEM MANAGER TO DIAGNOSE ROOT CAUSES FOR COMPLEX/HIGH IMPACT INCIDENT/PROBLEM MANAGEMENT INVESTIGATIONS
• COLLABORATES WITH DEVELOPMENT AND INFRASTRUCTURE TEAMS TO UNDERSTAND TECHNICAL SOLUTIONS AND DEVELOP SERVICE LEVEL INDICATORS AND SLOs TO MEASURE/IMPROVE THE RELIABILITY OF THE SERVICES THEY SUPPORT
• SERVE AS A SENIOR TECHNICAL AUTHORITY FOR PLATFORM RELIABILITY, RESILIENCE, OBSERVABILITY, AUTOMATION, AND PRODUCTION READINESS
• DESIGN AND IMPLEMENT ADVANCED RELIABILITY PATTERNS FOR LANDING ZONES, PRIVATE NETWORKING, DNS, FIREWALLS, REGIONAL RESILIENCY, SERVICE HEALTH CHECKS, AND WORKLOAD ONBOARDING
• LEAD COMPLEX PLATFORM RELIABILITY INITIATIVES SUCH AS SECONDARY-REGION READINESS, EGRESS/INGRESS OBSERVABILITY, PRIVATE DNS RESOLVER MONITORING, GENAI PLATFORM HEALTH CHECKS, AND ENTERPRISE DASHBOARD AUTOMATION
• DEFINE AND MATURE SLIs, SLOs, RELIABILITY INDICATORS, ALERTING STANDARDS, AND SERVICE HEALTH REPORTING FOR AZURE PLATFORM SERVICES
• DEVELOP REUSABLE TERRAFORM MODULES, AUTOMATION FRAMEWORKS, AND CI/CD PATTERNS THAT IMPROVE CONSISTENCY, COMPLIANCE, AND OPERATIONAL QUALITY
• DRIVE OBSERVABILITY IMPROVEMENTS USING LOG ANALYTICS, DYNATRACE, RESOURCE GRAPH, DASHBOARDS, AND ENTERPRISE MONITORING TOOLS
• IDENTIFY SYSTEMIC RELIABILITY RISKS AND TRANSLATE THEM INTO ENGINEERING ROADMAPS, REMEDIATION PLANS, AUTOMATION OPPORTUNITIES, AND OPERATIONAL CONTROLS
• LEAD DEEP TECHNICAL INVESTIGATIONS FOR MAJOR INCIDENTS, RECURRING PROBLEMS, PLATFORM DEFECTS, OR SERVICE DEGRADATION EVENTS
• PARTNER WITH SECURITY AND GOVERNANCE TEAMS TO INTEGRATE IAM, POLICY-AS-CODE, VULNERABILITY REMEDIATION, CONTROL VALIDATION, AND AUDIT READINESS INTO AZURE PLATFORM OPERATIONS
• PARTNER WITH SECURITY AND GOVERNANCE TEAMS TO INTEGRATE IAM, POLICY-AS-CODE, VULNERABILITY REMEDIATION, CONTROL VALIDATION, AND AUDIT READINESS INTO AZURE PLATFORM OPERATIONS
• PROVIDE TECHNICAL DESIGN INPUT FOR NEW AZURE SERVICES AND WORKLOADS TO ENSURE OPERATIONAL READINESS BEFORE PRODUCTION ADoption
• MENTOR SRE ENGINEERS AND RAISE THE TECHNICAL BAR FOR AUTOMATION, TRoubleshooting, DOCUMENTATION, RESILIENCE DESIGN, AND PRODUCTION SUPPORT
• CREATE EXECUTIVE-READY TECHNICAL SUMMARIES, RELIABILITY NARRATIVES, AND RECOMMENDATIONS FOR LEADERSHIP REVIEW
• ESTABLISH REUSABLE STANDARDS FOR RUNBOOKS, DASHBOARDS, HEALTH CHECKS, PROBLEM RECORDS, POST-INCIDENT REVIEWS, AND PRODUCTION-READINESS GATES

Requirements

• 5+ years of experience in platform, systems, or infrastructure engineering, with a strong focus on automation and integration
• Proficiency in SRE best practices; Proven ability to reduce toil and improve observability of the environment
• Experience with automation and orchestration tools (e.g., Ansible or similar), and scripting with golang, Python, or equivalent
• Experience with supporting enterprise service mesh platforms
• Experience with Infrastructure as Code (IaC) concepts and CI/CD pipelines supporting automated builds, validation, and deployments
• Experience integrating provisioning workflows with platform services such as virtualization, networking, identity, monitoring, and configuration management systems
• Strong focus on testing and reliability, including automated integration/validation testing and troubleshooting of complex workflows