Principal Site Reliability Engineer

About the Role

Responsible for designing, implementing, and continuously improving highly reliable, scalable, and resilient platform solutions across the enterprise. Operates as a subject matter expert (SME) in Site Reliability Engineering, driving reliability engineering practices, operational excellence, and automation across the Software Development Lifecycle. Leads complex initiatives, influences enterprise engineering standards, and partners with senior stakeholders to improve system stability, observability, and performance. Serves as a mentor and technical leader for less experienced engineers across Technology.

Responsibilities

• Accountable for defining and driving service reliability standards, including SLOs, SLAs, and error budgets across platforms.
• Design and implement highly available, fault-tolerant architectures aligned with enterprise scalability and resiliency requirements.
• Lead incident management practices, including detection, response, escalation, and recovery processes.
• Drive problem management and root cause analysis to prevent systemic issues.
• Develop and promote observability strategies, including logging, monitoring, alerting, and tracing.
• Lead automation initiatives for self-healing systems and operational workflows.
• Contribute to and review technical roadmaps with reliability and performance considerations.
• Partner with development, infrastructure, cybersecurity, and architecture teams.
• Serve as a technical authority for performance, resilience, and capacity planning.
• Drive production readiness practices including performance testing and failover capabilities.
• Lead cross-team reliability improvement initiatives.
• Participate in and lead post-incident reviews ensuring actionable outcomes.
• Mentor engineers on reliability engineering and best practices.
• Engage with stakeholders to identify risks and optimization opportunities.
• Ensure adherence to risk and regulatory standards and escalate issues when needed.
• Maintain internal control standards and compliance expectations.

Applies expert-level SRE practices across multiple platforms. Drives enterprise-wide reliability improvements and influences technical direction without direct authority.

Requirements

• Associate’s degree and a minimum of 9 years’ systems analysis and/or application development work experience or Bachelor's degree and a minimum of 7 years' systems analysis and/or application development work experience.
• In lieu of a degree, a combined minimum of 11 years’ education and/or relevant work experience, including a minimum of 7 years’ systems analysis and/or application development work experience.
• Expert experience in system design, reliability engineering, and production operations.
• Advanced proficiency in at least one programming or scripting language.

Preferred Qualifications

• Experience with observability and incident management tooling.
• Experience with cloud platforms such as AWS or Azure.
• Strong understanding of CI/CD, DevOps, and SDLC practices.
• Experience defining and implementing SLO/SLI frameworks.
• Experience in regulated environments such as financial services.
• Strong communication and stakeholder management skills.

Pay

The pay range for this position is $139,700.00 - $232,900.00 Annual (USD). The successful candidate’s particular combination of knowledge, skills, and experience will inform their specific compensation.

Location

Buffalo, New York, United States of America