Alert Management & Observability

Key Responsibilities

Alert Rationalization & Prioritization (Core)

• Evaluate alerts for business/service criticality and operational priority
• Remove or suppress low-value alerts
• Ensure clear escalation paths and ownership
• Perform regular alert reviews to maintain quality and alignment
• Reduce alert fatigue while preserving critical incidents

Standards, Policies, and Guardrails

• Define and enforce alerting standards including severity definitions, metadata, naming conventions, and routing rules
• Create a standardized Alert Design Checklist and approval workflow
• Partner with tool/platform owners to embed standards in monitoring tooling
• Determine which alerts should go to 24x7 Eyes-on-Glass, on-call engineering, or be suppressed

Routing Decisions to 24x7 Eyes-on-Glass

• Ensure routing aligns with operational responsibilities, department priorities, and service ownership
• Act as gatekeeper or lead governance process for alert routing

Runbook / Response Instruction Cataloging (Knowledge System)

• Create runbooks for every actionable alert
• Include symptoms, triage steps, remediation actions, and escalation triggers
• Link to dashboards, logs, SOPs, and known issues
• Maintain and review runbooks on a defined cadence

Reporting & Operational Outcomes

• Define and publish KPIs to measure alerting health and operational performance
• Facilitate governance forums with service owners and engineering leads
• Drive adoption of observability patterns and improve incident correlation

Cross-Functional Enablement

• Coach service teams on best practices in SLIs/SLOs, alert thresholds, dependency monitoring, and incident correlation
• Support major incident learning by refining alerts and runbooks

Required Qualifications

• 5+ years in IT Operations, SRE, Observability, Monitoring Engineering, or Incident Management
• Experience with common monitoring/observability tools
• Strong understanding of incident response workflows and operational coverage models
• Experience with automation for alert enrichment, correlation, and routing
• Experience with governance frameworks and operating rhythm design

Preferred Qualifications

• Experience with Operations Command Center / NOC / SOC-style “eyes-on-glass” model
• Familiarity with ITIL Event Management, SRE principles, and service reliability practices
• Experience with automation for alert enrichment, correlation, and routing
• Background in governance frameworks and operating rhythm design

Skills

• Ownership
• Alert management
• “Glass” (24x7 operations)
• Runbook cataloging
• Management
• Correlation
• 24x7 operations
• Teams
• Clear communication
• Observability
• Routing