SQL Always On Patch Automation Engineer

About the role

SQL Always On Patch Automation Engineer (Contract) Position Summary The SQL Always On Patch Automation Engineer is a contract role on the mission-critical Platform Operations team at HCHB and will build the automation that patches the production SQL Server Always On Availability Group estate today the largest remaining block of enterprise patching still performed by hand under coordinated downtime. This role sits inside HCHB s Server and Endpoint Vulnerability Management program, which is establishing remediation timelines driven by vulnerability severity and asset criticality. Automated patching of Always On Availability Group infrastructure is a named deliverable of that program. The work runs in parallel with the wider replatform of Windows patch automation from legacy SCORCH runbooks onto Ansible Automation Platform (AAP), and is delivered against the same change, approval, and evidence controls as the monthly cycle. The ideal candidate is equally comfortable in SQL Server high availability and in infrastructure automation: someone who understands what an availability group will and will not tolerate during a rolling patch, and who can express that safely and repeatably as code.

Responsibilities

• Design, build, and maintain automation that patches production SQL Server Always On Availability Group clusters without unplanned loss of service.
• Automate availability group discovery so each patch wave is driven from the live state of the estate replicas, roles, synchronization health, and routing rather than a static host list.
• Automate the pre-patch sequence, including removal of secondary replicas from read-only routing, placement of nodes into monitoring maintenance mode, and controlled stop of services on DR availability group instances.
• Orchestrate wave ordering across secondary replicas, DR replicas, and primaries, including failover and failback, so that each customer s availability group remains online throughout the cycle.
• Automate suppression and restoration of SQL Sentry (SentryOne) monitoring alerts, including instances not represented in the configuration database.
• Integrate patch execution with existing JAMS scheduled jobs and the established maintenance-mode job chain.
• Deliver the automation in Ansible Automation Platform as reusable, parameterized job templates, playbooks, and roles, replacing hand-run scripts and legacy SCORCH runbooks.
• Integrate with ServiceNow for change record creation, approval gating, execution status updates, and automated closure.
• Build verification and rollback into every stage, so that a failed node halts the wave rather than propagating, and so that any step can be safely re-run.
• Produce post-patch evidence replica health, synchronization state, and patch level in a form that supports vulnerability SLA and audit reporting.
• Support out-of-cycle and emergency patching of the database tier where a high-severity or CISA Known Exploited Vulnerability finding carries a remediation deadline shorter than the next scheduled cycle.
• Partner with the Database Administration, Security, Infrastructure Operations, and Application teams on maintenance window design and cutover planning.
• Support remediation of database-tier findings raised through Tenable and related vulnerability platforms.
• Follow HCHB change management practice, raising a change record before any production change and validating in non-production ahead of production.
• Maintain source-controlled automation in Azure DevOps, participate in code review, and leave behind runbooks, SOPs, and documentation the permanent team can support.

Skills

• Deep hands-on experience with SQL Server Always On Availability Groups, including replica roles, synchronization modes, read-only routing, listeners, failover and failback, and quorum behavior.
• Strong Windows Server Failover Clustering (WSFC) administration and troubleshooting.
• Proven experience patching or maintaining highly available database infrastructure in production, with a