Senior Platform Engineer

About the role

The Senior Platform Engineer will oversee the technical direction of Azure and Databricks platforms, ensuring alignment with the bank's Data & AI strategy. Responsibilities include guiding the team, managing infrastructure, implementing DevOps practices, and enhancing platform reliability and security.

Responsibilities

• Provide clear direction to the team, set goals, and keep the team accountable for their deliverables.
• Own the technical direction across Azure and Databricks, including networking and security architecture, Azure Identity & Access Management, and Databricks platform governance.
• Ensure high quality of support delivery for platform users and adherence to platform SLAs/SLOs and service objectives.
• Continually improve platform processes and SOPs for efficiency and automation, designing and developing reusable Terraform modules for Azure and Databricks.
• Build strong relationships with data engineers, analysts, and platform users, communicating with stakeholders and cross-functional teams to align priorities, manage expectations, and drive adoption of platform standards.
• Troubleshoot and resolve performance issues across Databricks jobs, clusters, SQL warehouses, and Azure dependencies, implementing Azure Monitor and Log Analytics for observability.
• Analyze, triage, and resolve platform issues promptly to achieve SLOs and platform reliability objectives, driving error budget awareness and resilience engineering practices.
• Manage incident response, communications, and root cause analysis, providing support during major incidents and after hours.
• Design, build, and deploy logging/monitoring solutions for early detection and actionable insights, standardizing ingestion to Log Analytics from Databricks and Azure resources, and building dashboards and alert rules to reduce MTTR.
• Maintain and enhance the Infrastructure & Platform release pipeline using Terraform, Azure DevOps, and CI/CD tools, enforcing approvals, change windows, and automated checks for safe, repeatable releases.
• Implement CI/CD for infrastructure and analytics workloads, automating Terraform plans, Databricks Bundle releases, and policy validation, ensuring compliance with security scanning.
• Set up Azure Key Vault and HashiCorp Vault for secret management, integrating with Databricks secret scopes and workload identities, and enforcing least privilege access via Azure RBAC and credential rotation.
• Partner with Microsoft and Databricks support and product teams to fine-tune and troubleshoot components, plan upgrades, and adopt new capabilities aligned with the roadmap and enterprise controls.
• Mentor junior engineers in best practices for building, deploying, testing, and supporting services on Azure and Databricks, promoting a culture of automation, documentation, and continuous learning.

Requirements

Required qualifications include 15+ years of IT experience in big organizations, 5+ years of hands-on experience with Microsoft Azure and Databricks, 5+ years with CI/CD, and a Bachelor’s degree in computer science, engineering, mathematics, or a related field. Experience with Terraform, Ansible, and config management is preferred.

Qualifications

Desired qualifications include experience with advanced monitoring and troubleshooting, Site Reliability Engineering, and observability tools development. Knowledge of Azure networking, security, and identity is essential, along with in-depth understanding of databases and data platforms, SDLC, and GitOps.

Skills

Essential skills include proficiency in development/scripting languages, container technologies, and database management. Experience with Azure networking, security, and identity is crucial, as is a strong understanding of Azure and Databricks platforms.

Benefits

Scotiabank offers comprehensive benefits to support your unique health and wellness needs, including flexible work arrangements, accommodation for individuals with disabilities, and a focus on holistic well-being.

Pay

Salary offered may vary based on factors such as relevant knowledge, skills, and experience.

Schedule

Flexible work arrangements are available to accommodate your unique needs.