AWS & Databricks Platform Architect

About the role

As an experienced AWS & Databricks Platform Architect, you will work in a collaborative environment under Deloitte’s Project Delivery Talent Model (PDM), a model tailored for long-term, onsite client service delivery with minimal travel demands. You will design, build, and maintain scalable data pipelines and analytics solutions on the Azure platform, partnering with data scientists, analysts, and engineers to drive data-driven innovation.

Responsibilities

• Design and implement secure, scalable Databricks platform architectures on AWS, including workspace deployment, networking, IAM, and PrivateLink connectivity.
• Build and maintain Terraform-based infrastructure automation for AWS and Databricks resources to support standardized and repeatable platform delivery.
• Define and apply platform security patterns across identity, access control, secret management, network security, encryption, and data protection.
• Lead architecture discussions, technical workshops, and enablement sessions with customer and internal teams to guide solution design and implementation.
• Architect multi-account and multi-workspace environments that support strong isolation, shared services, and environment separation across dev, test, and production.
• Partner with stakeholders across security, infrastructure, and data teams to support governance, resiliency, compliance, and operational readiness.
• Provide technical guidance and support to other team members.
• Communicate regularly with Engagement Managers, project team members, and functional/technical teams, escalating matters as needed.
• Independently and collaboratively lead client engagement workstreams focused on process improvement, optimization, and transformation.

Qualifications

• Strong understanding of Databricks classic and serverless workspace deployments on AWS, including S3, IAM, VPC design, and both front-end and back-end PrivateLink connectivity.
• 6+ years of hands-on Terraform experience across both AWS and Databricks resources to deploy, configure, and manage secure workspace infrastructure.
• Working knowledge of Databricks platform security architecture, including workspace isolation, identity integration, cluster access controls, secret management, and network security patterns.
• Threat model fluency—ability to articulate protect/detect/respond controls for all seven Databricks threat categories (account takeover, data exfiltration, insider threats, supply chain attacks, Databricks compromise, ransomware, resource abuse).
• Strong understanding of data warehousing concepts and ETL/ELT processes.
• Bachelor’s degree in Computer Science, Engineering, Information Systems, or related field.
• Ability to travel 10%, on average, based on client needs.

Preferred Qualifications

• Prior experience taking customer teams through Databricks security accreditation process.
• Deep understanding of multi-workspace topologies—hub-and-spoke VPC designs, Transit Gateway integration, shared services VPCs, and cross-account network connectivity patterns for environment isolation (dev/staging/prod).
• Knowledge of data exfiltration prevention architectures—S3 VPC endpoint policies, restrictive bucket policies, STS condition keys, regional restrictions, and Databricks data plane traffic flows.
• Expertise in AWS PrivateLink at scale, including endpoint services, interface endpoints, DNS resolution chains (Route 53 Private Hosted Zones, inbound/outbound resolvers), and troubleshooting connectivity in complex multi-account Landing Zone architectures.
• Deep knowledge of IAM trust chains in Databricks deployments—cross-account assume-role patterns, instance profiles vs. credential passthrough, IAM Roles for Service Accounts (IRSA), and the Databricks-managed IAM role boundary model.
• Understanding of credential vending and Unity Catalog’s storage credential architecture—temporary credential scoping, session policies, external location grants, and S3 access patterns auditable by CloudTrail.
• Expertise in encryption architecture—CMK for workspace storage, DBFS root, managed services (notebook/secret encryption), EBS encryption, and S3 SSE-KMS with key policies restricting decrypt to specific principals; key rotation implications.
• Understanding of data classification and governance controls—Unity Catalog row/column-level security, attribute-based access control patterns, dynamic views, and regulatory compliance (OCC, FFIEC, SOX).
• Understanding of Databricks control plane/data plane separation—what data leaves the customer’s AWS account, metadata stored in the Databricks-managed control plane, and articulation to security review boards.
• Knowledge of disaster recovery and high availability patterns—workspace regional failover, metastore replication, cross-region S3 replication, and RTO/RPO implications for Databricks-dependent pipelines.
• Familiarity with patch management and image hardening—Databricks Runtime AMI lifecycle, custom container support (Docker on Databricks), CIS benchmark applicability, and addressing vulnerability scanning findings on ephemeral compute.
• Understanding of multi-tenancy isolation guarantees—Databricks workload isolation at compute, storage, and network layers, and additional controls (dedicated VPCs, dedicated control plane for large deployments).
• Knowledge of identity federation patterns—SCIM provisioning, SAML/OIDC integration, token lifecycle management, and Databricks PAT/OAuth tokens interaction with enterprise session management.
• Enterprise Secrets Vault Integration: Practical experience architecting interim and native secrets retrieval pipelines using HashiCorp Vault or CyberArk for rotated service principal client secrets and runtime on-premises database credentials.
• Very strong understanding of Terraform modules for Databricks and AWS.

Skills

• Meticulous attention to detail and quality of work product.
• Ability to build and sustain professional relationships.
• Ability to lead projects or workstreams.
• Ability to manage and prioritize multiple tasks in a fast-paced and dynamic environment.
• Strong interpersonal skills and professional demeanor.
• Ability to meet deadlines.
• Ability to provide clear guidance to others.

Benefits

This position is aligned with the Project Delivery Model. To view the associated benefit package, please reference this document.

Deloitte offers opportunities for professional development, mentorship, and leadership growth. Our inclusive culture empowers individuals to contribute unique perspectives and make a collective impact.