Data Platform Lead

Location: Santa Clara, CA | Function: Data Platforms / Advanced Analytics | Role Type: Technical Lead / Solution Lead | Primary Platform: Databricks Lakehouse on Cloud | Scope: EPIC Data Platform (Dedicated Tenant and Multi-Tenant Capabilities)

About the role

The EPIC Data Platform Lead owns the technical direction and implementation leadership for a secure, governed, and scalable cloud data platform built on Databricks. This role combines hands-on data engineering and analytical problem-solving with enterprise architecture leadership across tenant isolation, data governance, identity and access, encryption, observability, production readiness, and platform operations. The lead will translate business, security, and engineering requirements into implementable platform capabilities for internal, customer-dedicated, and controlled multi-tenant use cases.

• Trusted Data Products: Curated, traceable, and analytics-ready data with clear ownership, quality controls, and validation.
• Secure Tenant Boundaries: Validated isolation across workspace, catalog, storage, identity, network, compute, jobs, APIs, and data exports.
• Production-Grade Operations: Observable, supportable, and cost-aware services supported by automated deployment, runbooks, and evidence-based security controls.

Responsibilities

• Platform Architecture & Technical Leadership: Define target architecture, engineering standards, roadmaps, Architecture Decision Records (ADRs), reusable patterns, and non-functional requirements for EPIC Databricks environments. Conduct design reviews and manage trade-offs across performance, security, operability, scalability, and cost.
• Databricks Implementation: Lead the setup and optimization of workspaces, Unity Catalog, Delta Lake, data pipelines, Databricks Workflows, SQL Warehouses, compute policies, external locations, storage credentials, and deployment topologies. Establish maintainable Medallion Architecture (Bronze/Silver/Gold) and production practices.
• Data Engineering & Analysis: Design batch, streaming, and event-driven ingestion, source-to-target transformations, reconciliation, data profiling, exploratory analysis, and root-cause analysis. Validate data latency, completeness, entity linking, accuracy, and business logic outcomes.
• Security by Design: Partner with cybersecurity, IAM, network, and cloud teams to enforce least privilege, SSO/federation, service principal authentication, secrets management, private connectivity (PrivateLink), controlled egress, hardening, and vulnerability remediation.
• Governance & Data Protection: Implement data classification, taxonomies, metadata management, automated lineage, retention rules, fine-grained access controls, row filters, column masks, controlled sharing, DLP-aligned controls, and evidence-driven compliance.
• Encryption & Key Management: Design and implement encryption in transit and at rest, Customer-Managed Keys (CM