Data Architect, Databricks

Why this role matters

Our team is early in its Databricks maturity journey, and this hire will establish foundational standards, processes, and scaling capabilities across the platform. Our preferred candidate would reside in the Columbus, OH area to support a hybrid work schedule, but we may consider a well-qualified full-remote candidate.

What You'll Do

• Own end-to-end architecture for the Databricks-on-Azure lakehouse platform, including workspace design, Unity Catalog governance, storage topology, and cluster policies
• Define and scale medallion architecture standards (Bronze, Silver, Gold) across domains
• Establish platform standards and reusable patterns to accelerate Databricks adoption across teams
• Design and implement secure, compliant data access frameworks (RBAC, ABAC, row/column-level security)
• Lead CI/CD and DevOps patterns for Databricks deployments, ensuring audit-ready pipelines
• Optimize performance and cost across Delta Lake pipelines (partitioning, clustering, Photon, optimization strategies)
• Build reusable ingestion frameworks for structured and streaming data (API, Kafka, CDC, batch pipelines)
• Partner with Data Governance, Security, SRE, and Legal teams to align platform design with healthcare compliance requirements
• Provide architectural guidance on PHI/PII data handling and lifecycle management
• Evaluate and recommend platform tooling (build vs. buy decisions)
• Mentor engineers and architects; establish design reviews and documentation practices

Basic Requirements

• Bachelor's degree in Computer Science, Information Systems, or a related field, or relative experience, and typically requires 7+ years of experience in data engineering and data architecture
• Proven experience building and scaling Databricks platforms from the ground up, including workspace provisioning, Unity Catalog, and governance frameworks
• Strong hands-on expertise in Databricks and Azure (ADLS Gen2, networking, identity, security)
• Demonstrated ability to remain technically hands-on while leading architecture decisions
• Deep expertise in Delta Lake, including performance optimization and data lifecycle management
• Experience designing enterprise data platforms in regulated environments (e.g., PHI/PII handling)
• Strong understanding of data governance, access controls, and compliance frameworks
• Proven ability to establish standards, drive adoption, and influence multiple engineering teams

Preferred Skills & Experience

• Experience in healthcare or other regulated industries (HIPAA, PHI, compliance-driven environments)
• Experience scaling early-stage Databricks platforms and building enterprise-wide standards
• Familiarity with streaming and event-driven architectures (Kafka, Structured Streaming)
• Experience with Infrastructure-as-Code tools (Terraform, Azure Bicep)
• Knowledge of Databricks advanced features (Delta Live Tables, Unity Catalog federation, Delta Sharing)
• Experience supporting AI/ML data readiness and feature engineering pipelines