AWS Databricks Lead

About the role

Design modern data architectures on Databricks, Snowflake, and Microsoft Fabric. Lead end-to-end data ingestion, transformation, and modeling using SQL, Spark, and Python. Support healthcare data domains: claims, eligibility, provider, member, clinical data. Contribute to pre-sales through RFP responses, estimations, architecture diagrams, and proposals.

Responsibilities

• Design modern data architectures on Databricks, Snowflake, and Microsoft Fabric
• Lead end-to-end data ingestion, transformation, and modeling using SQL, Spark, and Python
• Support healthcare data domains: claims, eligibility, provider, member, clinical data
• Working knowledge of various governance patterns: lineage, catalog
• Contribute to pre-sales through RFP responses, estimations, architecture diagrams, and proposals

Required Skills

• Strong hands-on experience with Databricks (Spark/Delta), Snowflake, or Microsoft Fabric
• Cloud experience with at least one major provider: Azure, AWS, or GCP (preferably GCP)
• Strong SQL, PySpark/Python, and data modeling (dimensional, lakehouse)
• Working experience in at least one healthcare data domain: claims, provider, eligibility, utilization, or clinical data
• Solid understanding of data governance, security, HIPAA, PHI/PII compliance
• Ability to lead client conversations and explain architectural trade-offs clearly
• Exposure to FHIR/HL7, interoperability, and payer/provider workflows

Qualifications

• Must have implemented data products on a Lakehouse either in a data mesh or data fabric construct in a production environment
• Must have utilized AI tools for data engineering and data analytics use cases like data copilot, AI/BI, etc.
• Must have experience in managing implementation life cycle from discovery through hyper care, transient and steady state data ops, and optimization
• Ability to participate and advise clients on well-architected framework together with partners like Databricks, Snowflake, etc.
• Strong understanding and implementation experience of production-like synthetic data sets in non-production environments to facilitate functional and integration tests
• Exposure to leading healthcare source systems like Pega, EHR, Salesforce Health Cloud and common information data models
• Exposure to managing and processing unstructured data sets in healthcare to derive meaningful insights and enhance experience