Senior Data Engineer

Role Responsibilities

• Enterprise Data Architecture & Platform Ownership
• Architect, design, and scale enterprise-grade data solutions using Databricks and Azure cloud services.
• Own and evolve the Databricks environment, including performance optimization, cost management, governance, and security controls.
• Define long-term data architecture strategy aligned to business growth and healthcare compliance, and the evolution of OnMed's CareStation and HomeStation platforms.
• Establish data modeling standards, metadata management practices, and scalable ETL/ELT frameworks.
• Integration Architecture & Interoperability
• Serve as lead integration architect across all system interfaces and API-driven data exchanges.
• Design and optimize integration patterns using Azure Data Factory, Azure API Management, and event-driven architectures.
• Ensure reliable, secure, and scalable interoperability between internal systems, third-party platforms, and healthcare data sources - including HL7/FHIR-based integrations with EHR and clinical systems.
• Oversee monitoring, observability, and fault tolerance across all integration touchpoints.
• Data Engineering & Pipeline Development
• Build and optimize high-performance, scalable data pipelines in Databricks using Python and SQL.
• Design and implement advanced data transformations and complex query optimization.
• Implement automated testing, validation frameworks, and data quality controls to ensure pipeline reliability.
• Drive CI/CD practices and infrastructure-as-code principles for data platform management.
• Governance, Security & Compliance
• Establish and enforce data governance standards using Databricks Unity Catalog, including lineage tracking, auditing, and role-based access controls.
• Ensure compliance with healthcare data privacy and security regulations including HIPAA, HITRUST, and SOC 2 requirements.
• Implement role-based access, encryption standards, and secure API management, and access controls for PHI/PII data handling.
• Cross-Functional Leadership
• Partner with BI, analytics, product, and engineering teams to translate business needs into scalable data solutions.
• Provide architectural guidance and mentorship to junior engineers and analysts.
• Influence data strategy at the leadership level through proactive recommendations and roadmap planning.
• Communicate technical concepts clearly to non-technical stakeholders including clinical, operational, and executive audiences.

Requirements

• Education & Experience
• Bachelor's degree in Computer Science, Information Technology, or a related field.
• 7+ years of progressive experience in data engineering, data architecture, or enterprise integration.
• Bachelor's degree in Computer Science or a related technical field.
• Experience with Databricks and Azure.
• Strong experience designing data models, data pipelines, and integration architectures.
• Advanced SQL and Python skills for ETL, APIs, and workflow automation.
• Hands-on expertise with Azure Data Factory and Azure API Management.
• Experience working with internal system owners, external integration partners, and cross-functional teams around data architecture and API integrations.