Senior Data Engineer

Primary Responsibilities

• Design, develop, and maintain scalable data pipelines using Python, PySpark, and other modern programming languages to support both batch and streaming workloads
• Build and optimize data processing frameworks on cloud platforms such as Databricks or Snowflake, ensuring performance, reliability, and cost efficiency
• Create and manage workflow orchestration using Apache Airflow or similar tools, ensuring reliable scheduling, dependency management, and monitoring
• Implement and enforce data governance and metadata standards through tools such as Microsoft Purview, including data lineage, classification, cataloging, and security policies
• Create automated data quality and validation frameworks to ensure accuracy, completeness, and reliability of production datasets
• Collaborate with cross-functional teams including data architects, analysts, scientists, and business stakeholders to understand requirements and deliver scalable, well-designed data solutions
• Lead technical design sessions and code reviews, promoting engineering best practices, reusability, and maintainability
• Support cloud infrastructure and DevOps practices, including CI/CD pipelines, version control, testing automation, and environment management
• Monitor and troubleshoot production data pipelines, proactively addressing issues, performance bottlenecks, and system failures
• Contribute to the evolution of the enterprise data platform, recommending tools, frameworks, and architectures to improve scalability and efficiency

Required Qualifications

• Bachelor's Degree in CS or IT related field
• 7+ years of experience in data engineering or software engineering
• 5+ years of SQL skills and experience with OLTP and OLAP data modeling
• 2+ years of experience working with cloud ecosystems (Azure preferred; AWS/GCP acceptable)
• 2+ years of experience in Python, PySpark, or similar languages
• 2+ years of hands-on experience in Snowflake
• 1+ years of experience with orchestration tools such as Apache Airflow or similar
• Must be able to obtain and maintain a suitability or determination of eligibility for a Confidential/Secret or Top-Secret security clearance
• United States Citizenship required for this position

Preferred Qualifications

• Healthcare industry experience, including claims, clinical, FHIR, HL7, or provider data
• Experience with containerization (Docker, Kubernetes) for data workloads
• Experience supporting machine learning workflows or analytical data science pipelines
• Familiarity with data governance tools, especially Microsoft Purview
• Knowledge of distributed computing concepts and performance tuning