Data Engineer

Data Pipeline & Integration

Data Pipeline & Integration

• Design, build, and maintain end-to-end data pipelines using Databricks (SparkSQL, PySpark) for data ingestion, transformation, and processing.
• Integrate data from various structured and unstructured sources, including medical imaging systems, EMRs, Change-Data-Capture from SQL Databases, and external APIs.

Analytics & Visualization

Analytics & Visualization

• Collaborate with the analytics team to create, optimize, and maintain dashboards in Looker.
• Implement best practices in data modeling and visualization for operational efficiency.

Cloud Infrastructure

Cloud Infrastructure

• Deploy and manage cloud-based solutions on AWS (e.g., S3, EMR, Lambda, EC2) to ensure scalability, availability, and cost-efficiency using IaC tooling (Terraform and Databricks Asset Bundles).
• Develop and maintain CI/CD pipelines for data-related services and applications.

Database Management

Database Management

• Oversee MongoDB and PostgreSQL databases, including schema design, indexing, and performance tuning.
• Ensure data integrity, availability, and optimized querying for both transactional and analytical workloads.

Security & Compliance

Security & Compliance

• Adhere to healthcare compliance requirements (e.g., HIPAA) and best practices for data privacy and security.
• Implement error handling, logging, and monitoring frameworks to ensure reliability and transparency.
• Implement data governance frameworks to maintain data integrity and confidentiality.

Collaboration & Documentation

Collaboration & Documentation

• Work cross-functionally with data scientists, product managers, and other engineering teams to gather requirements and define data workflows.
• Document data pipelines, system architecture, and processes for internal and external stakeholders.

About you

About you

• Education & Experience
• 7+ years of professional experience in data engineering or a similar role.
• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
• Technical Skills
• Databricks (Spark): Proven expertise in building large-scale data pipelines.
• Looker: Experience in creating dashboards, data models, and self-service analytics solutions.
• AWS: Proficient with core services like S3, EMR, Lambda, IAM, EC2, etc.
• MongoDB & PostgreSQL: Demonstrated ability to design schemas, optimize queries, and manage high-volume databases.
• SQL & Scripting: Strong SQL skills, plus familiarity with Python, Scala, or Java for data-related tasks.
• Soft Skills
• Excellent communication and team collaboration abilities.
• Strong problem-solving aptitude and analytical thinking.
• Detail-oriented, with a focus on delivering reliable, high-quality solutions.
• Preferred
• Experience in healthcare or imaging (e.g., DICOM, HL7/FHIR).
• Familiarity with DevOps tools (Docker, Kubernetes, Terraform) and CI/CD pipelines.
• Knowledge of machine learning workflows and MLOps practices.

Pay

Pay

• The base salary range for this position is $120,000 - $180,000.

Schedule

Schedule

• Not specified.