Senior Data Engineer

About the role

The Senior Data Engineer designs, builds, and maintains scalable data pipelines, transformations, and data products within a secure, regulated cloud environment. This fully remote role supports the development of a next-generation data and AI platform and collaborates with engineering, analytics, and data science teams to deliver reliable, governed, and analysis-ready data solutions.

Responsibilities

• Design, build, test, deploy, and maintain data ingestion, transformation, and pipeline solutions.
• Develop governed, analysis-ready datasets from raw, structured, unstructured, and synthetic data sources.
• Enhance and scale the data platform in alignment with established architectural standards and engineering best practices.
• Contribute to Infrastructure as Code using Terraform and support CI/CD pipelines for production deployments.
• Monitor data quality, pipeline reliability, and platform performance.
• Troubleshoot and resolve data pipeline, integration, and operational issues.
• Implement data governance, cataloging, lineage, and access-control practices.
• Support data science, analytics, and machine learning teams by delivering reliable datasets and supporting tools.
• Follow applicable security, privacy, data handling, and compliance requirements.
• Develop and maintain technical documentation for data pipelines, processes, architecture, and platform capabilities.
• Collaborate with engineering and cross-functional teams to support platform delivery and continuous improvement.

Required Qualifications

• 5+ years of experience designing, building, and maintaining production data pipelines.
• Hands-on experience with Databricks and Unity Catalog.
• Strong proficiency with Python and SQL.
• Experience developing ETL or ELT solutions and scalable data transformation workflows.
• Strong knowledge of data modeling principles and best practices.
• Experience with cloud-based data platforms; Azure experience is preferred, with AWS or Google Cloud experience also applicable.
• Experience with Infrastructure as Code tools, particularly Terraform.
• Experience with CI/CD pipelines, Git-based version control, and production deployment workflows.
• Familiarity with data governance, cataloging, lineage, and access-control technologies.
• Experience with orchestration and transformation tools such as Airflow or dbt.
• Strong analytical, problem-solving, communication, and documentation skills.

Preferred Qualifications

• Experience with Apache Spark, Delta Lake, dbt, Snowflake, or comparable lakehouse and data platform technologies.
• Experience working in secure, regulated, or accredited environments.
• Familiarity with FedRAMP, NIST 800-171, CMMC, or Controlled Unclassified Information requirements.
• Experience supporting government, defense, or other highly regulated programs.
• Ability to work independently and collaborate effectively with technical and non-technical stakeholders.
• Strong attention to detail and commitment to knowledge sharing and continuous improvement.

Education

Bachelor’s degree in Computer Science, Software Engineering, Information Systems, or a related technical field, or equivalent relevant education and practical experience.

Additional Requirements

• This is a fully remote position.
• This position requires eligibility for a U.S. Government security clearance.
• In accordance with federal law, U.S. citizenship is required.

Pay

Publishing Pay Range: $52.00 – $57.00 hourly