Lead Data Engineer

Overview

We are seeking a Lead Data Engineer with expertise in Apache Spark, Apache Iceberg, Apache Airflow, and AWS to design and build the next generation of our finance data platform. You will own finance-focused data products while contributing to foundational platform capabilities and engineering standards that scale across the enterprise. The ideal candidate is a hands-on engineer who thrives in modern Lakehouse architecture, large-scale pipeline development and analytical mindset.

Key Responsibilities

• Design, develop, and maintain data products
• Ensure data products meet quality, auditability, lineage, and compliance standards
• Build reusable data engineering frameworks and accelerators used across multiple teams
• Develop standardized patterns for ingestion, transformation, orchestration, monitoring, and data quality
• Establish and enforce best practices for Spark, Iceberg, Airflow, and cloud-native engineering
• Drive adoption of self-service platform capabilities and common engineering standards
• Build and optimize Apache Iceberg-based lakehouse solutions for analytical and operational workloads
• Build cloud-native solutions on AWS - S3, EMR, Glue, Lambda, ECS/EKS, CloudWatch
• Implement CI/CD pipelines, automated testing, and Infrastructure-as-Code

Qualifications

• 8+ years of experience in data engineering, data platform development, or related technical roles
• Leading design and implementation of scalable enterprise data solutions and reusable engineering frameworks
• Knowledge of data mesh or data product architectures in enterprise settings
• Advanced hands-on experience with Apache Spark (PySpark) and distributed data processing at enterprise scale
• Deep hands-on experience with Apache Iceberg or similar open table formats
• Solid understanding of CI/CD pipelines, infrastructure-as-code and DevOps practices
• Experience with data governance, data quality frameworks, and metadata management tools
• Strong experience with AWS Cloud services, including services such as Amazon S3, EMR, Glue, Lambda, ECS/EKS, and CloudWatch for developing, deploying, and managing cloud-native data solutions

Pay

• Miami, Florida: $140,000 - $231,000 USD