Data/Software Engineer

Key Responsibilities

• Design, develop, and maintain scalable end-to-end data pipelines using Python and modern data engineering frameworks.
• Build and optimize ETL workflows that process structured, unstructured, and geospatial data at scale.
• Deploy and manage containerized applications within AWS cloud environments.
• Develop and optimize SQL and NoSQL data solutions, including database design, performance tuning, and query optimization.
• Integrate AI/ML models and services into production data workflows.
• Utilize orchestration tools to automate data pipelines and manage Spark jobs.
• Maintain data governance, lineage, cataloging, and compliance best practices across enterprise datasets.
• Collaborate with stakeholders to gather requirements, design technical solutions, and support platform modernization initiatives.
• Troubleshoot pipeline failures, performance bottlenecks, and data quality issues while contributing to engineering best practices and technical documentation.

Required Qualifications

• 5+ years of experience building production data pipelines and scalable ETL solutions.
• Advanced Python development experience, including Pandas and NumPy.
• Experience developing with Apache Spark and PySpark.
• Experience deploying containerized applications using Docker or Podman.
• Experience working with AWS services including S3, Lambda, and Step Functions.
• Strong SQL experience, including Trino, MySQL, and PostgreSQL with performance tuning and schema design.
• Experience working with NoSQL technologies such as DynamoDB.
• Experience with workflow orchestration tools including Apache Airflow and Apache Iceberg.
• Familiarity with Infrastructure as Code using Terraform or CloudFormation.
• Experience using Git and modern source control practices.
• Experience with Bash scripting for automation and data processing.
• Understanding of data governance, security, privacy, lineage, and compliance principles.

Preferred Qualifications

• Experience with Unity Catalog OSS, Apache Polaris, and OpenLineage.
• Experience with Apache Superset or similar data visualization platforms.
• Experience working with geospatial data technologies including H3 and PostGIS.
• Experience supporting AI/ML model deployment and production data workflows.
• Experience supporting large-scale data migration or platform modernization initiatives.