ETL Developer

About The Role

We are looking for an ETL Developer with strong Python and Airflow expertise to build, manage, and optimize data pipelines. This role focuses on developing reliable workflows that transform and move data across systems efficiently. You’ll be responsible for writing production-ready code, ensuring pipeline performance, and maintaining clean, reusable solutions.

Responsibilities

• Develop, test, and deploy Python-based ETL pipelines using Apache Airflow.
• Write efficient, reusable Python scripts for transformations, validations, and data quality checks.
• Manage scheduling, orchestration, and monitoring of workflows within Airflow.
• Collaborate with data engineers and analysts to design pipelines aligned to business needs.
• Troubleshoot and optimize existing ETL jobs for performance, scalability, and reliability.
• Implement best practices for code quality, testing, and CI/CD integration.
• Contribute to documentation, pipeline observability, and knowledge sharing.

Requirements

• Strong experience with Python (pandas, SQLAlchemy, or similar libraries for ETL).
• Proficiency with Apache Airflow DAG design, task orchestration, and Airflow operators.
• Experience with dependency and environment management tools such as pipenv, poetry, or conda.
• Solid understanding of SQL and relational databases.
• Knowledge of data modeling and transformation patterns (star schema, slowly changing dimensions, etc.).
• Familiarity with Git-based workflows and CI/CD pipelines.
• Ability to work independently and collaboratively in a fast-moving environment.

Nice to Have

• Cloud platform experience (AWS, GCP, or Azure) for data pipelines and storage.
• Familiarity with containerization (Docker, Kubernetes).
• Exposure to data warehouses (Snowflake, BigQuery, Redshift).
• Extensive Databricks experience is ideal.