Data Engineering Manager

About the role

Lead data engineering initiatives, managing a team of data engineers to deliver scalable, reliable data solutions.

Responsibilities

• Lead, coach, and develop data engineering teams, setting expectations for delivery and quality.
• Plan and execute data projects across business domains, balancing modernization and operational support.
• Design and implement data pipelines, models, and products based on complex business requirements.
• Provide technical leadership through architecture and code reviews, troubleshooting, and guidance on Databricks, Spark, SQL, and lakehouse design.
• Establish standards for data layers, modeling, testing, source control, and CI/CD processes.
• Lead data migration from legacy systems and integrate data from various sources into cloud environments.
• Ensure data quality, security, observability, lineage, and monitoring in solutions.
• Collaborate with cross-functional teams to clarify requirements, manage dependencies, and communicate risks.
• Maintain operational continuity during system changes, promoting ownership and full lifecycle accountability.

Requirements

• Bachelor’s degree or equivalent experience in computer science, data science, or related fields.
• Extensive experience in data engineering, supporting production pipelines, and leading technical teams.
• Hands-on expertise with Databricks, Spark, PySpark, SQL, and cloud data platforms, especially AWS.
• Knowledge of data modeling, warehousing, and medallion architecture patterns.
• Experience with version control, CI/CD, and integrating diverse data sources.
• Strong communication skills and ability to translate business needs into technical solutions.

Preferred Skills

• Experience with AWS services like S3, IAM, Lambda, and related data tools.
• Familiarity with Databricks features such as Delta Lake, Unity Catalog, and workflows.
• Knowledge of SAP data and Python software engineering practices.