Lead Data Engineer

About the role

The Lead Data Engineer will be responsible for designing, implementing, and maintaining large-scale data processing pipelines. They will also lead the team in developing scalable data solutions and ensuring data quality.

Responsibilities

• Design and implement high-performance data pipelines using Apache Kafka, Spark, and Hadoop
• Collaborate with cross-functional teams to understand business needs and translate them into technical solutions
• Optimize data storage and retrieval processes to improve efficiency and reduce costs
• Develop and maintain data models and schemas to ensure consistency and integrity across the organization
• Monitor and troubleshoot data pipelines to identify and resolve issues promptly
• Stay up-to-date with emerging technologies and best practices in data engineering

Requirements

• Bachelor's degree in Computer Science, Information Systems, or a related field
• At least 5 years of professional experience in data engineering
• Experience with big data technologies such as Hadoop, Spark, and Kafka
• Strong understanding of SQL and NoSQL databases
• Proficiency in Python, Java, or Scala
• Excellent problem-solving and analytical skills
• Ability to work independently and as part of a team

Qualifications

• Experience with cloud platforms like AWS, Google Cloud, or Azure
• Knowledge of data warehousing tools such as Redshift or BigQuery
• Experience with machine learning and data analytics
• Good communication and presentation skills

Skills

• Advanced knowledge of data structures and algorithms
• Experience with distributed systems and microservices architecture
• Understanding of data governance and compliance regulations

Benefits

• Competitive salary package
• Flexible working hours
• Professional development opportunities
• Health insurance
• Employee discounts

Pay

Salary range: $80,000 - $120,000 per year

Schedule

Full-time position