Data Engineer (Databricks)

About the role

The Data Engineer will be responsible for designing, building, and maintaining scalable data pipelines using Databricks. They will work closely with data scientists and analysts to ensure that data is accessible and usable for various projects.

Responsibilities

• Design and implement scalable data pipelines using Databricks
• Collaborate with data scientists and analysts to understand requirements and integrate data solutions
• Optimize and maintain existing data pipelines
• Support data engineering tasks including ETL processes, data modeling, and database design
• Participate in code reviews and mentoring junior engineers

Requirements

• Bachelor's degree in Computer Science, Information Systems, or related field
• At least 3 years of professional experience in data engineering
• Experience with Databricks and other big data technologies (e.g., Apache Spark, Hadoop)
• Strong understanding of SQL and NoSQL databases
• Proficiency in Python or Scala
• Experience with cloud platforms (AWS, Azure, Google Cloud)
• Excellent problem-solving and analytical skills
• Ability to work independently and in a team environment

Qualifications

• Experience with machine learning frameworks (TensorFlow, PyTorch, etc.) is a plus
• Knowledge of data warehousing concepts and practices
• Experience with data visualization tools (Tableau, PowerBI, etc.) is a plus

Skills

• Strong programming skills in Python or Scala
• Experience with distributed computing systems
• Understanding of data privacy and security best practices

Benefits

• Competitive salary commensurate with experience
• Flexible remote work options
• Professional development opportunities
• Health insurance benefits
• Retirement savings plan

Pay

• Salary range: $80,000 - $120,000 annually

Schedule

• Full-time position
• Standard 40-hour workweek