Sr. Data Engineer

Responsibilities

• Design, build, and maintain scalable data pipelines and analytics solutions for big data environments.
• Develop and optimize workflows that support efficient, reliable data processing and delivery.
• Closely work with cross-functional partners to translate data needs into actionable technical solutions.
• Support data platform operations with a focus on quality, availability, performance, and stability.
• Apply cloud and DevOps practices to automate data workflows and improve delivery efficiency.
• Troubleshoot issues across data pipelines and environments to help ensure smooth day-to-day operations.
• Contribute to data governance, security, and compliance practices within the data ecosystem.
• Share knowledge and help mentor others while promoting collaborative ways of working.

Requirements

• 8+ years of experience in data engineering with a strong big data background.
• Hands-on experience with Databricks and Azure Data Factory.
• Experience with DevOps tools and practices related to data engineering and CI/CD.
• Strong working knowledge of Hadoop, Spark, Kafka, or similar distributed data technologies.
• Proficiency in Python, Scala, or Java.
• Solid understanding of SQL and NoSQL databases.
• Experience building ETL processes, data pipelines, and real-time or batch data solutions.
• Strong problem-solving skills and the ability to communicate effectively with technical and business stakeholders.

Preferred Skills

• Experience with cloud data platforms such as Azure, AWS, or GCP.
• Knowledge of Flink or other streaming/data processing frameworks.
• Familiarity with data modeling and distributed computing concepts.
• Background supporting machine learning pipelines or AI-enabled data solutions.
• Experience in Agile environments and with team leadership or mentoring.
• Knowledge of California emissions-related data or compliance work is a plus.