Senior Data Engineer

About the role

The Senior Data Engineer will support the Product Performance team within the Data Organization at a major entertainment and streaming company. This team owns the data that measures the streaming product's performance, browse, playback attribution, and search, which leadership relies on to make business decisions about the platform.

Responsibilities

• Design, implement, and deploy data solutions for batch ingestion and stream processing using Scala and Python
• Establish and enforce standards for code development, testing, and deployment to ensure data quality and governance
• Extend existing data models and build new data pipelines to introduce new metrics and dimensions
• Maintain and update existing source code powering core datasets
• Collaborate with Product Managers, Data Analysts, and cross-functional engineering teams to define requirements and deliver new data features
• Work within a modern data stack including Airflow, Spark, Databricks, Delta Lake, Snowflake, GitHub Actions, and Jenkins
• Investigate and resolve complex data pipeline failures and quality issues, performing root cause analysis and documentation
• Identify and implement technical optimizations to improve performance, scalability, and cost-efficiency

Requirements

• 5+ years of software engineering experience developing large data pipelines
• Strong Scala and Python software engineering skills
• Hands-on production experience with distributed systems such as Spark and Kafka
• Hands-on production experience with orchestration systems such as Airflow
• Strong SQL skills, able to create queries to analyze complex datasets
• Strong CI/CD and SDLC fundamentals
• Strong algorithmic problem-solving expertise
• Basic understanding of AWS or another cloud provider (S3)
• Scripting language experience (Bash, PowerShell)
• Local to New York, NY and able to commute onsite
• Master's degree in Computer Science or Information Systems
• Experience with an MPP/cloud database (Snowflake, Redshift, BigQuery)
• Experience with Jenkins and GitHub Actions
• Experience developing APIs with GraphQL
• Deep understanding of AWS or other cloud providers, plus infrastructure as code
• Familiarity with data modeling and data warehousing methodologies
• Familiarity with Scrum and Agile methodologies
• Self-starting problem solver with strong analytical and communication skills
• Comfortable owning production issues end-to-end, from root cause analysis through resolution
• Able to balance new pipeline development with maintaining existing critical systems
• Willing and able to learn new tools quickly in a fast-evolving stack

Qualifications

• Provides technical guidance and mentorship on best practices
• Detail-oriented with a strong focus on data quality and governance
• Reliable with onsite attendance

Benefits

• Medical, Dental, & Vision Insurance Plans
• Employee-Owned Profit Sharing (ESOP)
• 401K offered

Pay

The approximate pay range for this position is between $75.00 and $85.00 per hour. Please note that the pay range provided is a good faith estimate. Final compensation may vary based on factors including but not limited to background, knowledge, skills, and location. We comply with local wage minimums.