Lead Data Engineer

Overview

How You’ll Make an Impact

• Build the core framework and connectors powering our audience activation business.
• Operate at True Scale: Process and route trillions of records monthly.
• Work with Modern Data Tech: Innovate with Spark, Scala, Python, AWS EMR, and Databricks.
• See Your Impact: Enable direct business interfaces with the world’s largest tech, social media, and CTV platforms.
• Own the Solution: Take complex problems from ingestion and specification all the way through to production.

Who Does This Role Report To and Who Will They Collaborate With?

This role reports to the Engineering Director and collaborates closely with fellow data engineers, product managers, and partner teams.

Why Would a Top Candidate Evaluate Multiple Opportunities?

• Operate at True Scale: Process and route trillions of records monthly.
• Work with Modern Data Tech: Innovate with Spark, Scala, Python, AWS EMR, and Databricks.
• See Your Impact: Enable direct business interfaces with the world’s largest tech, social media, and CTV platforms.
• Own the Solution: Take complex problems from ingestion and specification all the way through to production.

What You’ll Achieve

Core Contribution

• Write robust, scalable, and maintainable code using Spark (Scala/Python) and SQL to build out our core framework and data pipelines.
• Pioneer AI-Assisted Engineering: Accelerate coding, debugging, and testing with tools like Cursor and Amazon Q Developer.
• Scale Integrations: Build, maintain, and optimize the high-concurrency data connectors that feed external publishers and ad networks.
• Optimize & Solve: Dive deep into complex data processing routines on EMR and Databricks to fix production issues, tune SQL queries, and solve for performance bottlenecks in a distributed environment.

Team Mentorship & Quality

• Participate in rigorous code reviews, help enforce engineering standard processes, and mentor mid-level/junior engineers to elevate the team's overall codebase.
• Build and maintain automated production processing routines that fit seamlessly into our existing scheduled cloud infrastructure.

Who You Are

• S. in Computer Science, Computer Engineering, or a related field.
• 5+ years of professional experience on a development team building and maintaining big data pipelines.
• Deep, hands-on expertise in Apache Spark and distributed computing concepts.
• Strong programming proficiency in Scala and/or Python.
• Fluent SQL skills with the ability to ingest complex use cases, refactor code, and tune queries for massive datasets.
• Proven experience working within cloud environments (AWS preferred) and managed platforms like Databricks or EMR.
• Ability to solve production issues autonomously and own a problem to the end.
• Excellent communication skills to work with internal partners, ask the right questions, and translate business requirements into technical solutions.

Additional Information

• When You Join Us, We’ll Create Something EPIC Together
• Epsilon is a global data, technology and services company that powers the marketing and advertising ecosystem.
• Epsilon is an Equal Opportunity Employer.
• Epsilon’s policy is not to discriminate against any applicant or employee based on actual or perceived race, age, sex or gender (including pregnancy), marital status, national origin, ancestry, citizenship status, mental or physical disability, religion, creed, color, sexual orientation, gender identity or expression (including transgender status), veteran status, genetic information, or any other characteristic protected by applicable federal, state or local law.
• Epsilon also prohibits harassment of applicants and employees based on any of these protected categories.
• Epsilon will provide accommodations to applicants needing accommodations to complete the application process.
• Compensation Range: USD $98,000.00 - USD $182,000.00/Annually.