Lead Data Engineer - Experimentation
Job Summary
Join the User Platforms organization as a Lead Data Engineer focused on Experimentation. In this role, you will build and evolve the data foundations that enable experimentation, causal analysis, and product optimization across streaming products. You will design scalable data pipelines, experimentation data platforms, and analytical datasets that transform raw behavioral data into trusted insights that drive product decisions. As part of the rapidly evolving streaming ecosystem, you'll tackle complex business challenges that directly impact millions of subscribers. Your insights will shape Product roadmaps, pricing strategies, and user experience optimizations that drive measurable business growth.
Technical Responsibilities
- Build Experimentation Data Platforms: Design, develop, and maintain scalable data infrastructure supporting A/B testing, geo experiments, feature launches, and product analytics.
- Develop Data Pipelines: Create reliable batch and streaming pipelines to ingest, transform, validate, and serve large-scale user, product, and engagement datasets.
- Enable Experimentation at Scale: Build reusable frameworks, datasets, and tooling that allow teams to efficiently define experiments, measure outcomes, and analyze results.
- Data Modeling and Architecture: Design dimensional models, feature-ready datasets, and data products optimized for analytics, experimentation, and machine learning use cases.
- Data Quality and Governance: Implement automated validation, monitoring, lineage, and quality controls to ensure experiment results are accurate and trustworthy.
- Production Data Systems: Build robust deployment workflows, CI/CD pipelines, observability, and operational processes for data products.
- Partner with Stakeholders: Collaborate with Product, Engineering, Data Science, and Business teams to translate experimentation needs into scalable technical solutions.
Basic Qualifications
- Bachelor's degree in Computer Science, Engineering, Mathematics, Statistics, Physics, or a related technical field plus 7+ years of experience in data engineering, analytics engineering, or large-scale data platforms.
- Strong experience designing and operating large-scale data pipelines and distributed data systems.
- Deep understanding of data modeling, ETL/ELT patterns, streaming architectures, and data warehouse/lakehouse technologies.
- Experience building data solutions that support experimentation, product analytics, personalization, or machine learning workflows.
- Proficiency with Python, SQL, and modern data engineering frameworks.
- Experience with cloud data platforms and technologies such as Databricks, Snowflake, Spark, Kafka, Airflow, or equivalent tools.
- Experience implementing data quality frameworks, automated testing, monitoring, and governance.
- Able to communicate complex technical concepts clearly to technical and non-technical audiences.
Preferred Qualifications
- MS in computer science or a related quantitative field +5 years of relevant experience OR PhD + 3 years of relevant experience
- Experience supporting experimentation platforms, A/B testing infrastructure, causal inference workflows, or product analytics systems.
- Experience with ML data pipelines, feature engineering platforms, and model lifecycle systems.
- Experience with production deployment practices including CI/CD, infrastructure automation, and observability.
- Experience working in subscription-based businesses, streaming, advertising, or consumer products.
- Proven track record of leadership and stakeholder/project management, including influencing cross-functional teams and delivering high-impact outcomes.
- Adept at adapting quickly to shifting priorities in a fast-moving environment while maintaining quality.
- Drive and maintain a culture of quality, innovation and experimentation.
- Demonstrated experience mentoring colleagues on best practices and technical concepts for building large scale solutions.
About INSPYR Solutions
INSPYR Solutions is a national expert in delivering flexible technology and talent solutions. We strategically align industry and technical expertise with our clients' business objectives and cultural needs. Our solutions are tailored to each client and include a wide variety of professional services, project, and talent solutions. By always striving for excellence and focusing on the human aspect of our business, we work seamlessly with our talent and clients to match the right solutions to the right opportunities.
Our benefits package includes:
- Comprehensive medical benefits
- Competitive pay
- 401(k) retirement plan