Data Engineer Lead
Horizontal Talent · Chicago, IL · 3 wk ago
HybridInformation Technology$51–$89/hrContract
We’re looking for a hands-on Data Engineer Lead to support an important Phase 0 discovery effort for contact identity, golden record, and marketing data model readiness. This role is ideal for someone who enjoys digging into complex data landscapes, partnering with cross-functional teams, and turning discovery findings into clear engineering handoff materials for Databricks build work.
Responsibilities
- Inventory priority source systems, data entities, identifiers, fields, relationships, and known data quality concerns.
- Review source documentation, sample data structures, field inventories, and SME input to understand the current data landscape.
- Assess data completeness, duplicates, freshness, conflicts, availability, lineage, and source reliability.
- Support source-to-target mapping for a canonical marketing data model.
- Translate identity rules, survivorship logic, source hierarchy, field ownership, account matching, and exception handling into engineering-ready inputs.
- Document data gaps, dependencies, assumptions, risks, and open questions for the internal engineering team.
- Help define acceptance criteria, data quality thresholds, and match-confidence considerations for the final handoff package.
- Partner with strategy and architecture stakeholders to ensure outputs are actionable and ready for Databricks implementation.
Skills
- 5+ years of experience in data engineering, data platforms, or a related field.
- Strong background in data discovery, profiling, mapping, and engineering handoff work.
- Hands-on experience with Python and PySpark.
- Strong Databricks experience.
- Experience with SQL and NoSQL databases.
- Experience working in enterprise data environments.
- Knowledge of ETL/ELT processes, data integration, and data modeling.
- Hands-on experience with AWS or another major cloud platform.
- Strong analytical, problem-solving, and communication skills.
Preferred Skills
- Experience with cloud data tools such as AWS Glue or Azure Data Factory.
- Familiarity with modern data architecture principles, including data lakes or data mesh.
- Exposure to governance, security, and compliance practices in data environments.
- Experience with performance optimization and scalable data solutions.
- Certifications in cloud data technologies are a plus.
- Ability to work collaboratively in a hybrid, fast-moving environment.
Benefits
- Medical, dental, and vision coverage.
- Retirement benefits.
Pay
The pay range for this role is $51 - $89 per hour based on qualifications and experience.