Data Engineer
Horizontal Talent · San Francisco, CA · 2 mo ago
On-siteInformation TechnologyContract
Responsibilities
- Design and build scalable data pipelines and data products on Google Cloud Platform.
- Develop and optimize enterprise data solutions using Dataproc, BigQuery, SQL, and dbt.
- Create reliable data models that support both analytical and operational reporting needs.
- Build efficient ETL/ELT processes to ingest, transform, and publish data from multiple enterprise systems.
- Partner with product managers, solution architects, data architects, analysts, and business stakeholders to turn requirements into technical solutions.
- Lead technical design discussions and participate in code reviews to maintain engineering quality and consistency.
- Improve performance, reliability, scalability, and cost efficiency across cloud data platforms.
- Support monitoring, testing, production issue resolution, and continuous improvement efforts.
- Contribute to reusable frameworks, documentation, and engineering standards.
- Work within Agile delivery teams and help with sprint planning, estimation, and backlog refinement.
- Mentor team members and help strengthen technical practices across the team.
Requirements
- 8+ years of experience in data engineering with technical leadership on enterprise projects.
- Strong hands-on experience with Google Cloud Platform.
- Advanced proficiency in Dataproc, BigQuery, SQL, and dbt.
- Solid understanding of modern ETL/ELT architecture and large-scale data processing.
- Experience with data modeling, including dimensional, normalized, and warehouse design approaches.
- Background building scalable, maintainable cloud-native data pipelines.
- Experience with Git, CI/CD pipelines, and engineering best practices.
- Strong troubleshooting, analytical, and problem-solving abilities.
- Clear written and verbal communication skills with the ability to collaborate across teams.
Skills
- Experience with Apache Airflow for workflow orchestration (preferred).
- Exposure to Apache Kafka or other streaming technologies (preferred).
- Experience with PySpark for distributed data processing (preferred).
- Proficiency in Python for automation, utilities, and data engineering tasks (preferred).
- Familiarity with data quality, metadata management, and data governance practices (preferred).
- Experience in retail, supply chain, transportation and logistics, warehouse management, or distribution center operations (preferred).
- Self-driven mindset with a strong sense of ownership and commitment to high-quality delivery (preferred).
- Ability to balance technical excellence with business priorities in a fast-paced environment (preferred).
Preferred Skills
(As listed above, these are preferred skills rather than required qualifications.)