Staff Data Engineer

About the role

The Staff Data Engineer will lead enterprise-wide data architecture strategy and long-term data platform design. Evaluate and approve major technical designs, including establishing engineering best practices. Serve as technical advisor to leadership and influence product/data roadmap. Champion data governance, democratization, and scalable data standards. Work with cross functional teams including the Solutions Delivery, Business Intelligence; Predictive Analytics, and Enterprise Services teams to assist with and support any data-related technical issues and/or needs.

Responsibilities

• Analyze, manipulate, and process large sets of data using statistical software.
• Develop ETL (Extract, Transform, Load) and data pipeline solutions using Airflow, DBT, Python, and Snowflake to load a data warehouse.
• Design and manage ETL workflows to support large-scale ingestion, transformation, and loading of data.
• Build and scale CI/CD pipelines and enterprise-wide data quality frameworks.
• Test internal data pipelines for reliability and performance.
• Test, validate, and reformulate models to ensure accuracy.
• Complete and improve monitoring and validation processes to maintain pipeline reliability and data accuracy.
• Write new functions or applications in programming languages to conduct analysis.
• Build streaming pipelines with Confluent Kafka for event-driven processing.
• Build dashboards and monitors for pipeline health, freshness, uptime, and SLA compliance.
• Apply data modeling to extract and analyze information from large structured and unstructured datasets.

Requirements

• Requires at least a Master’s degree or foreign equivalent in Data Science, Applied Computer Science, Computer Science or a related field.
• Developing and optimizing scalable pipelines using Apache Airflow and Python.
• Building streaming pipelines using Kafka.
• Tuning big data systems using Kafka and Hive.
• Using Spark to create jobs for batch and streaming analytics.
• Writing and using complex SQL queries to retrieve data from data warehouses for analysis and reporting.
• Using AWS for data processing and large-volume workloads.
• Automating deployment of code and processes using Python.
• Working with cloud-based infrastructure.
• Building Datadog dashboards to monitor pipelines and system performance.
• Using Docker and Kubernetes to improve and validate containerization and automation reliability.
• Implementing Continuous Integration/Continuous Deployment (CI/CD) using Jenkins.
• Utilizing GitHub as a code repository, and Windows and Linux operating systems.

Benefits

• Paid Parental Leave
• 401K Match
• Flexible Vacation Time
• Medical, Dental and Vision Coverage
• Commuter Benefits

Pay

Annual salary between $175,000 - $200,000.

Schedule

Telecommuting available anywhere in the United States.