Data Engineer

Role Overview

We are seeking a Data Engineer for a role in the financial services industry. The position involves authoring, debugging, and optimizing complex data scripts using Python and advanced SQL. The candidate will work with Apache Airflow to design, schedule, and monitor data pipelines and will be responsible for improving existing data pipelines by resolving failures and optimizing resource utilization.

Key Responsibilities

• Author, debug, and optimize complex data scripts using Python and advanced SQL queries.
• Utilize Apache Airflow for designing, scheduling, and monitoring complex data pipelines and DAGs.
• Manage SQL-based database systems, including schema design, query performance tuning, and data warehousing.
• Implement ETL/ELT data integration techniques, data transformation, and manage distributed storage systems.
• Install, configure, and maintain data pipeline utilities, custom Airflow providers, and database connectors.
• Create and maintain technical documentation for data architecture, DAG workflows, and operational procedures.
• Improve existing data pipelines by resolving recurring failures, eliminating bottlenecks, and optimizing resource utilization.
• Identify opportunities to innovate, automate, and scale data delivery across the organization.
• Perform Root Cause Analysis and Incident Management for data quality issues, DAG failures, and database outages.

Required Qualifications

• Experience: 4-7 years of data engineering experience is required.
• Technical Skills: Experience with cloud-based data platform infrastructure and containerization technologies such as Docker or Kubernetes.
• Understanding of modern data stack technologies and ability to assist with cloud-based data lake or warehouse integration efforts.
• Proficiency in Agile development methodologies and CI/CD deployment pipelines for code and workflows.
• Ability to thrive in a sense-of-urgency environment and leverage data engineering best practices.

Benefits

• Supplemental benefits including medical, dental, vision, life, disability, and other insurance plans
• ESPP (employee stock purchase program)
• 401K program with company match after 12 months of tenure
• HSA (Health Savings Account on the HDHP plan)
• SupportLinc Employee Assistance Program (EAP) with up to 8 free counseling sessions
• Corporate discount savings program and other discounts
• On-demand training program
• Access to certification prep and a library of technical and leadership courses/books/seminars after 6+ months of tenure
• Certification discounts and perks to associations including CompTIA and IIBA
• Dedicated customer service team for Consultants
• Certified Career Coach