Engineer 2, Data Engineering

Responsibilities

• Developing data structures and pipelines to collect, standardize, and transform data for insight generation and reporting
• Ensuring data quality throughout ingestion, processing, and loading to target tables, with a focus on standardization and transformation rules
• Creating ingestion frameworks for various data types and establishing methods for data consumption through views, extracts, and APIs
• Maintaining data warehouses and lakes, including API development for data access, while managing solutions on Kubernetes, Teradata, and Databricks platforms
• Determining suitable storage solutions on-prem (minIO, Teradata) and cloud (AWS S3, Redshift) based on data sensitivity and access requirements
• Understanding data lineage and transformation to enable efficient troubleshooting and impact analysis during system changes
• Collaborating with technology partners to optimize data sourcing, processing rules, and ensure process optimization and data quality
• Handling data migrations and conversions in response to evolving platforms and new standards, while recognizing and resolving technical issues
• Designing and monitoring ETL processes to support efficient data flow and collaborating with analytics teams for model deployments
• Participating in code reviews and contributing to data governance and compliance efforts in adherence to industry regulations

Qualifications

• Consistent exercise of independent judgment and discretion in matters of significance
• Regular, consistent and punctual attendance
• Must be able to work nights and weekends, variable schedule(s) as necessary

Skills

• Data Quality
• Data Management
• Collaboration
• Analytics