Data Engineer, Senior

Location: New Orleans, LA (Hybrid) or The Woodlands, TX (Hybrid)

About the Role

The Data Engineer Senior is an integral member of the data platform team, assisting the solution architect to drive end-to-end technical design. This role interacts with the solution architect to understand business and functional requirements and leads data engineers within the product team to design and deliver technical solutions, leveraging the data analytics platform where possible.

The goal is to provide data that is reliable, predictable, cost-effective, and secure to internal stakeholders within the data platform. The Data Engineer Senior will also work with other team members to create plans and strategies for adopting new tools, technologies, and alternative solution architectures, including leveraging the cloud.

Responsibilities

• Understand the complexity of data and design systems and models to handle different data sources/formats, including structured, semi-structured, and unstructured, as well as stream processing.
• Address governance and security challenges associated with solutions.
• Analyze data sources, design, and evaluate feasible data pipeline solutions, including database modeling and design, relational database architecture, metadata, and repository creation.
• Design and develop data pipelines or ETL processes.
• Support and adhere to the development of data management policies, standards, and procedures.
• Define integrated views of data drawing from across the enterprise, both in real-time and as extracts.
• Package deployments and collaborate with IT to migrate code between Development, Quality Assurance (QA), and Production environments.
• Support test automation, troubleshoot ETL job functionality, validate data, and create test data and table structures using SQL.
• Work in an agile, rapid delivery environment.

Requirements

• Bachelor’s or Master’s degree in a data-centric field: Computer Science, Mathematics, Information Systems, Information Processing, or Engineering.
• 6+ years of experience as a data engineer utilizing standard data management principles.
• Experience with AWS GovCloud (isolated AWS environment for U.S. federal agencies and export-controlled data) and AWS Commercial cloud.

Skills

• Technical Capabilities:
  • ETL experience
  • Cloud-based computing (AWS or Azure)
  • Big data technologies
  • Data warehousing, mining, analysis, self-service, or machine learning
  • Data governance
  • Structured and unstructured data
  • SQL and NoSQL database technologies
  • Data modeling
  • Python, SQL, Spark, Airflow
• General Capabilities:
  • Requires conceptual and practical expertise in own discipline and basic knowledge of related disciplines.
  • Strong communicator at technical levels with demonstrated writing and presentation skills.
  • Knowledge of best practices and how own area integrates with others; awareness of competition and market differentiators.
  • Acts as a resource for colleagues with less experience; may lead small projects with manageable risks.
  • Solves complex problems; takes new perspectives on existing solutions; exercises judgment based on multiple sources of information.
  • Explains difficult or sensitive information; works to build consensus.
• Proficiency in PC Windows environment (Word, Excel, Access, PowerPoint).

Travel up to 25% may be required.