Lead Data Engineer

About the role

This job leads in the design and management of sophisticated data engineering solutions to enhance decision-making and operational efficiency. It oversees the development of robust data pipelines and storage systems, ensuring high data quality and accessibility. The role is pivotal in strategizing and implementing data architecture across diverse platforms, driving the organization's data-centric objectives.

Responsibilities

• Leading data structure and pipeline development to standardize, transform, and deliver data for insights and reporting, following best practices
• Ensuring data quality throughout the lifecycle, from ingestion to processing and loading, through rigorous data quality frameworks
• Designing and implementing standardized methods for data consumption, including APIs, database views, and data extracts, to facilitate end-user access
• Designing and deploying data solutions on Kubernetes, Teradata, AWS, and Databricks to meet specific business requirements
• Choosing data storage solutions tailored to sensitivity and access needs using platforms like minIO, Teradata, AWS S3, and Redshift
• Mapping data lineage and applying transformation rules to streamline troubleshooting and impact analysis for data modifications
• Collaborating with cross-functional teams to refine data sourcing and processing, enhancing data quality, and optimizing operational processes
• Setting design standards and QA protocols for software/systems development to ensure data flows and storage solutions interoperate effectively
• Consistent exercise of independent judgment and discretion in matters of significance
• Regular, consistent and punctual attendance. Must be able to work nights and weekends, variable schedule(s) as necessary

Skills

• Data Engineering
• Data Storage
• Structured Query Language (SQL)
• Collaboration
• Data Architecture Development
• Databricks Platform
• Java (Programming Language)

Education

Bachelor's Degree

Relevant Work Experience

10 Years +

Pay

Virginia Pay Range: $158,057.06 - $237,085.58