Lead Data Engineer

About the role

Lead Data and AI Engineer for the Enterprise Data Organization responsible for designing, building, and managing data pipelines (data ingestion, transformation, distribution, quality rules, storage, etc.) using cutting-edge AI solutions (Generative AI, Agentic AI, etc.). This position is based in our Atlanta Support Center, with an expected on-site presence of 80%.

Responsibilities

• Lead and guide multiple technical data engineering team members
• Design, build, maintain, and use platforms and services to manage data
• Design, develop, deploy, and document AI solutions from concept to operationalization
• Demonstrate and guide implementation of Infrastructure as Code, automated testing, and modern observability strategies
• Grow an engineering culture of craftsmanship through mentoring, technical leadership, and presentations across data teams
• Work closely with product owners/managers to understand business needs
• Collaborate with architects, data modelers, and other team members to design Data and AI jobs
• Lead sessions with data engineers and provide guidance to solve technical challenges
• Develop new and/or enhance existing data processes (ingest, transformation, storage, management, quality)
• Contribute to setting up and refining data engineering best practices
• Ensure adherence to coding standards, best practices, and production-deployable code that is robust, scalable, and reusable
• Document technical artifacts for data engineering solutions
• Stay up-to-date with the latest AI trends and integrate emerging technologies to drive improved system performance

Requirements

• 8+ years of experience in a data engineering role
• 5+ years of experience with advanced SQL queries, scripts, stored procedures, materialized views, and views
• 5+ years of hands-on design and development experience in the data space, specifically focused on ELT to load data and perform transformations in databases
• 3+ years of experience with DevOps models utilizing CI/CD tools
• Experience in one of the cloud platforms (Azure, AWS, GCP); Azure preferred
• Demonstrated hands-on experience with Python
• Experience with Snowflake (a plus)

Skills

• Strong ability to use analytical SQL functions
• Ability to deliver AI/ML projects from POC to production using GitHub Copilot, Azure OpenAI, AI Search, Temporal, Pydantic AI, etc.
• Strong expertise in LLMs, building agents: open/closed-source models, advanced prompt engineering techniques, RAG, Agentic AI, document processing, vector/hybrid search, guardrails, observability & monitoring, and LLM evaluation
• Hands-on experience in Azure Cloud Platform (ADLS, Blob)
• Strong skills with ETL tools (Informatica, Talend, Azure Data Factory) and BI tools (Tableau preferred)
• Strong skills with orchestration tools like Apache Airflow
• Expert ability to analyze data models
• Strong skills in performance tuning and scaling of services
• Good interpersonal skills; comfort and competence in dealing with different teams
• Skilled in leveraging Agile practices and methodologies