Data Engineer III

About the role

Robert Half, one of FORTUNE’s World’s Most Admired Companies and a Fortune 100 Best Companies to Work For, is hiring for a Data Engineer III to join the ATI Data Science Innovation department.

Responsibilities

• Solution Design & Technical Leadership
• Lead architecture and design of complex data pipelines on Databricks lakehouse architecture (Unity Catalog, Delta Lake, Structured Streaming)
• Define technical approach for data engineering initiatives, mentor less-senior engineers, and set standards for code quality through leadership and code reviews
• Design and build data foundations that enable AI/ML capabilities — feature stores, embedding pipelines, vector search indexes, and model training datasets
• Align data engineering solutions with business strategy, including support for Agentic AI workloads
• Data Infrastructure & Platform
• Own health, scalability, and modernization of data infrastructure with Databricks as the strategic platform — including workload migration, compute optimization, and Unity Catalog adoption
• Optimize pipeline performance (Delta Lake table layouts, clustering, Z-ordering) and establish monitoring/alerting best practices with clear SLAs
• Build data infrastructure supporting Agentic AI systems — real-time data access layers, context retrieval pipelines, and agent-accessible data services
• Collaborate cross-functionally with DevOps, Platform Engineering, and MLOps roles to integrate data solutions into the broader technology environment and shared AI infratstructure – Mlflow registries, feature stores, and agent orchestration layers
• Provide consultation to Senior Leadership on complex projects and drive continuous improvement initiatives
• Data Quality, Governance & Collaboration
• Champion data governance at all layers for data, models, and AI assets
• Implement data quality strategies (master data management, validation rules, Delta Live Tables expectations) to ensure trust in enterprise data
• Serve as liaison across data engineering, AI engineering, and business teams; promote data literacy and stewardship

Requirements

• Required:
• Required Bachelor's in Computer Science, Engineering, or related field (Master's preferred)
• 5+ years with Python and SQL in data engineering for big data ML/analytics workloads
• 5+ years designing, building, and troubleshooting scalable ETL/ELT pipelines for business-critical production systems
• 3+ years with cloud data services (AWS), container orchestration (Docker, Kubernetes), and IaC (Terraform, CloudFormation)
• 3+ years architecting ML workflows and data platforms with CI/CD, automated testing, and distributed processing (Spark)
• 3+ years collaborating cross-functionally with Data Science, MLOps, Platform Engineering, and DevOps teams
• 3+ years implementing data quality testing and optimizing SQL/Python for cost/performance in the cloud
• Understanding of the full Data Science SDLC, and experience mentoring engineers
• Preferred:
• 2+ years hands-on with Databricks (Delta Lake, Unity Catalog, Databricks SQL)
• Experience with MLflow experiment tracking and model registry workflows
• Experience designing pipelines that serve AI/ML inference — real-time feature engineering, embedding generation, and context retrieval for LLM-based systems
• Understanding of how data engineering supports Agentic AI: agent-accessible data services, low-latency retrieval, and pipelines enabling autonomous multi-step workflows
• Familiarity with Databricks Mosaic AI, Vector Search, and/or Feature Store
• FinOps awareness — compute cluster optimization, cost attribution by workload

Benefits

The typical annual salary range for this position is $104,000.00 - $153,000.00. The position is eligible for a discretionary annual bonus. Robert Half provides exceptional earning potential and a competitive benefits package, including group health insurance benefits (medical, vision, dental), FSA and HSA healthcare accounts, life and accident insurance, adoption and fertility assistance, paid parental leave of up to 6 weeks, and short/long term disability. Robert Half Inc. is an Equal Opportunity Employer. M/F/Disability/Veteran.