Senior Artificial Intelligence Data Engineer

About the role

In this role, you will help build and enhance a modern, AI-ready data enablement platform that supports cross-domain analytics, governed data products, and reusable engineering patterns across the enterprise. You will enable data producers and analytics teams through guided pathways, reusable accelerators, metadata-driven frameworks, CI/CD patterns, Databricks Asset Bundles, dbt transformation standards, Unity Catalog governance, and Starburst/Trino analytical access. You will focus on helping teams create trusted, governed, reusable, and AI-ready derivative data products that support analytics, reporting, GenAI, semantic search, and emerging agentic platform use cases. As a hands-on senior individual contributor, you will combine platform engineering, data transformation, governance, and enablement to drive scalable and repeatable data product delivery across the organization.

Responsibilities

• Build and support scalable data engineering solutions using Azure Databricks, PySpark, SQL, Delta Lake, Azure Data Factory, dbt, and Unity Catalog.
• Improve metadata-driven Azure Data Factory and Databricks patterns for orchestration, configuration, monitoring, restartability, and operational support.
• Develop reusable accelerators including CI/CD templates, Databricks Asset Bundle patterns, deployment automation, environment configuration, and data product onboarding templates.
• Design, develop, and support dbt models, macros, tests, documentation, and transformation standards for governed analytical data products.
• Provide guidance on appropriate technology selection and implementation patterns across dbt, Databricks notebooks and workflows, Delta Live Tables, Spark, and Starburst/Trino.
• Support cross-domain analytics initiatives by transforming source-refined data into trusted, reusable, business-aligned derivative data products.
• Leverage Unity Catalog to establish and support governed catalogs, schemas, tables, lineage, access controls, naming standards, and certification practices.
• Support Starburst/Trino as an analytical and federated query layer for governed enterprise data consumption.
• Apply Azure DevOps, Git, CI/CD, and Infrastructure as Code (IaC) practices to create repeatable, testable, and environment-aware platform delivery processes.
• Troubleshoot and resolve production issues related to orchestration, transformations, data quality, access management, query performance, deployments, and operational workflows.
• Collaborate with data engineering, analytics, platform, governance, and business teams to establish reusable, scalable, and supportable data engineering patterns.
• Contribute to the evolution of enterprise data engineering standards, governance practices, observability capabilities, and AI-ready data product frameworks.

Qualifications

• Relevant degree preferred.
• 5 or more years of hands-on data engineering experience building production-grade data platforms, pipelines, or analytical data products required.
• Strong experience with Azure Databricks, PySpark, Spark SQL, Delta Lake, Azure Data Factory, SQL, and dbt required.
• Experience with Azure DevOps, Git, pull request workflows, CI/CD pipelines, and release management practices required.
• Working knowledge of lakehouse architecture, metadata management, data governance, lineage, access control, and operational support required.
• Demonstrated ability to function as a senior individual contributor with strong ownership, technical judgment, and cross-functional collaboration skills required.
• Experience supporting enterprise-scale analytical platforms and governed data product delivery preferred.
• Experience with Unity Catalog, Starburst/Trino, Pulumi or other Infrastructure as Code tools, Databricks Asset Bundles, Apache Iceberg concepts, and AKS/Kubernetes-based platform operations preferred.
• Experience building reusable frameworks, accelerators, templates, or platform capabilities for engineering teams preferred.
• Experience preparing governed structured data for AI/ML, GenAI, Retrieval-Augmented Generation (RAG), semantic search, copilots, or agentic workflows preferred.
• Experience within healthcare, analytics, supply chain, finance, or other regulated enterprise environments preferred.
• Strong problem-solving, communication, and collaboration skills with the ability to influence technical direction and establish best practices preferred.
• You must be authorized to work in the United States without sponsorship.

Pay

The current salary range for this role is $102,400.00 to $179,000.00. This position is also incentive eligible.

Benefits

Vizient has a comprehensive benefits plan. View our benefits here.