Principal Data Engineering

Primary Responsibilities

• Enterprise Data Platform Architecture
• Partner closely with Platform Solution Architect to evolve the end-to-end data platform architecture across Azure, Snowflake, dbt, and future architecture changes using lake house technologies (including Iceberg where applicable)
• Define reference architectures, reusable frameworks, and engineering standards for ingestion, transformation, and consumption
• Drive decisions around scalability, performance, resiliency, security, and cost optimization
• Partner with Tech Engineering Delivery Owner / Platform Architect / Data Architect / Offshore Tech Lead and Offshore Snowflake Admin defining best practices, data models, integration patterns, and implementation strategies

Advanced Data Engineering & Pipeline Development

• Design, build, and operate high-volume, high-reliability batch and near real-time data pipelines using Azure Data Factory, dbt, Python, SQL, Snowflake, and related tools
• Implement scalable transformation logic to deliver trusted, analytics-ready datasets
• Support event-driven and streaming architectures were needed
• Leverage canonical/common data modeling practices to ensure consistency and reusability across domains

AI / ML Data Enablement

• Enable AI/ML and GenAI use cases by building AI-ready data platforms and pipelines
• Support: Feature engineering and model training pipelines, Historical and point-in-time datasets, Data versioning and ML lifecycle integration
• Leverage enterprise tools such as Snowflake Cortex and MCP Server deliver governed AI capabilities
• Partner closely with Data Science and ML Engineering teams to enable scalable, production-ready AI solutions

Data Quality, Reliability & Production Ownership

• Establish and enforce data quality, observability, reconciliation, and auditability standards
• Own production stability, including monitoring, incident response, root cause analysis, and long-term remediation
• Implement practices for schema evolution, backward compatibility, and error handling
• Define and track KPIs for data freshness, pipeline reliability, platform performance, and cost efficiency

CI/CD, Governance & Security

• Design and manage CI/CD pipelines for data and ML workloads, including automated testing and release orchestration
• Ensure alignment with enterprise governance, security, and regulatory requirements (HIPAA, SDLC, data governance)
• Implement lineage, monitoring, and audit frameworks to support compliance and traceability

Technical Leadership & Mentorship

• Serve as a senior technical authority and escalation point, driving solution design and resolving complex challenges
• Provide design reviews, code reviews, and technical guidance to engineers across levels
• Mentor and develop engineers, fostering a culture of engineering excellence, ownership, and accountability
• Influence cross-team delivery and coordinate across onshore/offshore teams and stakeholders

Leadership Expectations

• Operate independently with platform-level ownership and accountability
• Balance hands-on execution with long-term architecture thinking and working with senior architect for OHBI
• Influence engineering strategy beyond immediate team scope
• Act as a trusted partner across engineering, architecture, and business teams

Stakeholder Collaboration

• Translate business and product requirements into scalable technical solutions and delivery plans
• Partner with data architecture, analytics, BI, and product teams to ensure alignment and value delivery
• Communicate complex technical concepts effectively to both technical and non-technical stakeholders

Required Qualifications

• Extensive experience with cloud data platforms, especially Azure and Snowflake
• Solid hands-on expertise in: SQL and Python, Azure Data Factory - Data pipeline design and orchestration, Snowflake - tasks and steams design and orchestration
• Deep understanding of data integration patterns (batch, micro-batch, streaming, near real-time)
• Experience with data quality, observability, monitoring, and production support
• Proven ability to design for performance tuning, scalability, and cost optimization
• Experience enabling AI/ML data pipelines and solutions
• Demonstrated technical leadership and mentorship experience

Preferred Qualifications

• Bachelor's degree, Computer Science or related field
• Experience with Databricks and/or modern data ecosystem tools
• Familiarity with BI/Analytics platforms (e.g., Power BI)
• Prior experience within Optum / UHG environments
• Experience with Lakehouse patterns and Apache Iceberg
• Solid communication skills with ability to influence senior stakeholders