Agentic AI Data Lead Software Engineer

About the role

Join our team as a core technical contributor responsible for architecting, building, and scaling the Data Products Framework—a next-generation platform that enables users to discover, design, build, and productionize governed data products at enterprise scale. You will lead a team of engineers, driving the technical strategy and execution of a platform that orchestrates the end-to-end data product lifecycle leveraging AI/Agentic AI, policy-based governance, and cloud-native architectures on AWS.

As part of the Consumer & Community Banking Marketing Process Automation Team, you are integral to an agile team enhancing, building, and delivering trusted market-leading technology products in a secure, stable, and scalable way. You will conduct critical technology solutions across multiple technical areas within various business functions in support of the firm's business objectives.

Responsibilities

• Lead, mentor, and grow a high-performing team of 5 – 7 engineers across multiple workstreams, fostering a culture of innovation, ownership, and technical excellence.
• Set the technical vision and engineering roadmap for the Data Products platform, aligning with firmwide priorities.
• Drive cross-functional collaboration with platform teams, domain Data Product Owners, AI/ML teams, and governance teams.
• Architect and own the end-to-end technical design of the Data Products Studio—a scalable, enterprise-grade platform that orchestrates the discovery, design, build, and productionization of data products from the CCB Data Lake and Snowflake.
• Design the platform's AI/Agentic AI layer, leveraging intent agents, NLP Text-to-SQL, Knowledge Graphs (KAG), RAG, Vector Databases, and Agent-to-Agent (A2A) communication to enable intelligent, automated data product creation and natural language interaction with the data estate.
• Define the platform's integration architecture with various firmwide systems.
• Establish and enforce architectural standards, design patterns, and engineering best practices across the team—ensuring scalability, security, resilience, and maintainability.
• Lead the design and development of Agentic AI capabilities that power the Data Products Framework—including autonomous discovery agents that profile and recommend data product candidates, design agents that auto-generate data contracts and schema recommendations, build agents that generate and optimize data pipelines, governance agents that auto-apply entitlements based on data classification, and quality agents that detect anomalies, drift, and trigger self-healing remediation.
• Architect the Agent-to-Agent communication layer enabling multi-agent orchestration across the data product lifecycle—from discovery through productionization.
• Leverage RAG (Retrieval Augmented Generation) and Vector Databases to enable contextual, knowledge-grounded AI interactions with metadata, lineage, and data catalog information.
• Implement NLP Text-to-SQL capabilities allowing business users to explore the CCB Data Lake and Snowflake using natural language, lowering the barrier to data product discovery.

Requirements

• Formal training or certification on software engineering concepts and 5+ years of applied experience.
• 10+ years of progressive experience in software engineering, data engineering, or platform engineering.
• Strong leadership experience in guiding and mentoring varying levels of Software Engineers.
• Proven track record of architecting and delivering large-scale, enterprise-grade data platforms or frameworks from concept through production in a large corporate environment.
• Deep hands-on expertise in Python, SQL, and at least one additional language (preferably Java 17+, Spring, Boot), with strong system design and distributed systems knowledge.
• Extensive experience designing, building, and optimizing ETL/ELT pipelines at scale, including batch and real-time data processing.
• Strong proficiency in PySpark for distributed data processing, including DataFrame and Dataset APIs and Spark SQL.
• Extensive experience with AWS cloud services including S3, Athena, Glue, Lambda, Step Functions, IAM, KMS, and Terraform.
• Basic knowledge of Snowflake (architecture, performance optimization, Tasks, Streams, Stored Procedures, Materialized Views, security model) is preferred, but not mandatory.
• Deep understanding of data governance principles including metadata management, data lineage, access control (RBAC/ABAC), data classification, and policy enforcement.

Preferred Qualifications

• Experience with Grafana or equivalent observability platforms for custom dashboards, APM, SLA monitoring, and alerting.
• Experience working with UI frameworks (React, Angular).

Benefits

We offer a competitive total rewards package including base salary determined based on the role, experience, skill set, and location. Those in eligible roles may receive commission-based pay and/or discretionary incentive compensation, paid in the form of cash and/or forfeitable equity, awarded in recognition of individual achievements and contributions. We also offer a range of benefits and programs to meet employee needs, based on eligibility. These benefits include:

• Comprehensive health care coverage.
• On-site health and wellness centers.
• A retirement savings plan.
• Backup childcare.
• Tuition reimbursement.
• Mental health support.
• Financial coaching.