Senior Engineer- Agentic AI
Bank of America · Pennington, NJ · 1 mo ago
EngineeringFull-time
About the role
This job is responsible for defining and leading the engineering approach for complex features to deliver significant business outcomes. Key responsibilities include delivering complex features and technology, enabling development efficiencies, providing technical thought leadership based on conducting multiple software implementations, and applying both depth and breadth in a number of technical competencies. Additionally, this role is accountable for end-to-end solution design and delivery.
Responsibilities
- Strategy & Roadmap: Define the enterprise Agentic‑AI‑in‑SDLC strategy, operating model, and multiyear roadmap, aligning with business objectives, enterprise architecture, and developer‑productivity goals. Prioritize epics and features in the backlog and drive cross‑portfolio execution to accelerate value realization and adoption.
- Architecture & Design: Architect agentic workflows using the Microsoft Agentic Framework and integrate them with SDLC systems including issue trackers, repositories, CI/CD pipelines, quality and security gates, and observability platforms. Design context architecture—grounding data, state management, retrieval patterns, prompt templates, and caching—to ensure reliable, high‑quality outcomes. Design RAG solutions using enterprise content, vector search, and knowledge graphs, and define reference architectures and guardrails for Copilot‑based coding, testing, and automation.
- Engineering & Delivery: Lead delivery of AI agents that automate SDLC tasks such as requirements analysis, design reviews, test generation, traceability, code‑quality enforcement, documentation updates, and release readiness. Build Copilots with Copilot Studio, integrating plugins, enterprise search, and workflow actions, and operationalize models in Azure AI Foundry with full lifecycle support for evaluation, monitoring, safety, and CI/CD/CT pipelines for prompts and agents.
- Data, RAG, and Knowledge Graphs: Define ontology and schema for domain knowledge graphs and integrate code metadata, service catalogs, architectural decisions, and control libraries to create robust retrieval ecosystems. Implement strong data governance for retrieval sources, ensuring correct handling of sensitive data, adherence to residency and retention rules, and delivery of high‑quality grounding pipelines.
- LLMOps / MLOps: Establish LLMOps practices including prompt/version management, evaluation suites for quality, safety, bias, and hallucination, and offline/online experiments such as A/B tests. Maintain golden datasets and benchmarks aligned to SDLC goals like code‑quality improvement, vulnerability reduction, and MTTR reduction.
- Security, Compliance & Risk: Partner with Risk, Legal, Privacy, InfoSec, and Model Risk to define responsible‑AI patterns including policy‑as‑code controls, auditable logs, data‑boundary enforcement, content safety, and human‑in‑the‑loop review. Navigate governance and control processes to ensure solutions meet regulatory and model‑risk expectations.
- Change Management & Adoption: Lead developer onboarding, enablement, and communities of practice by sharing prompt patterns, reusable tools, and exemplars. Track and communicate value through KPIs and OKRs and provide progress updates and insights to executive stakeholders.
Required Qualifications
- 10+ years in software engineering/architecture with 2+ years leading AI/LLM or intelligent automation solutions in production, ideally at enterprise scale.
- Proven experience implementing Agentic AI solutions (tool‑use orchestration, planning, memory/state) and integrating them with SDLC platforms.
- Hands‑on with GitHub Copilot (org‑level policies, telemetry, governance), Microsoft Copilot Studio (plugins/connectors), and Azure AI Foundry (Prompt Flow, eval/monitoring, safety and compliance).
- Deep knowledge of RAG, prompt & context engineering, and vector search; experience building knowledge graphs and integrating them into retrieval workflows.
- Strong grasp of LLMOps/MLOps: dataset curation, eval design, regression testing for prompts/agents, observability, safety/guardrails, rollout strategies.
- Expertise with SDLC toolchains: GitHub, CI/CD, IaC, testing frameworks, SAST/DAST, artifact repositories, service catalogs, and runbooks.
- Ability to lead cross‑functional programs, manage risk, and communicate with executive and engineering stakeholders.
- Familiarity with responsible AI principles, data privacy, and secure software development practices.
Desired Qualifications
- Experience in regulated industries and with Model Risk Management or similar governance.
- Knowledge of enterprise search, graph databases, and metadata management.
- Background with cloud platforms (Azure preferred), container orchestration (Kubernetes), and policy‑as‑code.
- Exposure to evaluation techniques (task success, safety, toxicity, grounding fidelity, hallucination rates) and A/B testing for agents.