Vice President, AI/ML Software Engineer

About the Role

BNY is seeking a Vice President AI/ML Software Engineer to design and implement agentic AI systems, RAG pipelines, and intelligent document processing services. This is a senior individual contributor role with high autonomy—you will own significant components of our AI platform, from embedding pipelines and vector retrieval to multi-agent extraction workflows. You will work closely with the SVP lead to translate architectural vision into production code, while mentoring mid-level engineers and driving technical excellence across the team.

This role is based in New York, NY.

What Sets This Role Apart

• You build the agent framework, not just configure one—custom orchestration engine, not a LangChain wrapper
• Production AI with real consequences—extraction accuracy directly impacts financial operations
• Full RAG ownership—from raw OCR bytes through embedding, retrieval, and generation
• Evaluation-driven culture—golden-truth datasets, automated regression, measurable quality gates
• Greenfield AI + enterprise integration—build new AI-native systems that plug into established platforms

Responsibilities

AI Systems Development

• Implement agentic pipelines: agent loops, tool registries, memory stores, reasoning traces, and self-correction mechanisms
• Build and optimize RAG systems end-to-end:
  • Document ingestion and preprocessing (OCR output, PDFs, structured/unstructured text)
  • Chunking strategies (section-aware, semantic, sliding window, hierarchical)
  • Embedding generation and vector index management
  • Retrieval orchestration: hybrid search, metadata filtering, re-ranking
  • Context assembly and prompt construction for downstream LLM calls
• Develop vectorization pipelines—embedding model integration, batch processing, incremental index updates, and similarity search tuning
• Implement multi-agent coordination patterns: shared blackboards, inter-agent messaging, task decomposition, and consensus mechanisms
• Build prompt engineering infrastructure: template management, few-shot example selection, chain-of-thought scaffolding, and output parsing
• Develop evaluation harnesses: automated accuracy measurement, retrieval quality metrics, regression detection, and A/B comparison tooling

Platform & Backend Engineering

• Build FastAPI services exposing AI capabilities as production APIs (extraction, validation, classification)
• Contribute to Java/Spring Boot platform services where AI integrates with business workflow
• Design and maintain database schemas for AI metadata: audit trails, pipeline runs, memory entries, knowledge graphs
• Implement content policy enforcement and data governance controls within AI pipelines

Mentorship & Collaboration

• Mentor 2-3 mid-level engineers on AI engineering practices
• Participate in architecture reviews and design sessions
• Document patterns, decisions, and runbooks for AI system operation
• Collaborate with product and business stakeholders to translate requirements into technical solutions

Requirements

• Bachelor's degree in Computer Science, Engineering, or related field (advanced degree preferred)
• 6+ years of professional software engineering experience
• 2+ years building production AI/ML systems (not just notebooks/prototypes)
• Strong problem-solving skills with the ability to manage complex data processes
• Excellent collaboration and communication skills to work effectively with cross-functional teams

Skills

• RAG expertise: Embedding models (OpenAI, sentence-transformers, Cohere, or similar), vector databases (FAISS, Pinecone, Weaviate, Chroma, pgvector, or similar), chunking and retrieval optimization, context window management and prompt assembly
• Agentic AI experience: Agent orchestration (custom frameworks, LangGraph, or similar), tool-use patterns, function calling, structured output parsing, memory and state management for multi-turn agent interactions
• Python proficiency (3.11+): FastAPI, async patterns, Pydantic, Poetry, pytest
• LLM integration: Prompt engineering, token management, streaming, error handling, rate limiting
• NLP & document processing: OCR post-processing, text segmentation, entity extraction
• Testing rigor: Unit tests, integration tests, golden-truth validation, retrieval metric evaluation
• API design: RESTful services, OpenAPI specifications, versioning strategies

Preferred Qualifications

• Experience with knowledge graph construction from unstructured text
• Familiarity with code AI concepts: code generation, automated testing, AI-assisted refactoring
• Java/Spring Boot experience for cross-stack contribution
• Angular/TypeScript for full-stack context
• Experience with model evaluation: F1 scores, precision/recall for extraction, MRR/NDCG for retrieval
• Exposure to fine-tuning or prompt optimization techniques
• Understanding of graph RAG or hybrid retrieval architectures
• Capital markets or financial services domain exposure
• Experience with enterprise deployment: Docker, CI/CD, artifact repositories

Technology Stack

• AI/Agentic: Multi-agent pipelines, tool-use, autonomous extraction, reasoning loops
• RAG: Embedding models, vector stores, hybrid search, chunking, re-ranking
• LLM: Azure OpenAI, GPT-4o, structured outputs, function calling
• Python: Python 3.12/3.13, FastAPI, Poetry, Pydantic, Gunicorn/Uvicorn
• Java: Java 21, Spring Boot 3.x (contributory)
• NLP/OCR: Azure Document Intelligence, NLTK, document graph parsing
• Database: Oracle, PostgreSQL, vector databases
• Infrastructure: Docker, GitLab CI/CD
• Testing: Pytest, golden-truth validation, retrieval metrics, evaluation harnesses

Benefits

BNY offers highly competitive compensation, benefits, and wellbeing programs rooted in a strong culture of excellence and our pay-for-performance philosophy. We provide access to flexible global resources and tools for your life’s journey. Focus on your health, foster your personal resilience, and reach your financial goals as a valued member of our team, along with generous paid leaves, including paid volunteer time, that can support you and your family through moments that matter.