Risk, Operational Risk (Artificial Intelligence Coverage), Vice President, Dallas or Salt Lake City
Goldman Sachs · Dallas, TX · 2 wk ago
EngineeringFull-time
Responsibilities
- Identify, monitor, and analyze operational risks arising from the design, development, and deployment of AI systems, with a focus on risks such as inadequate system alignment, lack of explainability, data quality and drift, prompt injection, hallucination and inaccurate outputs, non-deterministic behavior, bias and discrimination, model overreach/expanded use, reputational risk from AI failures, agent action authorization bypass, tool chain manipulation and injection, agent state persistence poisoning, and multi-agent trust boundary violations.
- Develop evidence-based challenges focused on improving architectural risk posture.
- Monitor the firm's AI architecture control inventory for sufficiency and completeness, challenging the absence of controls and the implementation of controls within engineering standards.
- This includes oversight of mitigations such as AI Firewall Implementation and Management, User/App/Model Firewalling/Filtering, AI System Observability, System Acceptance Testing, Data Quality and Classification/Sensitivity, Human Feedback Loop for AI Systems, LLM-as-a-Judge automated evaluation, Providing Citations and Source Traceability, AI Model Version Pinning, Agent Authority Least Privilege Framework, Tool Chain Validation and Sanitization, Agent Decision Audit and Explainability, Multi-Agent Isolation and Segmentation, Data Filtering From External Knowledge Bases, Preserving Source Data Access Controls in AI Systems, Role-Based Access Control for AI Data, Encryption of AI Data at Rest, and Quality of Service and DDoS Prevention for AI Systems.
- Champion secure-by-design principles across the AI technology stack, ensuring that security, privacy, and risk controls are embedded into AI system architecture from inception rather than retrofitted.
- Conduct data analysis to identify trends and patterns in AI system performance, model behavior, observability telemetry, and security events, augmenting such analysis with qualitative observations to monitor risk-taking trends through bespoke metrics at firmwide and divisional/sub-divisional levels.
- Escalate concerns to senior management when warranted.
- Contribute to divisional and functional risk profile assessments by highlighting AI architecture risk issues and trends to senior divisional managers and the senior Operational Risk management team.
- Conduct evidence-based scenario analysis by working with stakeholders to develop plausible tail risk scenarios around AI architecture failures, including prompt injection attacks leading to data exfiltration, hallucination-driven erroneous financial advice, cascading failures in multi-agent systems, agent authorization bypass leading to unauthorized transactions, data drift causing model degradation, and infrastructure resilience failures.
- Ensure alignment of technical architecture with the firm's AI risk appetite, reviewing architectural decisions for consistency with risk tolerance levels, regulatory requirements, and internal policies.
- Oversee infrastructure resilience for AI systems, including monitoring for availability risks, Denial of Wallet attacks, VRAM exhaustion, and GPU infrastructure dependencies.
- Ensure Quality of Service and DDoS prevention controls are implemented and effective.
- Facilitate operational risk event and data collection related to AI architecture incidents; perform detailed reviews of trends to identify significant risks and ensure monitoring and remediation.
- Review New Activities and ensure operational risks arising from new AI model deployments, new architectural patterns, agentic system rollouts, and infrastructure migrations are properly considered.
- Contribute to review and challenge of AI architecture control assessments to ensure the risk and control self-assessment outcomes are consistent, credible, and underpinned by appropriate evidence.
- Remain current on business drivers, regulatory and industry changes impacting the firm's AI architecture activities and obligations, including the EU AI Act, NIST AI 600-1, NIST Cybersecurity Framework, FFIEC IT Booklets, and ISO 27001.
- Identify and drive initiatives that improve AI architecture risk management activities at the firm.
Qualifications
- Strong understanding of AI/ML architecture concepts, including foundation models, LLMs, RAG systems, agentic AI frameworks, MCP servers, vector databases, embedding pipelines, and model deployment infrastructure.
- Experience with secure-by-design principles, AI firewalling, prompt injection defenses, model observability, and explainability frameworks.
- Knowledge of internal control frameworks such as NIST 800-53, NIST AI 600-1, ISO 27001, COBIT, Cloud Security Alliance Cloud Controls Matrix, and the EU AI Act.
- Strong business acumen with general awareness of technology-related processes, risks, and business flows in financial services.
- 7+ years of relevant experience, which could include working in operational risk, a financial institution's technology division, a technology company that builds or maintains enterprise AI/ML systems, cloud services, offensive or defensive cybersecurity, or IT/Information Security auditors.
- Strong verbal and written communication skills with the ability to present with impact and influence.
- Ability to work in a fast-paced environment with a strong delivery focus.
- Strong organizational skills; project management experience a plus.
- Proficiency in Word, Excel, PowerPoint, SharePoint/OneDrive; SQL, graph databases, and Tableau would be a plus.
- Relevant certifications like CISA, CISM, or related AI/ML and cybersecurity certifications.
- Familiarity with enterprise risk management best practices and controls.
- Bachelor's Degree in Computer Science, Cybersecurity, Business and Technology Management, Finance, Data Science, or related disciplines.