AI Auditor, Senior

About the role

The Internal Audit Services team provides independent, objective assurance and advisory services to help Blue Shield of California manage risk, strengthen internal controls, and improve business processes. The AI Internal Audit Consultant will support audit, compliance, fraud detection, and investigative activities focused on the use and misuse of artificial intelligence systems.

Responsibilities

• Support AI-focused audits, advisory reviews, and investigative activities designed to identify inappropriate, suspicious, fraudulent, unauthorized, unethical, or non-compliant AI activity
• Perform testing and analysis of AI system activity, user behavior, prompts, outputs, logs, access patterns, workflow data, and system usage trends to identify indicators of misuse or fraud
• Assist with investigations involving AI-enabled fraud, insider threat, policy violations, unauthorized data use, inappropriate AI activity, suspicious automation, or circumvention of controls
• Review system documentation, AI platform configuration, model documentation, data flows, policies, procedures, control evidence, monitoring reports, and technical artifacts to identify risks and control gaps
• Analyze AI platform data and business process data for anomalous behavior, unusual usage patterns, excessive access, suspicious outputs, unauthorized automation, or high-risk user activity
• Document audit procedures, investigative steps, evidence reviewed, testing results, exceptions, conclusions, and workpapers in accordance with internal audit methodology
• Support interviews, walkthroughs, and fact-gathering activities with business stakeholders, technology teams, data scientists, Compliance, Legal, HR, Privacy, Information Security, and control owners
• Contribute to AI fraud detection playbooks, audit templates, investigative procedures, control libraries, testing scripts, dashboards, and methodology improvements
• Track remediation activities and support follow-up testing to validate whether corrective actions address identified AI misuse, fraud, compliance, or control issues

Qualifications

• Requires a Bachelor’s degree in Accounting, Information Systems, Computer Science, Data Science, Business Administration, Cybersecurity, Criminal Justice, Engineering, or related field, or equivalent experience
• Minimum of 5 years of prior related experience in internal audit, IT audit, technology risk, compliance, fraud detection, investigations, information security, data governance, data analytics, model risk management, or related assurance functions
• Experience with artificial intelligence, machine learning, advanced analytics, automation, AI platforms, AI governance, model risk, or emerging technology controls
• CISA certification required, or equivalent certification such as CFE, CIA, CISSP, CRISC, CISM, CPA, CDPSE, or other relevant audit, risk, privacy, security, fraud, investigations, or technology certification
• Experience with fraud detection, compliance monitoring, investigations, insider threat, cybersecurity, suspicious activity monitoring, intelligence analysis, or forensic-style review preferred
• Working knowledge of internal audit methodology, investigative documentation, risk assessment, control testing, audit evidence, workpaper standards, issue development, and remediation follow-up
• Familiarity with AI misuse scenarios, including unauthorized data disclosure, AI-assisted fraud, synthetic identities, prompt manipulation, automated abuse, inappropriate data use, and circumvention of controls
• Experience performing analytical reviews using SQL, Python, Tableau, Databricks, Excel, Power BI, log analytics, or similar tools preferred
• Knowledge of fraud indicators, investigative techniques, compliance monitoring, insider threat risks, suspicious activity review, and evidence handling preferred
• Ability to communicate technical concepts, suspicious activity patterns, and control issues clearly to audit, business, technology, compliance, and investigations stakeholders
• Strong analytical, problem-solving, organizational, documentation, and professional skepticism skills
• Ability to manage multiple assignments, meet deadlines, handle sensitive information, and work both independently and collaboratively in a team environment
• Experience in health care, health insurance, financial services, technology, or another regulated industry preferred
• ISACA's Advanced in AI Audit (AAIA) certification highly desired