AI Observability Engineer

About the role

Join a groundbreaking team at AIG, exploring new possibilities for how artificial intelligence can be applied in insurance and beyond. As an AI Observability Engineer, you'll work alongside talented colleagues to innovate and lead projects that transform how we manage risk and serve our customers.

Responsibilities

• Ensure AI solutions align with Responsible AI principles and standards.
• Support the design and execution of the AI Governance program.
• Monitor ML, LLM, and AGENTIC AI KPIs for model effectiveness.
• Manage ongoing operations of the AI model management platform and pipelines.
• Review AI solutions for alignment with AIG's Organizational Policies, AI Governance Framework, and Responsible AI principles.
• Provide support to AI product teams, conduct independent reviews, and participate in workshops.
• Collaborate with legal, compliance, security, and other teams to coordinate the review of Responsible AI issues.
• Create executive-level reports on Responsible AI and provide strategic recommendations to senior leadership.
• Contribute to the development and improvement of Responsible AI assets.
• Monitor and stay updated on emerging global AI regulatory frameworks, standards, and policies.

Requirements

• Minimum 3 years’ experience and a Bachelor’s degree.
• Knowledge of machine learning, large language models, and various models, algorithms, and evaluation metrics.
• Data pipelines and analysis skills with the ability to handle, clean, and transform data for monitoring and analysis.
• Experience building production LLM, RAG, or model inference monitoring pipelines.
• Hands-on experience with AI Observability tools such as Fiddler, Arize, LangSmith, or similar tools.
• Ability to build dashboards and alerts tied to AI model behavior beyond system uptime.

Qualifications

• Experience and passion in Responsible AI or AI ethics.
• Understanding of AI technologies and frameworks such as Generative AI.
• Strong collaboration skills with cross-functional teams.
• Working knowledge of effective policies and processes around the usage of emerging AI tools and products.
• Proven track record of solving complex problems, developing mitigation strategies, and influencing others on key AI initiatives.
• Prior experience with OpenTelemetry distributed tracing or structured logs.
• Ability to create and update Responsible AI training content for technical and non-technical audiences.
• Ability to contribute to the development and improvement of Responsible AI assets.
• Ability to monitor and stay updated on emerging global AI regulations and translate them into practical, actionable updates.

Skills

• Machine Learning
• Large Language Models
• Data Pipelines
• Observability Tools
• Model Management
• Responsible AI
• Global AI Regulations

Benefits

• Comprehensive benefits package focusing on health, wellbeing, financial security, and professional development.
• Flexible work arrangements to support in-person collaboration.
• Continuous learning and development opportunities.
• A culture of inclusion and belonging through diverse Employee Resource Groups (ERGs).

Pay

TBD

Schedule

Primarily in-office