Generative AI Developer
Marsh McLennan Agency · White Plains, NY · 1 mo ago
RemoteRemoteInformation Technology$83k–$125k/yrFull-time
About the role
We are seeking Gen AI Developer who can build, test, and deploy applications powered by large language models and machine learning tools.
Key responsibilities
- Review system performance: Check logs for API errors, latency issues, and output quality from deployed models
- Prompt engineering: Refine and test system prompt to improve response accuracy and reduce hallucinations
- Integrate APIs: Connect large language models (like GPT-4 or Claude) with internal databases using retrieval-augmented generation (RAG) frameworks
- Build pipelines: Write Python code to process data, embed documents, and manage vector databases (such as Pinecone or Milvus)
- Fine-tune models: Adjust model weights or train smaller open-source models on domain-specific company data
- Collaborate with teams: Meet with product managers and software engineers to align AI features with user needs
- Run evaluations: Test the application for safety, bias, and edge-case failures using automated testing tools
- Optimize costs: Review token usage and API spending to keep cloud infrastructure efficient
- Deploy updates: Push new code updates to production environments using CI/CD pipelines
Requirements
- Designing and delivering AI-enabled applications and agentic systems in production
- Building multi-agent architectures, including planner, executor, reviewer, critic, and supervisor patterns
- Developing tool-calling, function-calling, and memory-enabled agents
- Using LangChain, LangGraph, and similar frameworks for orchestration and workflow design
- Applying DSPy, prompt engineering, and optimization techniques to improve quality and repeatability
- Building production-grade AI services in Python
- Integrating with leading LLM platforms such as Anthropic, OpenAI, and open-source models
- Designing RAG solutions using vector databases
- Integrating AI systems with enterprise platforms through APIs, workflows, events, and data services
- Implementing observability, evaluation, and optimization for AI systems, including tracing, decision logging, and performance monitoring
- Working within governance, security, privacy, and compliance frameworks, ideally in regulated industries
- Leading complex technical delivery while collaborating effectively across engineering, product, business, risk, and compliance teams
- Overall 10+ years of technology experience, including delivery of scalable, resilient digital products
Benefits
- Generous time off, including personal and volunteering
- Tuition reimbursement and professional development opportunities
- Remote work
- Charitable contribution match programs
- Stock purchase opportunities
Pay
The applicable base salary range for this role is $83,200 – 124,800. The base pay offered will be determined on factors such as experience, skills, training, location, certifications, education, and any applicable minimum wage requirements. In addition to the base salary, this position may be eligible for performance-based incentives.