Staff Engineer - Machine Learning

Key Responsibilities

• Lead Implementation: Act as the lead engineer for agent runtime orchestration services, focusing on reasoning, planning, and tool invocation
• API & SDK Design: Own the development of robust APIs and SDKs that enable internal and external teams to build and deploy agent workflows
• State & Memory Management: Build and optimize stateful dialog management and memory services for multi-turn, context-aware agents
• Agent Communication: Implement multi-agent communication (A2A) protocols and shared context systems
• System Architecture & Execution: Microservices Leadership - Execute the transition to cloud-native, event-driven backend services using microservices or service mesh architectures
• Workflow Design: Build and manage task scheduling and long-running operations using tools like Temporal or Airflow
• Performance Optimization: Optimize LLM orchestration for performance and cost through caching, batching, and token monitoring
• Integrations & Data Services: Data Pipeline Ownership - Implement and maintain RAG pipelines and integrations with vector databases (e.g., Pinecone, Weaviate, FAISS)
• Telemetry Integration: Ensure all backend services support LangSmith-based evaluation and comprehensive observability

Qualifications

• Required Qualifications:
• 6+ years of backend engineering experience
• 1-2+ years of hands-on experience in AI/ML platform development
• Expert Proficiency: Strong programming skills in Java and Python
• Framework Expertise: Hands-on experience with LangChain, LangGraph, or LangSmith
• Distributed Systems: Deep understanding of event-driven architectures, microservices, and Kubernetes
• Cloud & Data: Solid experience with AWS, Terraform, and a mix of relational (PostgreSQL) and Vector databases
• Orchestration: Practical experience with workflow engines like Temporal or Airflow and messaging systems like Kafka
• Preferred Qualifications:
• Experience with AI-driven planning systems or dialog state tracking
• Background in enterprise security (RBAC, SSO, OAuth2)
• Experience working in a high-growth SaaS or startup environment
• Active contributions to open-source backend or AI frameworks

Additional Information

• Please note this is a hybrid role with onsite expectations of 3x/week (Tues - Thurs) from our San Mateo, CA headquarters.
• The annual base salary range for this position is $211,000 - $261,000. This role is also eligible for a target bonus.
• Compensation is based on a variety of factors, including but not limited to location, experience, job-related skills, and level.
• Freshworks offers multiple options for dental, medical, vision, disability, and life insurance. Equity + ESPP, flexible PTO, flexible spending, commuter benefits, and wellness benefits are also offered. Freshworks also offers adoption and parental leave benefits.