Senior Software Engineer – Inference (SWE3) [D.26.0205]

Responsibilities

• Lead the evaluation, configuration, and deployment of inference models for our users.
• Translate high-level stakeholder goals into engineering outcomes.
• Mentor engineers and grow development best practices and standards for the team.
• Develop in-house services and techniques to guarantee continual high-quality inference service for our customers.
• Engage with other teams in our organization to establish solid infrastructure for our services and integrate LLM-powered tools for user needs.
• Collaborate with teammates on surge efforts to support short-term, high-priority inference needs from our customers.

Requirements

• Experience with Python and/or other modern programming languages.
• Experience with Argo CD and/or other CI/CD frameworks.
• Experience with Kubernetes/Helm.
• Experience with AWS or other cloud service providers.
• Strong communication skills and ability to mentor other engineers.
• Excellent leadership and stakeholder management skills.
• Able to balance hands-on engineering with leadership and coordination responsibilities.

Skills

• Experience with vLLM, LiteLLM, or similar inference-serving frameworks (Nice to Have).
• Experience with other LLM hosting frameworks and practices (Nice to Have).
• Experience supporting production software using Site Reliability Engineering (SRE) best practices (Nice to Have).
• Experience with Elastic, Grafana/Prometheus, or other observability frameworks and practices (Nice to Have).
• Experience with Docker and containerization (Nice to Have).
• Experience in traffic shaping and quality-of-service engineering (Nice to Have).
• Knowledge of and interest in hosting AI capabilities (Nice to Have).

YOE Requirement

12 years of relevant experience or 4 additional years of experience in lieu of a Bachelor's degree in a technical discipline.