Software Engineer – Inference (SWE1) [D.26.0202]
Dover Networks LLC · Maryland, United States · 3 wk ago
Engineering$163k–$178k/yrFull-time
Responsibilities
- Procure, configure, and test new inference models, preparing them for release to our user base.
- Develop in-house services and techniques to guarantee continual high-quality inference service for our customers.
- Collaborate with model vendor teams and representatives to create reliable pipelines for closed-source model usage.
- Engage with other teams in our organization to establish solid infrastructure for our services and integrate LLM-powered tools for user needs.
Requirements
- Experience with Python and/or other modern programming languages.
- Familiarity with Argo CD and/or other CI/CD frameworks.
- Experience with Kubernetes/Helm.
- Familiarity with AWS or other cloud service providers.
- Ability to learn new technologies quickly.
- Strong communication skills and willingness to ask questions.
Skills
- Experience with vLLM, LiteLLM, or similar inference-serving frameworks (Nice to Have).
- Experience with other LLM hosting frameworks and practices (Nice to Have).
- Experience supporting production software using Site Reliability Engineering (SRE) best practices (Nice to Have).
- Experience with Elastic, Grafana/Prometheus, or other observability frameworks and practices (Nice to Have).
- Experience with Docker and containerization (Nice to Have).
- Experience in traffic shaping and quality-of-service engineering (Nice to Have).
- Knowledge of and interest in hosting AI capabilities (Nice to Have).
Pay
$163k-$178k per year with an additional $41k-$44k in immediately vested company 401(k) contributions.