Model Serving Engineer

About the role

Bright Vision Technologies is a technology consulting and software development company delivering cloud, AI, data, and enterprise solutions across the United States. This is a fantastic opportunity to join an established and well-respected organization offering tremendous career growth potential.

Responsibilities

• Design, build, and operate high-performance, highly reliable inference platforms for serving large machine learning models in production.
• Focus on request routing, batching, caching, autoscaling, GPU utilization, and end-to-end observability across diverse model workloads.
• Apply strong distributed systems and performance engineering expertise to ship serving systems at scale.
• Balance latency, throughput, cost, and quality trade‑offs in ML serving.

Requirements

• Bachelor’s or Master’s degree in Computer Science or a related field
• Six or more years of experience in distributed systems, infrastructure, or ML platform engineering
• Strong proficiency in Python and a systems language such as Go, Rust, or C++
• Deep experience operating high‑throughput, low‑latency services in production
• Hands‑on experience with LLM or large model inference frameworks such as vLLM or TensorRT‑LLM
• Strong understanding of GPU architecture, memory hierarchies, and accelerator utilization
• Familiarity with Kubernetes, autoscaling, and modern cloud platforms
• Experience with observability stacks including metrics, tracing, and structured logging
• Solid grounding in performance engineering and capacity planning
• Strong communication and incident response skills

Preferred Qualifications

• Open‑source contributions to model serving infrastructure
• Experience with multi‑region or globally distributed AI serving
• Familiarity with model quantization, distillation, and compression techniques
• Exposure to FinOps for AI workloads and cost‑efficient serving design
• Experience supporting external‑facing AI APIs at scale

Pay

Salary range: $74,000–$98,000 annually.

Location: 100% Remote (U.S.). Position type: Full‑time, Direct W2. Sponsorship: U.S. Citizens, Green Card Holders, EAD Holders, and H‑1B transfer candidates are encouraged to apply; we are unable to sponsor new H‑1B visa petitions.