ML Platform Engineer

About the role

ML Platform Engineer - Remote Bright Vision Technologies is a technology consulting and software development company delivering cloud, AI, data, and enterprise solutions across the United States. This is a fantastic opportunity to join an established and well-respected organization offering tremendous career growth potential. Location: 100% Remote (U.S.). Position Type: Full-time, Direct W2. Salary Range: $100,000–$160,000 Annually. Experience Required: 10+ years. Sponsorship: U.S. Citizens, Green Card Holders, EAD Holders, and H-1B transfer candidates are encouraged to apply. We are unable to sponsor new H-1B visa petitions for this position.

Responsibilities

• Design and operate model serving platforms supporting diverse workloads including LLMs, vision models, and recommendation systems
• Optimize inference performance using continuous batching, paged attention, speculative decoding, and request multiplexing
• Implement multi-tenant routing, rate limiting, and quality-of-service policies across model endpoints
• Build autoscaling and capacity management systems that balance latency, throughput, and cost
• Tune GPU utilization, memory management, and KV cache strategies for LLM serving workloads
• Integrate model serving with API gateways, identity systems, and observability platforms
• Implement caching, prompt deduplication, and response reuse strategies where appropriate
• Drive end-to-end observability including latency histograms, queue dynamics, GPU utilization, and error tracking
• Develop deployment workflows including canary releases, shadow testing, and automated rollback
• Operate incident response for high-availability AI services and drive durable reliability improvements
• Collaborate with ML and product teams to support new model releases and capability rollouts
• Implement security controls including request signing, content filtering, and abuse detection at the serving layer
• Document operational procedures, performance characteristics, and tuning guidance for internal teams
• Stay current with AI serving research and translate advances into production capabilities

Requirements

• Bachelor’s or Master’s degree in Computer Science or a related field
• 10 or more years of experience in distributed systems, infrastructure, or ML platform engineering
• Strong proficiency in Python and a systems language such as Go, Rust, or C++
• Deep experience operating high-throughput, low-latency services in production
• Hands-on experience with LLM or large model inference frameworks such as vcLLM or TensorRT-LLM
• Strong understanding of GPU architecture, memory hierarchies, and accelerator utilization
• Familiarity with Kubernetes, autoscaling, and modern cloud platforms
• Experience with observ