Member of Technical Staff - Inference

About the role

We're looking for an Inference Engineer to build inference capabilities on top of Forge, our unified control plane, so customers can consume model tokens without managing GPUs and our NeoCloud partners get a full-stack path to their own token-factory offering. You'll own how models get deployed and served across clusters distributed around the world, on heterogeneous hardware. Deployment comes first: serving models on Forge and our Kubernetes offering, evaluating inference frameworks, and standing up the monitoring, gateways, and endpoints that make a deployment production-ready. From there the work expands into optimization, autoscaling, KV-cache orchestration, and customer inference debugging. This is the primary seat for inference at Hyperbolic — you'll build it end to end, with real influence over where the scope lands.

Responsibilities

• Build inference capabilities on top of Forge (unified control plane)
• Own model deployment and serving across globally distributed clusters on heterogeneous hardware
• Serve models on Forge and Kubernetes offering
• Evaluate inference frameworks
• Set up monitoring, gateways, and endpoints for production inference services
• Work on optimization, autoscaling, KV-cache orchestration
• Debug customer inference and performance issues

Requirements

• Strong general inference background with broad, high-level command of the stack (ability to reason about request to token)
• Deep Kubernetes experience, including hands‑on ability to operate clusters in production
• Solid grasp of inference performance concepts: TTFT, disaggregated inference, speculative decoding, KV cache and its inner workings
• Familiarity with modern inference frameworks and serving engines, and judgment to select among them for a given workload
• Working knowledge of NVIDIA Dynamo and how it fits into a distributed serving architecture
• Experience setting up monitoring, gateways, and endpoints for production inference services
• Proven ability to build a product end‑to‑end (taken something from nothing to serving real traffic)
• Strong self‑initiative and comfort operating as primary owner of an area with minimal direction
• Generalist instincts: willingness to pick up adjacent work when needed

Qualifications

• Experience spanning both inference deployment and inference optimization
• Hands‑on model optimization work — quantization, batching strategies, kernel‑level tuning, or similar
• Understanding of RDMA and high‑performance networking as they apply to distributed serving
• Experience deploying inference across heterogeneous accelerators
• Background supporting customers directly on inference debugging and performance issues
• Experience at a GPU cloud, inference provider, or AI infrastructure company