Cloud Inference Engineer

Qualifications

• CUDA + GPU inference optimization with LLMs such as vLLM, SGLang, or TensorRT-LLM
• Experience with KV caching, paged attention, batching, token streaming, etc.
• Distributed compute expertise (GPU support is a plus)
• No formal degree required

Company Overview

Luminal (YC S25) develops an AI compiler and serving stack that accelerates model performance and readiness with minimal effort. The company's mission is to make models 10 times faster and more accessible to developers.

Role

Founding, based in San Francisco downtown. The role involves deploying and tuning models with advanced optimizations, conducting performance reviews, improving scheduling, batching, and autoscaling, and profiling latency, costs, and utilization. Occasionally, there may be opportunities to contribute to kernel development and light humor.

Day-to-Day Responsibilities

• Deploy and tune models using techniques like KV caching, paged attention, sequence packing, etc.
• Conduct model performance evaluations
• Enhance the scheduler, batcher, and autoscaler functionalities
• Profile system performance metrics including latency, cost, and resource utilization
• Occasionally write kernels and engage in light humor