Inference Optimization Engineer (local / edge runtime)

About the role

Make models fast on the hardware people actually own. Optimize inference engines (llama.cpp, vLLM) for constrained local and edge environments — GPU/iGPUs, Vulkan backends — not datacenter H100 environment, mostly PC/edge. Profile and optimize local inference (llama.cpp-vulkan and vLLM) for latency, throughput, and memory on edge hardware.

Responsibilities

• Profile and optimize local inference (llama.cpp-vulkan and vLLM) for latency, throughput, and memory on edge hardware
• Tune KV cache, continuous batching, and scheduling for interactive agent workloads
• Drive quantization strategy (GGUF / AWQ / GPTQ) and validate quality impact with the Post-Training team
• Cut CPU overhead and improve engine startup, model load, and lifecycle (start / stop / health)
• Benchmark across hardware tiers and publish honest performance comparisons
• Upstream fixes and patches to open-source engines where it helps us

Qualifications

• BS/MS in CS, EE, Math or related STEM field
• 5+ years software development background
• Strong in C++ and/or Python; comfortable reading systems-level code
• Understands how LLM inference works (attention, KV cache, decoding)
• Has profiled and optimized real performance problems (CPU or GPU) and can prove the speedup
• Linux, build systems, and low-level debugging expertise

Preferred Qualifications

• Hands-on with llama.cpp, vLLM, ggml, or similar engines
• Experience with GPU / accelerator programming (Vulkan, CUDA, SYCL, Metal)
• Familiarity with quantization formats and their quality trade-offs
• Open-source contributions to inference engines

Requirements

• BS/MS in CS, EE, Math or related STEM field
• 5+ years software development background
• Strong in C++ and/or Python; comfortable reading systems-level code
• Understands how LLM inference works (attention, KV cache, decoding)
• Has profiled and optimized real performance problems (CPU or GPU) and can prove the speedup
• Linux, build systems, and low-level debugging expertise

Benefits

Intel offers a total compensation package that ranks among the best in the industry. It consists of competitive pay, stock bonuses, and benefit programs which include health, retirement, and vacation.

Pay

Annual Salary Range for jobs which could be performed in the US: $170,500.00 - 315,490.00 USD

Schedule

Shift Job Type: Shift 1 (United States of America)

Location

Primary Location: US, California, Santa Clara

Additional Locations

• US, Arizona, Phoenix
• US, California, Folsom
• US, Oregon, Hillsboro

Business Group

The Client Computing Group (CCG) is responsible for driving business strategy and product development for Intel's PC products and platforms, spanning form factors such as notebooks, desktops, 2 in 1s, all in ones. Working with our partners across the industry, we intend to deliver purposeful computing experiences that unlock people's potential - allowing each person use our products to focus, create and connect in ways that matter most to them.