Principal Software Engineer, Inference

We need to produce HTML fragment with sections: About the role? The posting includes "Who We Are" (company description) and "Job Description". Also "Responsibilities", "Knowledge and Skills Required", "Preferred", "Experience and Education", "Accessibility", "What We Can Offer You" (benefits). Also salary info. Also schedule: hybrid requirement (2 days per week from HPE office). Also primary work location etc. We must not repeat job title as heading. Use

for section headings. Use

• for any list of duties/requirements/benefits. Use

  for prose. Remove boilerplate: cookie/navigation/legal/EEO/equal-opportunity/scam-warning, duplicated lines, application instructions. Also remove Accessibility? It's not boilerplate but maybe keep? It's about accommodation request; might be considered EEO? It's about accessibility accommodation; could be considered part of EEO but it's specific. The instruction: remove cookie/navigation/legal/EEO/equal-opportunity/scam-warning boilerplate. The Accessibility section is about accommodation request; might be considered EEO/legal. Safer to remove it. Also remove "What We Can Offer You" subsections? They are benefits; we should keep benefits. But we need to include only sections with real specific content. Benefits: Health & Wellbeing, Personal & Professional Development, Unconditional Inclusion. Those are specific enough. Also salary range. Also schedule/hybrid. We must not create Pay/Schedule/Benefits sections unless posting actually states them. It does state salary range and hybrid schedule. So we can have sections: About the role? Actually "Who We Are" could be About the role? Could be "About the role" or "About HPE". Use heading "About the role" maybe. But we have "Who We Are" describing company. Could map to "About the role". The instruction: use

  for section headings (e.g. About the role, Responsibilities, Requirements, Qualifications, Skills, Benefits, Pay, Schedule). So we should use those exact headings when appropriate. So we need sections: About the role (company description), Responsibilities, Requirements (knowledge and skills required), Qualifications (experience and education), Skills (maybe preferred?), Benefits, Pay, Schedule. Also maybe "Preferred" could be under Skills or Requirements? We'll treat preferred as additional skills under Skills. We must not repeat job title. No Job ID. We must preserve all substantive information. So we need to include details from each section. Let's extract: Who We Are: Hewlett Packard Enterprise is the global edge-to-cloud company advancing the way people live and work. We help companies connect, protect, analyze, and act on their data and applications wherever they live, from edge to cloud, so they can turn insights into outcomes at the speed required to thrive in today’s complex world. Our culture thrives on finding new and better ways to accelerate what’s next. We know varied backgrounds are valued and succeed here. We have the flexibility to manage our work and personal needs. We make bold moves, together, and are a force for good. If you are looking to stretch and grow your career our culture will embrace you. Open up opportunities with HPE. Job Description: HPE's Private Cloud AI organization is seeking a Principal Software Engineer to lead the model runtime within HPE AI Essentials, the inference platform used by enterprises to operate large language models on infrastructure they own, including air-gapped and sovereign environments. The principal engineering challenge in this domain is not model deployment but sustained execution efficiency: achieving low tail latency and high GPU utilization on customer-owned hardware of varying generation and configuration. In this role you will define the architecture of that runtime – engine integration, batching, KV cache management, and distributed execution – together with the Kubernetes orchestration layer that supports it. The primary work location is as listed, but could be any other HPE site location in the US; however, remote work options will be considered. Responsibilities: bullet list: - Define and own the technical direction of the LLM serving deployment, including engine integration, continuous batching, KV cache management and reuse, and quantized execution - Partner with inference performance engineering teams, with accountability for time-to-first-token, inter-token latency, throughput per GPU, and P95/P99 tail latency - Define distributed inferencing strategy, including disaggregated prefill/decode, tensor and pipeline parallelism, KV cache offload across GPU memory, host memory, and RDMA-attached storage - Evaluate emerging runtimes, quantization schemes, speculative decoding, and mixture-of-experts serving, and determine whether each runtime is adopted, developed in-house, or declined - Define the orchestration layer supporting the runtime, including model admission, GPU scheduling and partitioning, cache-aware request routing, and autoscaling - Mentor engineers, lead design and architecture reviews, and present technical direction to business unit and executive audiences Knowledge and Skills Required: - Production experience with LLM inference engines such as vLLM, SGLang, TensorRT-LLM, TGI, or NVIDIA NIM, including modification of engine internals - Comprehensive understanding of inference internals, including continuous batching, paged attention, KV cache reuse and prefix caching, chunked prefill, quantization, and speculative decoding - Tensor and pipeline parallelism, NCCL collective operations, and the GPU memory hierarchy and interconnect characteristics that govern them - Expert level proficiency in Kubernetes platform architectures, including operators, custom resources, controllers, and scheduling - Strong programming proficiency in Go and Python, with the ability to read, debug, and profile C++/CUDA using tools such as Nsight - Experience with debugging/profiling multi-tier application workloads such as RAG, Agents, etc - Excellent analytical, debugging, and problem-solving abilities Preferred: - Upstream contribution to vLLM, SGLang, TensorRT-LLM, LLM-D, LMCache, or KServe - Disaggregated prefill/decode serving, or KV cache offload and reuse at scale - RDMA, GPUDirect Storage, InfiniBand, or RoCE - MIG, fractional GPU allocation, and multi-tenant GPU isolation - On-premises, air-gapped, or regulated enterprise software delivery Experience and Education: - Minimum of 12 years of experience in Software Engineering, including +1 years working directly on LLM inference runtimes or production model serving - Degree in Computer Science or related field Accessibility: we will remove. What We Can Offer You: - Health & Wellbeing: We strive to provide our team members and their loved ones with a comprehensive suite of benefits that supports their physical, financial and emotional wellbeing. - Personal & Professional Development: We also invest in your career because the better you are, the better we all are. We have specific programs catered to helping you reach any career goals you have — whether you want to become a knowledge expert in your field or apply your skills to another division. - Unconditional Inclusion: We are unconditionally inclusive in the way we work and celebrate individual uniqueness. We know varied backgrounds are valued and succeed here. We have the flexibility to manage our work and personal needs. We make bold moves, together, and are a force for good. Also Let's Stay Connected: social media; we can drop as boilerplate. Pay: Salary range: United States of America: Annual Salary USD 160,000 - 303,000 in Colorado // 152,000 - 349,000 in North Carolina & Texas. The listed salary range reflects base salary. Variable incentives may also be offered. Schedule: Hybrid: requirement that you will work on average 2 days per week from an HPE office. Primary work location as listed, could be any other HPE site location in the US; remote work options considered. We need to output HTML fragment only. We'll structure:

  About the role

  ... (company description)

  Responsibilities

  • ...
  ...

  Requirements

  • ... (knowledge and skills required)
  ...

  Qualifications

  • Minimum of 12 years of experience in Software Engineering, including +1 years working directly on LLM inference runtimes or production model serving
  • Degree in Computer Science or related field

  Skills

  • Upstream contribution to vLLM, SGLang, TensorRT-LLM, LLM-D, LMCache, or KServe
  • Disaggregated prefill/decode serving, or KV cache offload and reuse at scale
  • RDMA, GPUDirect Storage, InfiniBand, or RoCE
  • MIG, fractional GPU allocation, and multi-tenant GPU isolation
  • On-premises, air-gapped, or regulated enterprise software delivery

  Benefits

  • Health & Wellbeing: comprehensive suite of benefits supporting physical, financial and emotional wellbeing for team members and their loved ones.
  • Personal & Professional Development: investment in career with specific programs to help reach career goals, become knowledge expert or apply skills to another division.
  • Unconditional Inclusion: unconditionally inclusive workplace celebrating individual uniqueness, flexible work‑personal balance, bold moves together.

  Pay

  United States of America: Annual Salary USD 160,000 - 303,000 in Colorado; 152,000 - 349,000 in North Carolina & Texas. Base salary range; variable incentives may also be offered.

  Schedule

  Hybrid role: average 2 days per week from an HPE office; primary work location as listed, with possibility to work at other HPE sites in the US; remote work options considered.

  We need to ensure we didn't include any boilerplate like EEO, scam warnings, etc. Also we removed "Let's Stay Connected". Also removed "Accessibility". Also removed "Job Engineering Job Level TCP_05". Also removed "Information about employee benefits offered in the US can be found at ..." (we already captured benefits). Also removed "The estimated job application period closure is December 30 2027". Also removed "HPE is an Equal Employment Opportunity..." etc. Check that we preserved all substantive information: company description, job description, responsibilities, required knowledge/skills, preferred skills, experience/education, benefits, pay, schedule. Also we included hybrid schedule details. Also we included primary work location note. Also we included remote work options considered. Check that we didn't miss any piece from job description: "HPE's Private Cloud AI organization is seeking a Principal Software Engineer to lead the model runtime within HPE AI Essentials, the inference platform used by enterprises to operate large language models on infrastructure they own, including air-gapped and sovereign environments. The principal engineering challenge in this domain is not model deployment but sustained execution efficiency: achieving low tail latency and high GPU utilization on customer-owned hardware of varying generation and configuration. In this role you will define the architecture of that runtime – engine integration, batching, KV cache management, and distributed execution – together with the Kubernetes orchestration layer that supports it. The primary work location is as listed, but could be any other HPE site location in the US; however, remote work options will be considered." We included that in About the role? Actually we placed company description in About the role. The job description details could be part of About the role or maybe a separate section but we don't have a heading for job description. Could incorporate into About the role paragraph. Let's merge both company description and job description into About the role paragraph. Ensure we keep all details. Thus About the role paragraph: combine both. Let's craft:

  Hewlett Packard Enterprise is the global edge-to-cloud company