Staff AI Product Engineer
for section headings (e.g. About the role, Responsibilities, Requirements, Qualifications, Skills, Benefits, Pay, Schedule). Do NOT repeat the job title as a heading. Do NOT put Job ID / requisition numbers as headings — fold minor metadata into a short
or drop it. Preserve ALL substantive information; do NOT invent or summarize away detail. Remove cookie/navigation/legal/EEO/equal-opportunity/scam-warning boilerplate, duplicated lines, and application instructions. Include a section ONLY when the posting has real, specific content for it — never emit a heading followed by filler/placeholder text (e.g. "commensurate with experience", "not specified", "N/A", "varies", "TBD") and never create Pay/Schedule/Benefits sections unless the posting actually states them. We have salary range: "Salary Range: $220,000 USD - $293,333 USD". Also mention benefits: "Nscale may offer a competitive benefits package including medical, dental, vision, flexible paid time off, parental leave, and retirement plan participation." So we have Benefits and Pay sections. No schedule info. So we should include Pay and Benefits sections. We also have "About Nscale" which is company description; maybe we can include as a short paragraph before sections? The instruction says use
for section headings (e.g. About the role, Responsibilities, Requirements, Qualifications, Skills, Benefits, Pay, Schedule). It doesn't forbid other headings like About Nscale? It says use for section headings (e.g. About the role, Responsibilities, Requirements, Qualifications, Skills, Benefits, Pay, Schedule). It doesn't say we cannot have other headings, but likely we should stick to those headings. However "About Nscale" is not one of the listed examples; but we could treat it as a paragraph without heading, or we could include as
. The instruction: "Do NOT repeat the job title as a heading (the page already shows it) and do NOT put Job ID / requisition numbers as headings — fold minor metadata into a short
or drop it." It doesn't forbid other headings, but we should follow the pattern: use
for section headings (e.g. About the role, Responsibilities, Requirements, Qualifications, Skills, Benefits, Pay, Schedule). So we can have headings: About the role, Responsibilities, Requirements, Benefits, Pay. Possibly also Qualifications? The posting has Requirements section; we could treat that as Requirements. There's no separate Qualifications. There's also "Preferred" but that's not a separate heading; we could include under Requirements as a subsection? But we need to preserve all substantive info. We could keep Requirements as a list, and include Preferred as part of that list maybe with a note? But better to keep as part of Requirements list, maybe separate for Preferred? The instruction: Use - for any list of duties/requirements/benefits. So we can have multiple lists.
We need to preserve all substantive info: About Nscale paragraph, About The Role paragraph (the role description), Responsibilities (with subheadings Inference, Reinforcement learning & post-training, Developer APIs & platform, Leadership & direction). The responsibilities are detailed with bullet-like sentences. We need to output as
- for any list of duties/requirements/benefits. So we should convert each responsibility bullet into
- . The responsibilities section includes headings like "Inference", "Reinforcement learning & post-training", etc. Those could be treated as subheadings? But we only allowed
for section headings; we could still use or ? Not specified. Safer to keep them as part of - text, maybe using inside
- . But we must not invent headings not allowed. The instruction only restricts using
for section headings; doesn't forbid other heading levels. But to be safe, we can keep them as plain text within
- . However we need to preserve the structure; we could keep the subheadings as inside
- . That's acceptable.
Similarly, Requirements list includes many items; we can output as
- . Preferred items also as
- maybe under a "Preferred" heading? But we shouldn't create a heading unless there is real specific content; we have Preferred content, so we could add a heading "Preferred" using
? The list of allowed headings includes Qualifications, Skills, etc. Not Preferred. But we could still use for Preferred? The instruction says use for section headings (e.g. About the role, Responsibilities, Requirements, Qualifications, Skills, Benefits, Pay, Schedule). It doesn't say we cannot use other headings, but to be safe we could avoid extra headings and just include Preferred items as part of Requirements list, maybe preceded by a note like "Preferred:" inside a
. But we need to preserve all substantive info; we could add a
"Preferred:" then a
. That seems okay.
Benefits: we have description; we can output as - items.
Pay: we have salary range; we can output as
maybe.
We must not include boilerplate like "For information on how Nscale handles candidate personal data, please see our Employee & Candidate Privacy Notice: Here." Remove that.
Also remove "About Nscale" maybe we keep as a paragraph without heading.
Let's craft HTML:
About the role
[the About The Role paragraph]
Responsibilities
- Inference: Set technical direction for Nscale’s inference serving architecture: request routing, scheduling, continuous batching, KV cache management, prefix caching, and speculative decoding; Drive the strategy for model efficiency in production — quantization (FP8, INT8/4), sparsity, pruning, distillation, and MoE serving — and the trade-offs between cost, latency, throughput, and model quality; Lead the resolution of systemic performance and reliability challenges across the serving stack, from kernel-level bottlenecks to fleet-level capacity and multi-tenant isolation
- Reinforcement learning & post-training: Own the architecture of Nscale’s RL and post-training systems: RLHF, DPO/GRPO-style methods, reward modelling, and agentic RL with tool calling, off-policy training, and decoupled sampling and policy updates; Define how inference and training share infrastructure in RL loops — rollout generation, sample buffering, weight synchronization, and the serving engine’s role inside the training system; Establish standards for fine-tuning services (LoRA, QLoRA, adapters, full fine-tuning) and the data curation and processing workflows that feed them
- Developer APIs & platform: Set the design standards for Nscale’s developer-facing APIs, SDKs, and tooling — OpenAI-compatible and native interfaces, OpenAPI 3.x specifications, versioning, and rate limiting — so that inference and RL capabilities are consumable by engineers who never see the underlying systems; Create reusable frameworks and tooling that multiply the effectiveness of other AI engineers across Nscale
- Leadership & direction: Coach and grow more junior engineers across teams; raise technical capability and engineering quality broadly; Collaborate with research, product, and infrastructure leadership to align inference and RL platform strategy with customer demand and business direction; Evaluate emerging serving engines, RL frameworks, and accelerator technologies; make decisive build/adopt/contribute recommendations; Represent the inference and RL domain in cross-organizational architecture reviews and strategic planning
Requirements
- 8–12 years of engineering experience, with significant depth in production AI systems at scale (AI labs, hyperscalers, or leading ML infrastructure companies)
- Demonstrated ability to set technical direction for an AI systems domain at scale
- Deep expertise in production LLM inference: serving architectures, KV cache and memory management, batching and scheduling strategies, speculative decoding, and low-precision inference
- Strong hands-on expertise in RL for LLMs — RLHF, DPO and related preference-optimization methods, reward modelling, or agentic/multi-turn RL — including the systems that make them run efficiently on GPU clusters
- Proven ability to design developer-facing APIs and SDKs that are clean, versioned, and adopted by other engineers and external customers
- Strong cross-team influence: track record of creating standards and practices adopted by multiple teams
- Experience designing AI infrastructure with clear control plane / data plane separation and cell-based architecture patterns for scale-out and blast-radius isolation
- Experience with large-scale GPU/accelerator workloads: CUDA or ROCm, memory bandwidth optimization, and distributed compute paradigms (data/model/tensor parallelism, sharding, scheduling)
- Strong proficiency in Python and PyTorch, with a track record of building maintainable, well-tested, production-grade ML systems
- Deep understanding of transformer and LLM architectures and their behavior under production load
- Ability to design architecture that is both technically excellent and practically adoptable across a diverse engineering organization
Preferred:
- Contributions to widely-used open-source inference or RL frameworks (vLLM, SGLang, TensorRT-LLM, verl, OpenRLHF, TRL, DeepSpeed, etc.)
- Experience at an AI lab (OpenAI, DeepMind, Anthropic, Meta AI, etc.), hyperscaler AI team, or leading ML infrastructure company
- Published research or technical writing on inference systems or RL infrastructure (MLSys, NeurIPS, ICLR systems tracks, blog posts)
- Experience with hardware-software co-design for AI accelerators (custom CUDA kernels, Triton, etc.)
- Experience operating inference platforms in containerized, distributed environments (Kubernetes, large-scale clusters)
- Experience building evaluation and benchmarking systems for model quality, safety, and system performance
Benefits
- Medical
- Dental
- Vision
- Flexible paid time off
- Parental leave
- Retirement plan participation
Pay
Salary Range: $220,000 USD - $293,333 USD
We also need to include About Nscale paragraph maybe as a before sections. Let's add:
Nscale is taking on the hyperscalers by building a vertically integrated GenAI cloud platform. We own the data centers, software, and applications that power today's AI stack using sustainable technology solutions. We thrive on a culture of relentless innovation, ownership, and accountability, where every team member takes pride in their work and drives it with excellence and urgency. As a Nscaler, you'll build trust through openness and transparency, where everyone is inspired to do their best work. Collaboration is key, and we work together swiftly and respectfully, embracing adaptability and resilience in all we do.
Then About the role paragraph:
Nscale is looking for a Staff AI Engineer (Specialized) to set technical direction for the inference and reinforcement learning systems at the core of our AI services platform — and for the APIs through which other engineers consume them. This role owns the architecture of how models are served on Nscale’s GPU cloud, how RL and post-training workloads run
- for Preferred? The instruction: Use
- for any list of duties/requirements/benefits. So we can have multiple lists.
We need to preserve all substantive info: About Nscale paragraph, About The Role paragraph (the role description), Responsibilities (with subheadings Inference, Reinforcement learning & post-training, Developer APIs & platform, Leadership & direction). The responsibilities are detailed with bullet-like sentences. We need to output as
- for any list of duties/requirements/benefits. So we should convert each responsibility bullet into
- . The responsibilities section includes headings like "Inference", "Reinforcement learning & post-training", etc. Those could be treated as subheadings? But we only allowed
for section headings; we could still use
or ? Not specified. Safer to keep them as part of
- text, maybe using inside
- . But we must not invent headings not allowed. The instruction only restricts using
for section headings; doesn't forbid other heading levels. But to be safe, we can keep them as plain text within
- . However we need to preserve the structure; we could keep the subheadings as inside
- . That's acceptable. Similarly, Requirements list includes many items; we can output as
- . Preferred items also as
- maybe under a "Preferred" heading? But we shouldn't create a heading unless there is real specific content; we have Preferred content, so we could add a heading "Preferred" using
? The list of allowed headings includes Qualifications, Skills, etc. Not Preferred. But we could still use
for Preferred? The instruction says use
for section headings (e.g. About the role, Responsibilities, Requirements, Qualifications, Skills, Benefits, Pay, Schedule). It doesn't say we cannot use other headings, but to be safe we could avoid extra headings and just include Preferred items as part of Requirements list, maybe preceded by a note like "Preferred:" inside a
. But we need to preserve all substantive info; we could add a
"Preferred:" then a
- . That seems okay.
Benefits: we have description; we can output as
- items.
Pay: we have salary range; we can output as
maybe. We must not include boilerplate like "For information on how Nscale handles candidate personal data, please see our Employee & Candidate Privacy Notice: Here." Remove that. Also remove "About Nscale" maybe we keep as a paragraph without heading. Let's craft HTML:
About the role
[the About The Role paragraph]
Responsibilities
- Inference: Set technical direction for Nscale’s inference serving architecture: request routing, scheduling, continuous batching, KV cache management, prefix caching, and speculative decoding; Drive the strategy for model efficiency in production — quantization (FP8, INT8/4), sparsity, pruning, distillation, and MoE serving — and the trade-offs between cost, latency, throughput, and model quality; Lead the resolution of systemic performance and reliability challenges across the serving stack, from kernel-level bottlenecks to fleet-level capacity and multi-tenant isolation
- Reinforcement learning & post-training: Own the architecture of Nscale’s RL and post-training systems: RLHF, DPO/GRPO-style methods, reward modelling, and agentic RL with tool calling, off-policy training, and decoupled sampling and policy updates; Define how inference and training share infrastructure in RL loops — rollout generation, sample buffering, weight synchronization, and the serving engine’s role inside the training system; Establish standards for fine-tuning services (LoRA, QLoRA, adapters, full fine-tuning) and the data curation and processing workflows that feed them
- Developer APIs & platform: Set the design standards for Nscale’s developer-facing APIs, SDKs, and tooling — OpenAI-compatible and native interfaces, OpenAPI 3.x specifications, versioning, and rate limiting — so that inference and RL capabilities are consumable by engineers who never see the underlying systems; Create reusable frameworks and tooling that multiply the effectiveness of other AI engineers across Nscale
- Leadership & direction: Coach and grow more junior engineers across teams; raise technical capability and engineering quality broadly; Collaborate with research, product, and infrastructure leadership to align inference and RL platform strategy with customer demand and business direction; Evaluate emerging serving engines, RL frameworks, and accelerator technologies; make decisive build/adopt/contribute recommendations; Represent the inference and RL domain in cross-organizational architecture reviews and strategic planning
Requirements
- 8–12 years of engineering experience, with significant depth in production AI systems at scale (AI labs, hyperscalers, or leading ML infrastructure companies)
- Demonstrated ability to set technical direction for an AI systems domain at scale
- Deep expertise in production LLM inference: serving architectures, KV cache and memory management, batching and scheduling strategies, speculative decoding, and low-precision inference
- Strong hands-on expertise in RL for LLMs — RLHF, DPO and related preference-optimization methods, reward modelling, or agentic/multi-turn RL — including the systems that make them run efficiently on GPU clusters
- Proven ability to design developer-facing APIs and SDKs that are clean, versioned, and adopted by other engineers and external customers
- Strong cross-team influence: track record of creating standards and practices adopted by multiple teams
- Experience designing AI infrastructure with clear control plane / data plane separation and cell-based architecture patterns for scale-out and blast-radius isolation
- Experience with large-scale GPU/accelerator workloads: CUDA or ROCm, memory bandwidth optimization, and distributed compute paradigms (data/model/tensor parallelism, sharding, scheduling)
- Strong proficiency in Python and PyTorch, with a track record of building maintainable, well-tested, production-grade ML systems
- Deep understanding of transformer and LLM architectures and their behavior under production load
- Ability to design architecture that is both technically excellent and practically adoptable across a diverse engineering organization
Preferred:
- Contributions to widely-used open-source inference or RL frameworks (vLLM, SGLang, TensorRT-LLM, verl, OpenRLHF, TRL, DeepSpeed, etc.)
- Experience at an AI lab (OpenAI, DeepMind, Anthropic, Meta AI, etc.), hyperscaler AI team, or leading ML infrastructure company
- Published research or technical writing on inference systems or RL infrastructure (MLSys, NeurIPS, ICLR systems tracks, blog posts)
- Experience with hardware-software co-design for AI accelerators (custom CUDA kernels, Triton, etc.)
- Experience operating inference platforms in containerized, distributed environments (Kubernetes, large-scale clusters)
- Experience building evaluation and benchmarking systems for model quality, safety, and system performance
Benefits
- Medical
- Dental
- Vision
- Flexible paid time off
- Parental leave
- Retirement plan participation
Pay
Salary Range: $220,000 USD - $293,333 USD
We also need to include About Nscale paragraph maybe as abefore sections. Let's add:
Nscale is taking on the hyperscalers by building a vertically integrated GenAI cloud platform. We own the data centers, software, and applications that power today's AI stack using sustainable technology solutions. We thrive on a culture of relentless innovation, ownership, and accountability, where every team member takes pride in their work and drives it with excellence and urgency. As a Nscaler, you'll build trust through openness and transparency, where everyone is inspired to do their best work. Collaboration is key, and we work together swiftly and respectfully, embracing adaptability and resilience in all we do.
Then About the role paragraph:Nscale is looking for a Staff AI Engineer (Specialized) to set technical direction for the inference and reinforcement learning systems at the core of our AI services platform — and for the APIs through which other engineers consume them. This role owns the architecture of how models are served on Nscale’s GPU cloud, how RL and post-training workloads run
- items.
Pay: we have salary range; we can output as
- maybe under a "Preferred" heading? But we shouldn't create a heading unless there is real specific content; we have Preferred content, so we could add a heading "Preferred" using
- text, maybe using inside