Member of Technical Staff, ML Compilers & Code Generation

About the role

Build the compiler path that turns a workload into a correct, target‑aware executable across General Diffusion’s multi‑architecture testbed. You will own compilation and code generation—not fleet placement policy, hand‑authored kernel hot paths, or the independent safety decision to permit an action.

Responsibilities

• Design and evolve target‑independent IR contracts, legality rules, and staged lowering boundaries that preserve workload semantics while enabling backend specialization.
• Implement and maintain compiler passes, analyses, and target backends that lower supported ML workloads into executable programs for heterogeneous external accelerators.
• Build generated‑code conformance, semantic‑equivalence, and numerical‑correctness tests against defined references; make unsupported cases and assumptions explicit.
• Profile compiled workloads to identify graph, memory, scheduling, and code‑generation bottlenecks, then turn findings into reproducible optimizations and benchmark regressions.
• Define machine‑readable backend capability, compilation‑cost, and execution‑characteristic reports that Runtime & Placement can consume without owning placement decisions.
• Work with Kernels on clear compiler‑to‑kernel interfaces and fallback paths.
• Collaborate with Safety & Formal Verification on evidence and hooks needed for independent enforcement.
• Establish a documented compilation path for representative workloads with explicit IR/lowering contracts, target coverage, unsupported‑case behavior, and automated legality checks.
• Maintain a cross‑target evidence suite that compares generated programs with defined references, records numerical and semantic results, and turns failures into minimized regression cases.
• Provide versioned backend capability and performance signals with traceability from workload decisions to compiled artifacts and measured outcomes.

Requirements

• Experience shipping compiler passes, lowering pipelines, code generation, or accelerator backends—not solely using compiler frameworks.
• Strong command of MLIR, LLVM, XLA/StableHLO, Triton, TVM, Mojo, or comparable compiler stacks, including IR design and dialect/lowering mechanics.
• Proven performance‑engineering practice: use profiling and controlled benchmarks to distinguish graph, memory, runtime, and generated‑code effects.
• Ability to reason rigorously about compiler correctness, numerical behavior, shape/layout constraints, and target‑specific legality across more than one hardware class.
• Production‑quality systems engineering in C++, Rust, Python, or a comparable language, with disciplined debugging, tests, and reproducible performance investigations.