Member of Technical Staff - Research Software Engineer
Reflection · San Francisco, CA · 2 wk ago
On-siteEngineeringFull-time
About the role
The Roles at Our Mission Reflection bridge the gap between research and production by transforming cutting-edge algorithms into scalable training systems. This team owns and evolves the core infrastructure behind our training systems, focusing on reinforcement learning training loops, distributed GPU systems, and large-scale data pipelines.
Responsibilities
- Designing and optimizing large-scale training loops and data pipelines.
- Implementing state-of-the-art techniques and ensuring they are numerically stable and computationally efficient.
- Building internal tooling for launching, monitoring, and reproducing complex experiments.
- Diagnosing deep bottlenecks across the training stack (GPU memory issues, communication overhead, dataloader stalls).
- Translating research prototypes into reusable, production-grade infrastructure.
What This Team Does
- Distributed Training
- GPU parallelism (data, tensor, pipeline, expert)
- Communication optimization (NCCL, RDMA, GPU interconnects)
- FSDP / ZeRO and model sharding
- Orchestration & Runtime Systems
- Ray, Kubernetes, Slurm
- Distributed runtimes and async systems
- Containerization and sandboxing
- Frameworks
- PyTorch, JAX, Megatron-style training stacks, Triton / custom kernels
- Data Infrastructure
- Large-scale dataset curation pipelines
- Deduplication and filtering systems
- Tokenization and preprocessing
- Distributed data processing frameworks
About You
- You are a strong software engineer who speaks the language of machine learning.
- You may not have a PhD, but you know how to implement a research paper.
- You have deep experience in at least one of the following: Distributed Training & Inference or Data Infrastructure.
- You enjoy working at the boundary between: Machine learning algorithms, Distributed systems, High-performance computing.
- You care deeply about performance, numerical stability, and reproducibility.
- You thrive in high-agency environments and enjoy solving hard technical problems.
What We Offer
- Top-tier compensation: Salary and equity structured to recognize and retain our talent globally.
- Stock options: Everyone who joins and contributes to Reflection's success gets to share in the upside through stock options.
- Health & wellness: Comprehensive medical, dental, vision, and life, with an annual wellness allowance.
- Meals: Lunch and dinner are provided in the office daily.
- Life & family: 22 weeks paid parental leave for all new birthing and non-birthing parents, including adoptive and surrogate journeys.
- Vacation days: Unlimited paid time off in the U.S. and 30 days in the U.K.
- Sponsorship support: We sponsor visas to help exceptional talent join our team and support long-term immigration pathways where applicable.
- Team building: We have regular off-sites, happy hours, and team celebrations.