Research Engineer - Pre-training

We need to output HTML fragment with sections: headings using

for section headings like About the role, Responsibilities, Requirements, Qualifications, Skills, Benefits, Pay, Schedule. We must include only sections that have real specific content. We must not repeat job title. No Job ID etc. Remove boilerplate. Must preserve all substantive info. We need to parse the posting. Content: - Intro paragraph about Pluralis Research, Protocol Learning, Agora, etc. This is "About the role" maybe? Actually it's description of company and mission. Could be "About the role" or maybe "About the company". The instruction says use headings like About the role, Responsibilities, Requirements, Qualifications, Skills, Benefits, Pay, Schedule. So we need to map content accordingly. We have: - Description of work: "As a Research Engineer you'll build the training system that takes Protocol Learning from the 8B run to frontier scale: large models on heterogeneous hardware, in physically different regions, connected by ordinary internet." Then "Key Responsibilities" bullet list: - Distributed pretraining: Implement and optimize model-parallel training. Data, pipeline, and tensor parallelism for large models on heterogeneous GPUs under low-bandwidth, high-latency links. - Performance optimization: Implement techniques that reduce communication overhead while maintaining model convergence in challenging network environments. - Elasticity and fault tolerance: Make runs survive node churn. Robust checkpointing, state synchronization, and recovery as participants join and leave. - Run instrumentation: Build the monitoring that shows throughput, bottlenecks, and model quality across hundreds of devices. Then "What We're Looking For" bullet list: - Hands-on distributed training (required): You've trained models across many devices in PyTorch with FSDP, DeepSpeed, Megatron, or your own implementation. You understand data, tensor, and pipeline parallelism. - Strong engineering: Production-quality Python. Concurrency, failure handling, profiling before optimizing. - Evidence of execution: Shipped systems, research code, open-source work, or serious personal projects. - Mission alignment: You believe Protocol Learning is the viable third path for collective, trustless, and sovereign AI. Then "Nice to Have" bullet list: - Hands-on experience training or serving large language models such as Nemotron, Qwen or OLMo. - Experience with P2P networking and NAT traversal. - Experience with post-training and RL. - Experience with inference and serving systems. - Experience at proprietary, open-weight and open-source AI labs Then "Compensation & Benefits" paragraph: - Equity-Heavy Package: We offer significant ownership for key technical contributors in addition to a high base salary. - Remote-First Culture: Flexible work environment with team members distributed globally. - Visa Sponsorship: Optional full visa sponsorship and relocation support to either Australia or the US. - Open Problems: Training and serving frontier models on hardware you don't control, over networks you don't own, mostly has no published answers yet. You'll write some of the first ones. Then "FYI's" paragraph: - We work remotely across the world, with the main teams in Australia and North America. You'll need to be comfortable working across timezones. - Applicants must have professional-level English proficiency (written and spoken). - Recruiters: we aren't looking for agency support at this time. We'll reach out if we need help. - We are backed by Union Square Ventures and other tier-1 investors, and we are a world-class, deeply technical team of ML researchers. - Pluralis is unapologetically ideological. We believe AI, and the world, end up on a better path if we succeed in implementing the protocol for intelligence. If this resonates, please apply. We need to remove boilerplate: cookie/navigation/legal/EEO/equal-opportunity/scam-warning boilerplate, duplicated lines, and application instructions. The last sentence "If this resonates, please apply." is application instruction; should be removed. Also "Recruiters: we aren't looking for agency support at this time. We'll reach out if we need help." is maybe not boilerplate but could be considered application instruction? It's about recruiters; maybe keep? It's not essential job info but maybe we can keep as it's about process. The instruction says remove application instructions. That line is about recruiters, not exactly application instruction but could be considered. Safer to remove it. Also "FYI's" heading is not a standard heading; we can incorporate its content into appropriate sections: maybe "About the role" or "Benefits". The content about remote work, timezones, English proficiency could go under "About the role" or maybe a separate section like "Work Environment". But we only allowed headings list; we could put under "About the role" or "Benefits". Since remote-first culture already in Benefits, we can add remote work details there. English proficiency maybe under "Requirements". Visa sponsorship under Benefits. Open Problems maybe under "About the role". The "FYI's" also includes backed by investors etc; could be under "About the role". We need to decide sections: - About the role: include company mission description, what they do, open problems, maybe investor backing, ideological statement. - Responsibilities: the key responsibilities bullet list. - Requirements: What We're Looking For (required) bullet list. - Qualifications: maybe Nice to Have? Could be "Qualifications" or "Skills". The instruction includes Skills heading. We could put Nice to Have under Skills. - Benefits: Compensation & Benefits paragraph (equity, base salary, remote-first culture, visa sponsorship, open problems? maybe open problems is not benefit). Also remote-first culture already there. - Pay: Not explicitly stated besides "high base salary". No specific number. The instruction says never create Pay/Schedule/Benefits sections unless posting actually states them. It does state "high base salary" but not a number. Is that enough? It says "We offer significant ownership for key technical contributors in addition to a high base salary." That's mention of compensation but not specific. The instruction says avoid filler like "commensurate with experience", "not specified", "N/A", "varies", "TBD". "high base salary" is vague but still a statement. Might be okay to include Pay section with that statement? However they said "never create Pay/Schedule/Benefits sections unless the posting actually states them." It does state compensation (equity-heavy package, high base salary). So we can include Benefits section (maybe combine Pay and Benefits). But we have separate headings for Pay and Benefits. Could we put compensation details under Benefits? The instruction says we can use Benefits heading for benefits. Pay heading for pay. Since we have both equity and base salary, we could put base salary under Pay, equity and other benefits under Benefits. But we risk creating thin sections. Better to combine into Benefits section (since they said prefer few substantive sections over many thin ones). So we can have Benefits section covering equity, base salary, remote-first culture, visa sponsorship, open problems? open problems is not benefit. Maybe keep open problems under About the role. Schedule: Not mentioned. So we omit Schedule. Thus sections: About the role, Responsibilities, Requirements, Skills, Benefits. Now we need to format:

About the role

... paragraphs ...

Use

• for lists. We need to preserve all substantive info, not invent or summarize away detail. Let's craft About the role paragraph(s): - Pluralis Research works on Protocol Learning: training and serving large models in a fully decentralized way on small consumer-grade devices connected via the internet. Despite being dismissed as infeasible, we have made significant advances on this problem, most recently Agora, a permissionless run that pretrained an 8B model from scratch on consumer GPUs spread over the internet, with no single participant ever holding the full weights (tech report). While many of the core research problems have been solved, Protocol Learning unlocks a series of new challenges. For the mission in full, read A Third Path: Protocol Learning. This setting breaks nearly every assumption of datacenter training: communication-efficient training across different parallelism axes, fault tolerance as nodes join and drop mid-run, heterogeneous compute and networks, and robustness to malicious participants. Our published methods include Subspace Networks, Factored Gossip DiLoCo, AsyncMesh, and Sentinel. As a Research Engineer you'll build the training system that takes Protocol Learning from the 8B run to frontier scale: large models on heterogeneous hardware, in physically different regions, connected by ordinary internet. We work remotely across the world, with the main teams in Australia and North America. You'll need to be comfortable working across timezones. Applicants must have professional-level English proficiency (written and spoken). We are backed by Union Square Ventures and other tier-1 investors, and we are a world-class, deeply technical team of ML researchers. Pluralis is unapologetically ideological. We believe AI, and the world, end up on a better path if we succeed in implementing the protocol for intelligence. We removed "If this resonates, please apply." and recruiter line. Now Responsibilities: list as
  • each bullet. Bullet points: - Distributed pretraining: Implement and optimize model-parallel training. Data, pipeline, and tensor parallelism for large models on heterogeneous GPUs under low-bandwidth, high-latency links. - Performance optimization: Implement techniques that reduce communication overhead while maintaining model convergence in challenging network environments. - Elasticity and fault tolerance: Make runs survive node churn. Robust checkpointing, state synchronization, and recovery as participants join and leave. - Run instrumentation: Build the monitoring that shows throughput, bottlenecks, and model quality across hundreds of devices. Now Requirements (What We're Looking For required): - Hands-on distributed training (required): You've trained models across many devices in PyTorch with FSDP, DeepSpeed, Megatron, or your own implementation. You understand data, tensor, and pipeline parallelism. - Strong engineering: Production-quality Python. Concurrency, failure handling, profiling before optimizing. - Evidence of execution: Shipped systems, research code, open-source work, or serious personal projects. - Mission alignment: You believe Protocol Learning is the viable third path for collective, trustless, and sovereign AI. Now Skills (Nice to Have): - Hands-on experience training or serving large language models such as Nemotron, Qwen or OLMo. - Experience with P2P networking and NAT traversal. - Experience with post-training and RL. - Experience with inference and serving systems. - Experience at proprietary, open-weight and open-source AI labs. Now Benefits: - Equity-Heavy Package: We offer significant ownership for key technical contributors in addition to a high base salary. - Remote-First Culture: Flexible work environment with team members distributed globally. - Visa Sponsorship: Optional full visa sponsorship and relocation support to either Australia or the US. - Open Problems: Training and serving frontier models on hardware you don't control, over networks you don't own, mostly has no published answers yet. You'll write some of the first ones. But open problems maybe not benefit; but it's under Compensation & Benefits originally. It's more like a description of the work. Could move to About the role. However we already have a lot in About the role. Might be okay to keep under Benefits as it's part of the compensation & benefits section. But it's not a benefit; it's a description of challenges. Might be better to put under About the role. Let's think: The original "Compensation & Benefits" paragraph includes those four bullet-like sentences: Equity-Heavy Package, Remote-First Culture, Visa Sponsorship, Open Problems. The open problems sentence is more about the work. But we could keep it under Benefits as it's part of that paragraph; but the instruction says prefer few substantive sections over many thin ones. Having a Benefits section with four items is fine. We could also incorporate open problems into About the role to avoid thin Benefits section. But we already have About the role long; adding open problems there is fine. Let's decide: Keep Benefits with equity, remote-first, visa sponsorship. Move open problems to About the role (makes sense). Then Benefits section will have three items. Thus Benefits: -