ML Engineer - Remote

About the Company

Pluralis Research conducts foundational research into Protocol Learning, a multi-participant approach to training foundation models where no single participant has, or can obtain, a complete copy of the model. The company aims to facilitate the development of community-trained and community-owned frontier models supported by self-sustaining economics. Pluralis Research is building novel infrastructure for distributed machine learning across decentralized environments.

About the Role

Pluralis Research is seeking Senior and Staff Machine Learning Engineers with 5+ years of experience in distributed systems and large-scale machine learning training. In this role, you will implement a novel technical substrate for training distributed ML models across consumer-grade internet connections. You will work on distributed training architecture, model parallelism, GPU optimization, decentralized networking, resilience, and communication efficiency.

Responsibilities

Distributed Training Architecture & Optimization

• Design and implement large-scale distributed training systems for heterogeneous hardware.
• Optimize training systems for low-bandwidth and high-latency environments.
• Develop model-parallel training strategies covering data, tensor, and pipeline parallelism.
• Implement custom sharding techniques to minimize communication overhead.
• Optimize GPU utilization, memory efficiency, and compute performance across distributed nodes.
• Build robust checkpointing, state synchronization, and recovery mechanisms for long-running training jobs.
• Develop monitoring and metrics systems to track training progress, model quality, and system bottlenecks.

Decentralized Networking & Resilience

• Architect resilient training systems capable of handling node failures and network partitions.
• Support environments where participants can dynamically join or leave.
• Design and optimize peer-to-peer topologies for decentralized coordination.
• Implement NAT traversal, peer discovery, dynamic routing, and connection lifecycle management.
• Profile and optimize communication patterns to reduce latency and bandwidth overhead.
• Build reliable systems for coordinating distributed workloads across non-co-located nodes.

Required Qualifications

• 5+ years of experience in distributed systems and large-scale machine learning training.
• Strong experience building and operating distributed systems in production.
• Hands-on expertise with distributed training frameworks such as FSDP, DeepSpeed, Megatron, or similar technologies.
• Deep understanding of data, tensor, and pipeline parallelism.
• Expert-level Python development experience.
• Production experience with concurrency, error handling, retry logic, and clean software architecture.
• Strong networking fundamentals.
• Experience with P2P systems, gRPC, routing, NAT traversal, and distributed coordination.
• Experience optimizing GPU workloads, memory management, and large-scale compute efficiency.

Technical Focus

• Distributed machine learning.
• Large-scale model training.
• FSDP, DeepSpeed, Megatron, or comparable frameworks.
• Data, tensor, and pipeline parallelism.
• GPU utilization and memory optimization.
• Python and concurrent systems.
• Peer-to-peer networking.
• gRPC and distributed coordination.
• NAT traversal and peer discovery.
• Fault-tolerant distributed infrastructure.

Compensation

• Equity-heavy compensation with meaningful ownership.
• Competitive base salary for senior engineering roles in Australia.

Work Arrangement

• Remote position.
• Based on the provided listing, the role is associated with San Francisco, CA.
• Work involves distributed systems operating across geographically separated nodes and consumer-grade internet connections.