Software Engineer, ML Data Infrastructure

About the role

We are hiring a Software Engineer to build and run the data infrastructure that feeds our ML training and inference systems. You will design high‑throughput, distributed data systems on top of columnar and lakehouse formats, so that GPUs and models are never starved for data. This is a hands‑on role for someone who enjoys performance engineering and wants to work close to the boundary between data systems and machine learning.

Responsibilities

• Design, build, and operate large‑scale distributed data systems that serve ML training and inference workloads in production.
• Develop strong Python engineering in Rust (or C++/Go) with Python bindings for ML practitioners to build high‑performance data loading, storage, and retrieval layers.
• Evaluate and adopt columnar and lakehouse formats (Parquet, Iceberg, Delta, Lance), making trade‑offs explicit for schema evolution, random access, scan performance, and versioning.
• Optimize I/O‑bound pipelines using Arrow, zero‑copy techniques, memory mapping, async I/O, and efficient object storage access patterns (request coalescing, prefetching, caching, parallel range reads).
• Profile and remove bottlenecks across the data path, from object storage to host memory to accelerator, keeping training and inference compute‑bound.
• Partner with ML researchers and engineers to translate training loops, evaluation, and inference data consumption into system requirements.
• Define reliability, observability, and cost standards for data infrastructure, including SLOs, monitoring, capacity planning, and incident response.
• Write design documents, review code, and mentor engineers on performance engineering and distributed systems practices.

Requirements

• Extensive experience building and running large‑scale distributed data or ML infrastructure in production.
• Strong programming skills in Python and a systems language for performance‑critical work (Rust strongly preferred; C++ or Go acceptable).
• Deep familiarity with columnar and lakehouse formats (Parquet, Iceberg, Delta, or Lance) and their trade‑offs.
• Hands‑on performance engineering for I/O‑bound workloads: Arrow, zero‑copy, memory mapping, async I/O, and high‑throughput object storage access patterns.
• Working knowledge of the end‑to‑end ML workflow and how training and inference workloads consume data.

Preferred Qualifications

• Production Rust experience, including async runtimes (Tokio), FFI, and Python bindings (PyO3 or similar).
• Contributions to open‑source data or ML infrastructure projects such as Arrow, Lance, Iceberg, Ray, or PyTorch data loading.
• Experience with distributed computing and orchestration frameworks (Spark, Ray, Dask, Kubernetes).
• Familiarity with multimodal or large‑scale unstructured data (images, video, audio, embeddings) and vector or random‑access storage.
• Experience with GPU‑aware data pipelines, including pinned memory, GPUDirect Storage, and overlapping I/O with compute.
• Track record of cost optimization for cloud object storage and egress at petabyte scale.
• Experience leading technical design across teams and mentoring other engineers.
• BS/MS/PhD in Computer Science or related field, or equivalent practical experience.

Skills

• Python, Rust, C++, Go
• ML data infrastructure, distributed data systems, high‑throughput data pipelines
• Performance engineering, profiling, bottleneck removal
• Columnar and lakehouse formats: Parquet, Apache Iceberg, Delta Lake, Lance
• Apache Arrow, zero‑copy, memory mapping, async I/O
• Object storage optimization: caching, prefetching, parallel range reads
• GPU‑aware data pipelines, pinned memory, GPUDirect Storage
• Multimodal data, embeddings, vector storage
• Distributed computing frameworks: Spark, Ray, Dask, Kubernetes
• Observability, monitoring, SLOs, reliability engineering, capacity planning, incident response
• Cost optimization for cloud storage
• Technical design, code review, mentoring

Pay

Pay Rate: $60 – $70 per hour (W2 contract).

Location: Cupertino, CA – Remote.

Bayside Solutions, Inc. is not able to sponsor any candidates at this time. Candidates must qualify as a W2 candidate.