Developer - I/O Acceleration
Hakkōda, an IBM Company · San Jose, CA · Yesterday
Information TechnologyFull-time
About the role
The I/O Acceleration team builds the storage-to-GPU data path that makes the promise of breakthrough price/performance on state-of-the-art GPUs and high-performance storage real.
Responsibilities
- Design, build, and optimize accelerated I/O and decompression paths for data-intensive analytics workloads.
- Improve end-to-end throughput across the storage → network → host → GPU boundary, eliminating copies, syscalls, and stalls.
- Integrate with GPU-aware runtimes and high-bandwidth fabrics (GPUDirect Storage, RDMA, NVMe-oF) and tune for Blackwell-class hardware.
- Build benchmarks and microbenchmarks that expose I/O cliffs, queue contention, and tail latency under realistic query mixes.
- Instrument the data path so cost-per-query, bandwidth-per-GPU, and CPU overhead are first-class, observable metrics.
- Collaborate with the query engine, storage, and hardware teams to co-design APIs that make accelerated I/O usable, not just possible.
Requirements
- Strong modern C++ and deep comfort with Linux systems internals (page cache, O_DIRECT, io_uring, NUMA, scheduling).
- Hands-on experience in at least one of: storage I/O subsystems, decompression and codec implementation, or query-engine data paths.
- Working knowledge of GPU-aware pipelines or adjacent acceleration frameworks (CUDA, GPUDirect, or similar).
- Strong performance-profiling and bottleneck-isolation skills — you can read a flame graph, an nsys trace, and an fio result and know what to do next.
- Familiarity with distributed data systems and the realities of running them at scale.
- Production experience with GPUDirect Storage, RDMA, or NVMe-oF integrations.
- Exposure to ESS6000, Lustre, GPFS, or other parallel and clustered file systems.
- Track record of delivering production software in Agile, collaborative environments, including contributing to automated CI/CD pipelines.
- Contributions to open-source data, storage, or GPU-runtime projects (Arrow, cuDF, Velox, DuckDB, Spark, and similar).
Qualifications
United States Software Engineering Professional San Jose, US (0147)
Skills
- Strong modern C++ and deep comfort with Linux systems internals (page cache, O_DIRECT, io_uring, NUMA, scheduling).
- Hands-on experience in at least one of: storage I/O subsystems, decompression and codec implementation, or query-engine data paths.
- Working knowledge of GPU-aware pipelines or adjacent acceleration frameworks (CUDA, GPUDirect, or similar).
- Strong performance-profiling and bottleneck-isolation skills — you can read a flame graph, an nsys trace, and an fio result and know what to do next.
- Familiarity with distributed data systems and the realities of running them at scale.
- Production experience with GPUDirect Storage, RDMA, or NVMe-oF integrations.
- Exposure to ESS6000, Lustre, GPFS, or other parallel and clustered file systems.
- Track record of delivering production software in Agile, collaborative environments, including contributing to automated CI/CD pipelines.
- Contributions to open-source data, storage, or GPU-runtime projects (Arrow, cuDF, Velox, DuckDB, Spark, and similar).
Benefits
N/A
Pay
N/A
Schedule
N/A