Member of Technical Staff, Software Engineer (Data Infrastructure)

About the role

Physical AI runs on data that behaves nothing like text. Multi-camera video, depth, sensor streams, actions and annotations, all time aligned, arriving continuously, and useful only when a researcher can pull out the exact few seconds they care about from an enormous corpus without scanning all of it. You own that half of the platform: ingest, storage layout, indexing, versioning, lineage and delivery. The systems that take raw capture and make it durable, queryable, reproducible, and fast enough to read that a distributed training run never waits on it. This is the layer that quietly decides whether anything above it is trustworthy. If a dataset cannot be reconstructed exactly six months later, no result built on it means very much. It is a founding seat, and the architecture is yours to set.

Responsibilities

• Build the ingest path for high-volume multimodal capture arriving continuously from many sources: validation, deduplication, time alignment, and backpressure that holds when a partner sends a month of data in a day.
• Design the storage layer. Formats, partitioning, compression and tiering at petabyte scale, treating read throughput and storage cost as equally real constraints.
• Make the corpus findable. Frame-level indexing and metadata so researchers query by what is in the data rather than reading through it.
• Own versioning and lineage. Every delivered dataset is reproducible, and every sample traceable back through its processing history to the moment it was captured.
• Keep the GPUs fed. Data loading and streaming that saturate large distributed training and evaluation jobs instead of starving them.
• Own the video path end to end: codecs, containers, hardware decode, frame-accurate seeking, so that decode stops being the thing everyone works around.
• Defend correctness. Schema evolution, validation, and automated checks that catch calibration drift, synchronisation error and bad samples before anyone downstream does.
• Deliver to partners in the formats and conventions their training stacks already expect, rather than asking them to adapt to ours.

Requirements

• 3+ years building data infrastructure in production, with strong Python and comfort in a systems language where it matters.
• You have owned a large-scale data platform, and you can talk specifically about what broke at scale and what you changed because of it.
• Real depth in storage and data layout: object storage, columnar and binary formats, partitioning strategy, caching, and the cost model underneath all of it.
• Experience with a distributed processing framework such as Ray, Spark, or equivalent.
• Unusual seriousness about correctness and reproducibility. Silent data corruption is the failure mode that costs the most and announces itself the least.
• Comfortable in a problem space where the naive approach simply does not run, and where the interesting engineering is in making it run anyway.

Skills

• Video or sensor data at scale: codecs, hardware decode, frame indexing, multi-stream synchronisation.
• Robotics or autonomous vehicle data formats and the realities of multi-sensor time alignment.
• Dataset versioning, catalog or lineage tooling, built or operated.
• Ray Data, Arrow, Parquet, Lance, or comparable modern data stacks.
• Owning the data loading path for large distributed training runs.
• Open-source contributions to a data, storage, or media processing stack.