Perception Engineer

About LightSpeed LightSpeed Build Technologies is revolutionizing the construction industry through AI-powered robotics. Our flagship systems—BRUTE for automated wall panel manufacturing and DEX for on-site collaborative construction—are addressing the global housing crisis by delivering unprecedented speed, precision, and affordability in homebuilding. Position Overview As a Perception Engineer, you will own how LightSpeed's robots see. You will design, calibrate, and deploy the multi-camera systems on our robot cells and end-of-arm tooling, and build the perception that guides manipulation in real time. Your camera streams also feed our teleoperation and robot learning pipelines, so the quality of what you build directly shapes how well our robots learn. This role sits at the intersection of sensor hardware, 3D vision, and deep learning, all deployed on physical robots where reliability matters. What You'll Work On Camera & Sensor Systems Select, specify, and deploy multi-camera systems (RGB, stereo, depth) for robot cells, including wrist- and tool-mounted cameras on end-of-arm tooling and dexterous handsOwn intrinsic, extrinsic, and hand-eye calibration, and keep calibration reliable across tool changes and daily operationSynchronize cameras with robot state and teleoperation streamsWork with mechanical engineers on camera placement for coverage, occlusion, and lighting Perception Development Estimate and track robot and end-effector pose using vision systems to verify kinematics and identify driftCompare 3D scans against CAD or BIM models to check part placement and as-built accuracyLocalize robots and tools in the workspace using fiducials (AprilTag or ChArUco) and multi-view fusionBuild object detection, segmentation, and 6-DoF pose estimation for parts, fasteners, and tools in cluttered, confined, and poorly lit workspacesBuild point cloud pipelines (filtering, segmentation, registration such as ICP, and meshing) for manipulation and scene understandingTrack hands and objects in egocentric video of human demonstrationsBuild CUDA-optimized perception pipelines: GPU preprocessing, custom CUDA kernels where needed, TensorRT inference, and minimal host–device copiesProfile and optimize end-to-end latency, from photon to action, using tools like Nsight Systems Data & Robot Learning Support Deliver calibrated, time-synchronized camera data in the formats our teleoperation recording and training pipelines consumePartner with AI/ML engineers to define observation spaces for imitation learning and vision-language-action policiesSet perception accuracy targets tied to task success, and build benchmarks to measure them Integration & Deployment Integrate perception with ROS2-based robot controlHarden systems for real-world conditions: lighting changes, reflective surfaces, occlusion, and vibrationCollaborate closely with robotics, mechanical, and controls engineers Required Qualifications Perception Expertise 3+ years building and deploying perception systems on physical robots or real-time systemsHands-on experience calibrating multi-camera systems (intrinsic, extrinsic, hand-eye) and synchronizing sensorsExperience with RGB-D and stereo cameras in real deploymentsStrong computer vision fundamentals: camera geometry, multi-view geometry, and 3D reconstructionDeep learning experience with detection, segmentation, or pose estimation using PyTorch Software Engineering Strong Python and C++Experience with OpenCV and point cloud libraries such as Open3D or PCLExperience with ROS or ROS2Proficiency with Linux, Docker, Git, and CI/CD workflows Preferred Qualifications MS or PhD in Computer Vision, Robotics, Computer Science, or a related fieldPerception for robotic manipulation, especially dexterous or tool-based manipulationExperience with hardware video encoding (NVENC, Jetson Multimedia API), GStreamer, or WebRTCExperience with NVIDIA Isaac ROS / NITROS, VPI, or CV-CUDAExperience with point cloud registration and scan-to-CAD/BIM comparisonExperience with vision foundation models (for example SAM, DINOv2, FoundationPose)Experience with egocentric video or hand pose estimationExperience producing sensor data for robot learning datasets (e.g., rosbag or MCAP)Edge deployment on NVIDIA Jetson with TensorRTSimulation and synthetic data generation with Isaac Sim, MuJoCo, or similarBackground in manufacturing, industrial automation, or construction technology Why Join LightSpeed Build the eyes of robots that are addressing the housing crisisOwn the full perception stack, from sensor selection to deployed modelsWork on rare real-world challenges: confined-space manipulation, dexterous hands, learning from human demonstrationHands-on culture with direct access to robots, sensors, and manufacturing environmentsCompetitive compensation and comprehensive benefits Employment Relationship This position is at-will, meaning either you or the Company may terminate employment at any time, with or without cause or notice. Equal Opportunity LightSpeed Build Technologies is an equal opportunity employer committed to building a diverse and inclusive workplace. We welcome candidates from all backgrounds and experiences. Compensation Range: $100K - $120K