AI Inference Engineer - Speech

About the role

We are looking for an AI Inference Engineer with a solid background in speech recognition and model inference. In this role, you will develop state-of-the-art automatic speech recognition systems and ship them to various Zoom products. You will work on the most cutting-edge speech modeling and inference technologies with world-class speech scientists, collaborating with cross-functional teams including product, science engineering, and infrastructure teams to deliver high-impact projects from the ground up.

About the team

Zoom’s AI Speech Team is developing speech recognition technologies to improve Zoom’s conversational AI experience across products such as Zoom AI Companion, Zoom Meetings and Workplace, Zoom Contact Center, Zoom Phone, and Zoom Revenue Accelerator. As an AI Inference Engineer, you will develop novel speech model inference solutions on modern AI inference hardware, including GPUs, TPUs, and AI-specific chips, with the goal of delivering the most unique AI-powered collaboration platform to users globally.

Responsibilities

• Develop state-of-the-art speech services for Zoom products.
• Devise novel techniques where off-the-shelf solutions are not available.
• Optimize ASR inference systems for production deployment, including inference latency, throughput, memory footprint, and resource utilization.
• Optimize model inference performance by diving deep into the lower stack of inference frameworks, with a focus on hardware-specific optimizations for Nvidia GPUs.
• Propose new model structures by joint optimization of model accuracy and inference speed.
• Design and develop ASR systems with low latency and high accuracy requirements, while ensuring scalability of GPU infrastructure and improving throughput of ASR service.
• Profile and debug ASR runtime performance bottlenecks across different deployment hardware and environments.

Requirements

• Master’s degree in Computer Science, Electrical Engineering, or a related field.
• 3+ years of experience in speech recognition, speech-LLM, or AI model inference.
• Deep learning knowledge and hands-on programming skills in Python, shell scripts, and C/C++.
• Familiarity with ML frameworks such as PyTorch and TensorFlow.
• Deep understanding of transformer encoder-decoder frameworks for speech recognition, including attention mechanisms, beam search, and sequence-to-sequence modeling for end-to-end ASR systems.
• Understanding of recent advancements in speech foundation models and speech-LLMs that integrate acoustic and linguistic representations for unified modeling of speech understanding and transcription tasks.

Qualifications

• Experience optimizing deep learning model inference on NVIDIA GPUs, including profiling and accelerating AI models using CUDA, TensorRT, and mixed-precision computation to achieve low latency and high-throughput performance.
• Experience developing and tuning custom CUDA kernels, leveraging CUDA Graphs for efficient execution scheduling, and minimizing kernel launch overhead to maximize GPU utilization.
• Proficiency in end-to-end performance analysis, memory optimization, and deployment of large-scale ML models on GPU clusters.
• Experience with stream management, asynchronous execution, and integrating frameworks such as PyTorch and TensorFlow for real-time inference.

Pay

Minimum Salary Range or On Target Earnings: $151,800.00 – $332,200.00. Starting pay will be based on a number of factors and commensurate with qualifications and experience. Zoom has a Total Direct Compensation philosophy that takes into consideration base salary, bonus, and equity value. There may be a different range for candidates based on location.