Senior ML Infrastructure Engineer

Role Description

As the first and founding ML Operations Engineer at Tennr, you’ll play a crucial role in building and iterating on foundational Machine Learning and AI systems. You’ll own building machine learning training and inference pipelines that can handle increasing traffic demands and proliferation of product surface as we grow. You will be critical in ensuring our AI-driven healthcare platform is powered by robust, scalable, and efficiently deployed models.

Candidate Qualifications

• 5+ years of experience in ML model deployment, infrastructure, and scaling in production environments
• Strong software engineering fundamentals, with proficiency in Python and TypeScript
• Experience in software design and architecture for highly available ML systems for use cases like inference, evaluation, and experimentation
• Strong knowledge of observability, including logging, metrics, tracing, model performance monitoring, and alerting
• Experience with distributed systems, reliability, and production incident response
• Comfortable working in ambiguity with high ownership, moving quickly in a fast-paced startup environment, and proactively driving projects from idea to production

Nice to have

• Experience working with ML CI/CD and common ML frameworks like Pytorch, Tensorflow, etc.
• Experience working with common inference frameworks like vLLM, TensorRT, Triton, etc
• Experience with GPU orchestration, including managing GPU workloads/scheduling, cost management, cluster utilization, etc
• Experience with GPU optimization (training/inference) involving CUDA profiling, memory optimization, multi-GPU communication, etc