Senior ML Infrastructure Engineer
Project Overview
We're building a large-scale document intelligence platform that processes text files up to 5 TB in size, extracts insights using BERT-class NLP models, and surfaces answers to analysts via a low-latency query interface. The platform runs on Azure Kubernetes Service (AKS) with dedicated GPU node pools, uses KEDA for event-driven autoscaling, and integrates with Azure Data Lake Storage Gen2 and Azure OpenAI.
Skill / Technology
- Level: Kubernetes / AKSExpert: Multi-node-pool design, taint/toleration, autoscaler, GPU node pools (NC/ND series)
- Senior: Device plugin, driver compat, resource limits, KEDA
- Senior: Scaled Job, queue triggers, cooldown tuning, CUDA / cuDNN
- Mid–Senior: Runtime config via PyTorch; raw kernel dev not required, PyTorch
- Senior: Batching, FP16, memory management, profiling, Hugging Face Transformers
- Senior: Async workers, Azure SDK, queue consumers, Azure infrastructure
- Senior: VNet, private endpoints, Key Vault, ADLS, AD, Docker / Helm
- Senior: Multi-stage builds, Helm chart authoring, IaC (Terraform / Bicep)
- Preferred: willingness to learn is acceptable
About the Role
This is a hands-on role that sits at the intersection of platform engineering and applied ML, and requires someone who is equally comfortable debugging a CUDA out-of-memory error and designing a Kubernetes autoscaling policy. As the Senior ML Infrastructure Engineer the resource will own the end-to-end infrastructure layer — from GPU cluster configuration and CUDA runtime management to Kubernetes job orchestration and model serving.
Qualifications
Orion is an equal opportunity employer, and all qualified applicants will receive consideration for employment without regard to race, color, creed, religion, sex, sexual orientation, gender identity or expression, pregnancy, age, national origin, citizenship status, disability status, genetic information, protected veteran status, or any other characteristic protected by law.