Kubernetes Platforms System Engineer

Job Responsibilities

• Work with the team to define and implement best practices and standards within the organization
• Keep the Kubernetes platform reliable, available, and fast
• Architect solutions to problems that improve the reliability, scalability, performance, and efficiency of our services
• Respond to, investigate, and fix service issues all the way from bare metal through the OS to the application code
• Cook with vendors to resolve hardware and software problems
• Coordinate with users to help them use Kubernetes
• Participate in an on-call rotation providing 24-hour, 7-day support and off-hours maintenance windows

Basic Qualifications

• Bachelor’s degree in a scientific field and a minimum of 5-8 years of relevant experience.
• An equivalent combination of education and experience will be considered.
• Experience with Kubernetes as a cluster administrator for on-premises deployments
• Excellent interpersonal/communications skills, and the ability to work as part of a team
• Strong working knowledge of Linux systems fundamentals and networked computing environment concepts
• Experience with code reviews, code quality, CI/CD tooling, GitOps, SCM (e.g. GitLab)
• Able to identify requirements and to define, plan, and implement requisite solutions for small and medium projects
• Ability to develop and maintain programs and scripts that aid in the operation and automation of tasks using various shell and scripting languages (primarily bash, Python, and Go)
• Able to obtain and maintain a Department of Energy "Q" clearance is required. This requires US Citizenship.

Preferred Qualifications

• Bachelor’s degree in a scientific field and 8-10 years of relevant experience.
• Subject matter expert in Kubernetes as a cluster administrator for bare metal, on-premises deployments
• Excellent interpersonal/communications skills, be able to effectively communicate with other teams and organizational leadership. Convey technical details to a non-or semi-technical audience.
• Able to identify requirements and to define, plan, and implement requisite solutions for large, organizationally impactful projects
• Self-driven with the ability to work in a dynamic, loosely structured research & development environment
• Experience with RKE2 (nice to haves: Red Hat OpenShift and Talos).
• Multi-cluster management tools for Kubernetes (e.g. Fleet), and container security tools (Neuvector, SCC, pod admission control)
• Experience with managing image registries such as Quay or Harbor
• Experience using tools such as Prometheus, Nagios, and Grafana to monitor systems, metrics and create dashboards
• Experience designing and implementing highly-available systems/services
• Experience with Infrastructure-as-Code tooling such as Terraform, Helm, and Puppet
• Experience implementing systems-level security technologies (e.g. SELinux, Seccomp, linux capabilities), experience with DevSecOps, and general security best practices
• Experience with AIOps and MLOps tooling – e.g. KServe, Kubeflow, vLLM, NVidia Enterprise AI, AMD Silo AI, ClearML, MLFlow
• Experience using HPC hardware for Kubernetes – e.g. RDMA, DPUs, Infiniband, many-core CPUs
• Experience with declarative CI/CD tools such as ArgoCD
• Experience with workflow engines such as Apache Airflow or Argo Workflows
• Experience with infrastructure automation
• Experience with Cloud engineering experience with at least one cloud service provider
• Experience with reusable, automated workflows such as PagerDuty playbooks