HPC Systems Engineer

Responsibilities

• Applies advanced systems / infrastructure concepts to define, design, implement, and operate highly complex, research cyberinfrastructure systems, services and technology solutions.
• Proposes and implements highly complex system or device enhancements such as software, hardware and network configuration, updates and installations for projects or services of broad scope.
• Sets standards for monitoring and maintaining the health and integrity of CI systems including upgrading and patching.
• Independently manages systems and services for a large facility, campuswide, medical center or Office of the President and / or institution-wide scope and makes recommendations for purchases or upgrades.
• Performs complex and advanced analysis to acquire, install, modify and support operating systems, databases, utilities and web-related tools.
• Selects methods and techniques to obtain solutions. Interacts with senior management.
• Performs complex network integration tasks and interoperability assessments for interconnected servers or components of clusters for communication.
• Support and collaborate with researchers and other key IT (e.g. network and security) and Data Center partners in a timely manner.
• Specifies, writes and executes highly complex software and scripts to support systems management, log analysis, monitoring, deployment, configuration management, and other system administration duties for multiple, highly integrated systems.
• Provides consultation, training, support, and guidance to researchers enabling them to utilize HPC resources effectively.
• Maintains complex security systems. Interprets and adopts campus, medical center or Office of the President, system and regulation-based security policies to control access to networked resources. Provides recommendations and requirements on network access controls.
• Collaborates and may provide leadership with other Systems Engineers within the CI ecosystem/higher-education community.
• Regularly contribute best practices documentation, present at conferences, or publish in peer reviewed journals.
• Define and track performance metrics to ensure efficient current and future use of cyber infrastructure resources.

Qualifications

• Bachelor's degree in a related area such as computer science or engineering, and 6+ years of experience with large-scale or HPC systems * or* 10+ years of related experience with large-scale or HPC systems.
• Expert knowledge of HPC systems infrastructure design.
• Advanced knowledge of computer security best practices and policies including demonstrated experience securing research cyberinfrastructure systems to meet NIST 800-171 / 800-223, HIPPA or IS-3 requirements.
• Demonstrated testing and test planning skills. Demonstrated ability to create automated testing.
• Knowledge of HPC job scheduler system design and operation such as SLURM or PBS, Demonstrated skill (5 years +) deploying, managing, and troubleshooting Warewulf (or similar) infiniband based clusters.
• Ability to elicit and communicate technical and non-technical information in a clear and concise manner.
• Self-motivated and works independently and as part of a team. Demonstrates problem-solving skills.
• Able to learn effectively and meet deadlines.
• Demonstrated advanced knowledge, skills and abilities associated with system problem identification and resolution.
• Experience with design, configuration, operation, repair, and tuning of technology systems.
• Advanced experience writing and editing the most complex scripts used to perform system maintenance and administration.
• Ability to write technical documentation in a clear and concise manner. Ability to develop runbooks defining complex technical processes in a clear and concise manner.