Software Engineer - LLM Infrastructure

About the role

Swoop Technologies is dedicated to organizing and making accessible the world’s military and critical infrastructure. We are developing SwoopOS, a distributed operating system that breaks down equipment into a network of robotic components, enabling the creation of new systems, autonomous entities, and artificial intelligence through our SDK, Valhalla, and browser, Surf. Our mission is to unlock the potential of global assets like the electrical grid, communication networks, manufacturing plants, and militaries, turning them into modular components that can be rapidly integrated and optimized through software.

Responsibilities

• Develop and maintain Swoop’s LLM offering
• Expand the capabilities of the LLM to interact with the system via tool calls
• Expand the data searching capabilities of the LLM
• Work hand-in-hand with frontend developers to build out new LLM features and improve existing ones
• Maintain and optimize inference engine architecture
• Tune data storage configurations to optimize for scale and near real-time availability in a streaming architecture
• Ensure strong availability and service level agreements across the codebase, particularly concerning the runtime of our Kubernetes cluster in production

Requirements

• Bachelor's degree in Computer Science or related technical field, or equivalent technical experience
• Firm understanding of scalable large language model infrastructure
• Experience with low-level NVIDIA drivers and NVIDIA Kubernetes Container Toolkit
• Familiarity with designing RAG information retrieval systems and time-series anomaly detection
• Experience with PyTorch, training and fine-tuning Machine Learning models for resource-light environments
• Experience with Kubernetes in a production environment
• Proficient Python coding ability with good understanding of data structures and data models
• Active US Security clearance or ability and willingness to be sponsored for a US Security clearance

Qualifications

• Experience with on-premise or self-hosted AI
• Experience with numerous GPUs and understanding of performance characteristics
• Experience standing up inference engines such as vLLM

Skills

• Scalable large language model infrastructure
• Low-level NVIDIA drivers and NVIDIA Kubernetes Container Toolkit
• RAG information retrieval systems and time-series anomaly detection
• PyTorch, training and fine-tuning Machine Learning models
• Kubernetes in a production environment
• Python coding ability with data structures and data models
• On-premise or self-hosted AI experience
• Numerous GPUs and performance characteristics
• Inference engines such as vLLM

Benefits

• Equal opportunity employer

Pay

• Competitive salary commensurate with experience

Schedule

• Hybrid position requiring in-office 3+ days per week