Infrastructure Engineer
Overview
The AI Infrastructure Engineer at HOAi is responsible for scaling and maintaining the infrastructure that powers our AI-driven products and services. This role sits at the intersection of infrastructure engineering, machine learning operations, and product development, ensuring our AI systems operate with exceptional reliability, performance, and efficiency.
Responsibilities
Profile and optimize database queries, API endpoints, and ML inference pipelines
Implement caching strategies, connection pooling, and distributed systems for scale
Monitor and optimize GPU utilization, memory usage, and compute costs
Design load balancing and auto-scaling policies for variable AI workloads
Build disaster recovery systems with redundancy
Build and maintain CI/CD pipelines specifically for model deployment
Create automated testing frameworks for model quality and performance regression
Develop infrastructure for model monitoring, drift detection, and retraining workflows
Manage experiment tracking and model registry systems
Implement comprehensive monitoring, logging, and alerting across the AI stack
Refine dashboards for real-time visibility into system health and performance
Conduct post-mortems and implement reliability improvements
Design circuit breakers, retry logic, and graceful degradation for critical services
Requirements
8+ years of experience in infrastructure engineering, DevOps, or SRE
Strong cloud platform expertise
Experience building and maintaining deployment pipelines
Experience with PostgreSQL, Redis, or other production databases
Familiarity with APM tools, metrics, logging, and alerting
Familiarity with vector databases, model serving frameworks, and cross-system observability and traceability
Real-time inference with low-latency serving infrastructure
LLM deployment track record of achieving 10x performance improvements
Skills & Competencies
Able to debug complex distributed systems and find root causes
Obsessed with latency, throughput, and resource efficiency
Defaults to automating repetitive tasks and building scalable solutions
Understands security implications and implements best practices
Able to explain complex technical concepts clearly
Works effectively across teams and functions
Takes initiative to identify and solve problems before they become critical
Comfortable with ambiguity and changing priorities in a fast-moving startup
Core Values
Always Growing: Likes change and enjoys finding new ways to improve their knowledge and the product.
Win as a Team: Builds trust and works together by making sure everyone communicates well. Actively involved in daily work, working closely with the team, listening to their ideas, and celebrating successes together.
Accountability Starts with Me: Notices problems and takes personal action to solve them.
Innovate Boldly: We challenge the status quo and push boundaries to create meaningful change. We act with urgency and purpose, knowing that innovation drives our success.