Principal AI Engineering Architect

About the role

The Principal AI Engineering Architect leads the design and delivery of complex, multi-domain systems spanning cloud, data, and AI, with a focus on multi-agent agentic AI solutions. This role requires deep, hands-on mastery of these technologies and a proven track record in leading technical strategy and delivering production-grade systems.

Responsibilities

• Define technical strategy and lead architectural design across cloud, data, and AI/ML systems for end-to-end engagements, owning architecture decisions and driving solutions from research through production at scale
• Architect and ship production-grade multi-agent agentic AI systems, including agent orchestration, tool use, memory, and inter-agent communication patterns
• Partner closely with leadership and clients on technical strategy, lead the design and delivery of complex, production-grade multi-agent systems, mentor experienced engineers, and own high-stakes decisions that shape the long-term success of the systems you build
• Build and evolve scalable ML platforms, pipelines, and infrastructure that support reliable, repeatable model development and deployment across teams
• Drive performance, scalability, cost, and reliability optimization across deployed systems
• Ensure architecture meets security, governance, and compliance requirements (e.g., GDPR, HIPAA, SOC2)
• Lead cloud migrations and platform modernization initiatives
• Define infrastructure as code, CI/CD, and DevOps standards across engagements (e.g., Terraform, CloudFormation, GitHub Actions)

Requirements

• 8+ years of software engineering experience, with at least 5 years in technical leadership roles and 4+ years focused on AI/ML systems in production
• Expert software engineering background (Python or similar) with strong design sensibilities for scalable, maintainable systems
• Deep, hands-on expertise designing and shipping production multi-agent agentic AI systems, including agent orchestration, planning, tool use, and multi-agent coordination patterns
• Strong background in microservices, serverless, containers, and event-driven systems (e.g., Kubernetes, Docker, Lambda, EventBridge)
• Proficiency with infrastructure as code and CI/CD (e.g., Terraform, CloudFormation, Pulumi, GitHub Actions)
• Strong data architecture expertise across relational, NoSQL, and big data systems (e.g., PostgreSQL, MongoDB, Snowflake, BigQuery, Spark, Kafka)
• Hands-on experience with data modeling, ETL/ELT pipelines, and orchestration (e.g., Airflow, Prefect, dbt)
• Mastery of AI frameworks and orchestration tools for building agentic systems (e.g., LangChain, LangGraph, AgentCore, CrewAI, AutoGen, or equivalents)
• Strong experience designing AI/ML systems for production, including LLMs, MLOps, and model serving (e.g., SageMaker, Bedrock, Vertex AI, MLflow, Hugging Face, PyTorch, TensorFlow)
• Strong experience with evaluation frameworks and observability tools for LLM and agentic apps, including building these capabilities where they don't yet exist
• Deep understanding of AI safety, responsible AI principles, prompt injection defenses, and PII handling
• Extensive experience building RAG pipelines: chunking strategies, embedding models, vector databases, and advanced retrieval techniques
• API design experience, including architecting and integrating with internal and third-party services at scale
• Solid understanding of networking, security, identity, and access management in cloud environments
• Experience with governance, compliance, and observability frameworks
• Track record of senior technical leadership and mentoring experienced engineers
• Strong stakeholder communication skills, with the ability to translate technical depth across audiences
• Demonstrable, day-to-day usage and expert knowledge of AI-forward coding tools such as Claude Code and Cursor

Qualifications

• Expertise with AWS, including in-depth knowledge of AWS GenAI offerings and hands-on experience with Amazon Bedrock AgentCore; broader multi-cloud experience (Azure, GCP) is a plus
• Advanced cost optimization expertise: token economics, caching strategies, model routing, quantization
• Solid understanding of networking, security, identity, and access management in cloud environments
• Experience with governance, compliance, and observability frameworks
• Track record of senior technical leadership and mentoring experienced engineers
• Strong stakeholder communication skills, with the ability to translate technical depth across audiences
• Demonstrable, day-to-day usage and expert knowledge of AI-forward coding tools such as Claude Code and Cursor

Skills

• Expert software engineering background (Python or similar)
• Microservices, serverless, containers, and event-driven systems (e.g., Kubernetes, Docker, Lambda, EventBridge)
• Infrastructure as code and CI/CD (e.g., Terraform, CloudFormation, Pulumi, GitHub Actions)
• Data modeling, ETL/ELT pipelines, and orchestration (e.g., Airflow, Prefect, dbt)
• AI frameworks and orchestration tools for building agentic systems (e.g., LangChain, LangGraph, AgentCore, CrewAI, AutoGen, or equivalents)
• AI/ML systems for production, including LLMs, MLOps, and model serving (e.g., SageMaker, Bedrock, Vertex AI, MLflow, Hugging Face, PyTorch, TensorFlow)
• Evaluation frameworks and observability tools for LLM and agentic apps
• AI safety, responsible AI principles, prompt injection defenses, and PII handling
• RAG pipelines: chunking strategies, embedding models, vector databases, and advanced retrieval techniques
• API design experience, including architecting and integrating with internal and third-party services at scale
• Networking, security, identity, and access management in cloud environments
• Governance, compliance, and observability frameworks
• AI-forward coding tools such as Claude Code and Cursor

Benefits

Robots & Pencils offers competitive compensation and benefits, including health insurance, retirement plans, and paid time off.

Pay

$180,375 – $230,625 USD

Schedule

Full-time