Automation Engineer

About the role

We are looking for an Automation Engineer to own, extend, and scale our automated testing frameworks that validate the runtime behavior of code generated by our AI platform. This role is critical in ensuring that our platform’s output performs correctly in production environments.

Responsibilities

• Expand and maintain automated testing frameworks that validate the runtime behavior and performance of AI-generated applications.
• Develop intelligent test suites that automatically assess application functionality in production environments.
• Extend existing testing frameworks to increase coverage and validate runtime behavior across multiple languages and frameworks.
• Establish quality metrics and testing protocols that measure real-world performance of generated applications.
• Build systems to automatically detect and flag runtime issues in deployed applications.
• Collaborate with our AI team to improve the platform based on runtime performance data.
• Implement automated integration and end-to-end testing that ensures generated applications work as intended in production.
• Develop metrics and monitoring systems to track runtime performance across different customer deployments.

Areas of Ownership

• Own the evolution and expansion of our automated testing frameworks for AI-generated code, increasing coverage and reliability.
• Drive and expand end-to-end and integration testing coverage across production environments.
• Drive runtime monitoring and quality assurance standards across multi-language deployments.
• Manage CI/CD pipeline integration for continuous quality validation.
• Collaborate cross-functionally with AI engineers to surface and resolve platform-level quality issues.

Requirements

• 3+ years of experience with Playwright and automated testing frameworks.
• Strong expertise in Python (our primary automation language).
• Experience with CI/CD tools (GitHub Actions or similar).
• Proficiency in version control systems (Git).
• Experience testing distributed systems.
• Understanding of modern software development practices.
• Experience working with cloud platforms (GCP preferred).

What Makes You Stand Out

• Experience with runtime monitoring and testing of distributed systems.
• Knowledge of performance testing and APM (Application Performance Monitoring).
• Experience with end-to-end testing of complex, enterprise-grade applications.
• Understanding of distributed tracing and monitoring.
• Experience with chaos engineering and resilience testing.

What Makes This Role Different

This isn't a traditional QA role. You'll be at the forefront of a category-defining company, building testing infrastructure for AI-generated software that is transforming how Fortune 500 enterprises build products. You will have direct impact on the quality and reliability of an autonomous development platform moving at unprecedented speed—where your frameworks determine whether AI-built software is production-ready at scale.

Culture

• We move Blitzy Fast: Time is both our company's and our clients' most precious asset. We move quickly and decisively to innovate internally and deliver exceptional software externally.
• We operate like a professional sports team. We win as a team by holding ourselves and each other to high standards, collaborating in-person, and remaining focused on the mission.
• We're pushing the frontier of what's possible, requiring constant innovation and iteration.
• We focus on delivering outsized value to the customers we work with and expanding those relationships into deep, meaningful partnerships.
• We believe in being 'everyday athletes'—taking care of ourselves so we can bring our best minds to work. We promote great sleep, movement, and restorative activities for optimal mental performance.
• We are an equal opportunity employer committed to building a diverse and inclusive team. We believe different perspectives make us stronger.