Software quality engineer (US)
About the role
Define and implement comprehensive quality assurance strategies and test plans for our AI agents and LLM-powered applications, ensuring exceptional product reliability and performance.
Designing and developing automation frameworks: creating robust, scalable, and maintainable automated test frameworks from scratch or enhancing existing ones. You’ll need proficiency in at least one language like Typsecript, Python.
Collaborate closely with product managers, machine learning engineers, and data scientists to understand complex AI features and model behaviors, translating them into effective test cases and validation criteria.
Drive the continuous improvement of our testing processes and infrastructure, integrating automated checks within our CI/CD pipelines to ensure rapid, high-quality releases.
Identify, document, and track software defects and inconsistencies, performing root cause analysis to provide actionable feedback to development teams.
Monitor production systems and AI model performance, proactively identifying potential issues and contributing to post-release quality validation.
Champion quality best practices across engineering teams, fostering a culture of ownership and continuous improvement in delivering world-class AI solutions.
Designing, managing, and maintaining test data strategies and mock services to ensure stable, isolated, and repeatable test execution.
Experience designing, developing, or integrating agentic AI systems, AI skills, and the Model Context Protocol (MCP).
Manage the full defect lifecycle by analyzing customer feedback and debugging logs to identify, prioritize, and track software bugs, collaborating closely with development teams to ensure timely resolution.
Requirements
- 5+ years of hands-on experience in software quality assurance or engineering, with a strong focus on testing complex distributed systems or AI/ML applications.
- 5+ years of experience working with GitHub actions.
- Proficiency in programming languages like Python or Typescript for test automation, and experience with modern testing frameworks such as Playwright.
- Solid understanding of AI/ML concepts, including model evaluation metrics, data pipelines, and the unique challenges of testing generative AI outputs.
- Experience with cloud platforms (AWS, Azure, GCP) and containerization technologies (Docker, Kubernetes) in a CI/CD environment is a big plus.
- Exceptional analytical skills to dissect complex problems and a keen eye for detail, ensuring no bug goes unnoticed.
- A collaborative spirit that helps you Connect cross-functional teams to common quality goals, Challenge existing assumptions to build better systems, and Own the end-to-end quality of our products.
- Demonstrated ability to drive initiatives independently and thrive in a fast-paced, evolving startup environment.
Qualifications
- BS/MS in Computer Science, Engineering, or related field.
- Experience with agile methodologies and DevOps principles.
- Strong problem-solving and troubleshooting skills.
- Excellent communication and interpersonal skills.
Skills
- Programming Languages: Python, TypeScript
- Testing Frameworks: Playwright
- Cloud Platforms: AWS, Azure, GCP
- Containerization Technologies: Docker, Kubernetes
- Agentic AI Systems, AI Skills, Model Context Protocol (MCP)
Benefits & Perks
- Generous PTO, plus company holidays
- Medical, dental, and vision coverage for you and your family
- Paid parental leave for all parents (16 weeks)
- Fertility and family planning support
- Early-detection cancer testing through Galleri
- Flexible spending account and dependent FSA options
- Health savings account for eligible plans with company contribution
- Annual work-life stipends for: Wellness stipend for gym, massage/chiropractor, personal training, etc.
- Learning and development stipend
- Company-wide off-sites and team off-sites
- Competitive compensation, company stock options and 401k