Senior QA Analyst / Tester

About the role

Orion Innovation is a premier, award-winning, global business and technology services firm. Orion delivers game-changing business transformation and product development rooted in digital strategy, experience design, and engineering. The Red Team Tester will conduct testing of AI models to find vulnerabilities, develop new evaluation frameworks, communicate risks to stakeholders across global teams, and collaborate with defensive and development teams.

Responsibilities

• Design and execute comprehensive adversarial testing campaigns against AI models, including large language models, multimodal systems, and autonomous agents.
• Research attack vectors and prompt injection techniques to identify model vulnerabilities, jailbreaks, and unintended behaviors.
• Evaluate and introduce AI Security tools that decrease mean time to detect and respond to AI-specific threats.
• Conduct red team exercises simulating real-world deployment scenarios and edge cases. Systematically probe for bias, toxicity, misinformation generation, and other harmful outputs across diverse contexts and demographics.
• Create adversarial datasets and benchmarks to evaluate model robustness under various attack conditions.
• Collaborate with security teams (e.g., GSOC, member firm security teams, developers) to perform red teaming activities, document and present findings, vulnerabilities, and remediation recommendations to drive mitigation of identified risks.
• Develop and maintain process documentation, create reports on testing activities, track operational metrics, and perform other programmatic tasks to support the AI Red Team's function.

Requirements

• Minimum 5 years of penetration testing, red team operations, or AI Red Team experience.
• Computer science, information technology, programming, development, Artificial Intelligence, or cybersecurity degree (Bachelor's or higher preferred) from an accredited college or university or equivalent work experience.
• Background in AI red teaming, red teaming, or offensive cyber operations.
• Understanding of Artificial Intelligence, Machine Learning, software applications, cloud computing, and networking.
• Excellent communication and stakeholder management skills.
• Ability to work effectively across geographies and time zones.

Skills

• Testing tools: SPLX, Garak, TextAttack, PyRIT, Burp Suite, Metasploit, Nessus, Cobalt Strike/Mythic/C2, NMAP, sqlmap, or similar tools
• Web application technologies and protocols: HTTP, DNS, FTP
• Technical knowledge of: Generative AI, Agentic AI, A2A, and/or MCP
• Programming languages: Python, Ruby, Go, C++, PowerShell, bash or similar
• Certifications (desired): OSCP, GPEN, GPXN, AI Red Teamer (HackTheBox), AISEC+, GASAE, GOAA, OSAI+, or other industry AI or Red Team related certifications