Software Developer in Test

Responsibilities

• Build and implement robust automated test suites for API and backend services.
• Design and implement test strategies for AI/ML models and AI enabled in-house applications, including validation of model outputs, prompt behavior, determinism, and regression across releases.
• Establish automated evaluation frameworks for AI systems (i.e. accuracy, consistency, hallucination detection, bias, and drift) and integrate them into CI/CD pipelines.
• Partner with AI/ML Engineers, Platform teams, and Governance stakeholders to ensure responsible AI practices, including testability, traceability, and production readiness of AI features.
• Integrate automated test suites into existing GitLab CI/CD pipelines and deployment workflows.
• Maintain and improve automation frameworks to accommodate evolving system architecture, including blockchain based components.
• Collaborate with developers, product managers, and DevOps engineers to ensure testability and quality across distributed systems.
• Create and manage defect workflows integrated with tools such as JIRA, ensuring clear communication and traceability.
• Define, track, and publish meaningful QA metrics to support automated test development, execution, and overall quality reporting.
• Build and maintain performance and load testing suites for critical backend services and smart contract interactions where applicable.
• Perform all types of testing as needed like Functional, API, Automation, Performance, and Integration testing.
• Contribute to continuous improvement of QA processes, tooling, and best practices within an Agile environment.

Qualifications

• 10+ years of experience with QA engineering delivery experience with strong focus on backend or API testing
• 6+ years of delivery within Agile SDLC teams ideally with CI/CD
• Experience testing AI/ML or LLM-powered systems, including model validation, prompt testing, regression testing of AI behavior, and evaluation of non-deterministic outputs
• Familiar with AI testing techniques and metrics such as precision and recall, drift detection, bias assessment, and offline vs online evaluation approaches
• Working knowledge of AI-enabled development tools or platforms (i.e. AI coding assistants, test-generation tools, or internal AI platforms) and their impact on SDLC and quality governance
• Strong experience testing REST APIs, backend microservices, and distributed systems
• Experience working with blockchain technologies (i.e. smart contracts, distributed ledgers, blockchain node interaction) is preferred
• Solid understanding of CI/CD tooling and pipelines, especially GitLab CI/CD
• Experience with Python, Behave, PyTest, Postman/REST Tools, JMeter or similar performance tools
• Experience with Gitlab, Git, Jenkins, and Docker
• AI Integration experience is preferred
• Exposure to banking or finance environments is preferred