QA Engineer
DataServ · North Andover, MA · 1 wk ago
On-siteEngineeringFull-time
Qualifications
- Team player
- Excellent written and verbal communicator
- Tech savvy
- Strategic thinker
- Service-oriented
- Motivated self-disciplined professional
- Flexible in a fast-paced environment
- Works independently with minimal supervision
- Exhibits strong work ethics
- Detail-oriented
- Handles demanding situations/timelines
- Organized
- Responsible/dependable
- Willing to receive/provide feedback to improve performance
- Supportive
- Strong desire to support and mentor
What You’ll Do
- Quality Assurance & Test Automation
- Test Planning & Execution
- Design, develop, and execute manual and automated test plans and test cases for enterprise features across web, mobile, and backend systems
- Automation Development
- Develop and maintain automated test scripts (e.g., Selenium, Playwright, Cypress) using a scripting/programming language such as Python or JavaScript to increase efficiency and coverage
- Regression & Release Testing
- Run regression suites before releases to ensure existing functionality remains stable as new features and models are deployed
- Defect Management
- Identify, document, prioritize, and track defects using tools like Jira, and collaborate with developers to drive timely resolution
- CI/CD & Process Improvement
- Integrate tests into CI/CD pipelines (e.g. GitLab CI) and participate in continuous improvement of QA processes, tools, and methodologies
- AI & LLM Evaluation
- Evaluation Planning
- Partner with Product Managers and Data Scientists to define scope, metrics, and goals for evaluating AI and LLM-powered features (e.g., accuracy, robustness, fairness, hallucination rate, safety)
- Data Curation & Preparation
- Coordinate collection, labeling, and preparation of datasets used for AI model testing and human-in-the-loop evaluations
- Evaluation Execution & Coordination
- Help organize evaluation sprints, red-teaming sessions, and feedback loops, acting as a point of contact between AI/ML, Product, QA, and external evaluators (if applicable)
- Results Analysis & Reporting
- Compile and summarize evaluation results, providing clear, actionable feedback on model performance and readiness for production deployment
- Documentation & Traceability
- Maintain detailed documentation of evaluation criteria, test data, prompts, and results, ensuring traceability and auditability for AI/LLM releases
- AI For QA (Tooling & Enablement)
- Use LLMs and AI tools (e.g., ChatGPT, Copilot, AI testing platforms) to generate test ideas, draft test cases, and assist with log/defect analysis, then validate and refine these outputs
- Contribute to best practices and guardrails for using AI safely and effectively in QA and evaluation workflows
Education
- Bachelor’s degree in computer science, or related professional experience in Quality Assurance
- Experience 2+ years in Software Quality Assurance or a similar role
- Proficiency with at least one scripting/programming language (e.g., Python, JavaScript/TypeScript, or Java)
- Familiarity with test management tools (e.g., TestRail or similar)
- Defect tracking systems (e.g., Jira)
- Version control (e.g., Git)