Senior Specialist - Quality Engineering

Job Summary

We are seeking an experienced AI Quality Engineer to validate and assure the quality of Generative AI applications, RAG systems, AI agents, APIs, and data pipelines. The ideal candidate will have expertise in test automation, AI evaluation frameworks, prompt testing, hallucination detection, and performance validation to ensure AI solutions are accurate, reliable, safe, and production-ready.

Key Responsibilities

• Design and implement automated testing frameworks for AI GenAI applications.

• Validate LLM, RAG, and Agent-based systems for functionality, accuracy, reliability, and safety.

• Develop and execute prompt testing, retrieval testing, and response quality validation.

• Perform hallucination testing, guardrail validation, and adversarial security testing.

• Build automated API and integration test suites for AI services and data pipelines.

• Establish AI evaluation metrics and quality benchmarks using tools such as LangSmith and Galileo.

• Conduct regression, performance, load, and scalability testing for AI solutions.

• Validate data quality, retrieval accuracy, and end-to-end RAG workflows.

• Collaborate with AI engineers, data engineers, and platform teams to identify and resolve quality issues.

• Integrate test automation into CI/CD pipelines and support continuous quality monitoring.

Required Skills

• Strong proficiency in Python

• Test Automation Framework Development

• API Testing (REST APIs, Microservices)

• AI Evaluation Frameworks (LangSmith, Galileo)

• Prompt Testing Validation

• RAG Testing Retrieval Validation

• Hallucination Detection Response Evaluation

• Guardrail Validation

• Responsible AI Testing

• Regression Testing

• Performance Load Testing

• SQL and Database Testing

• Git, Jenkins, CI/CD Pipelines

Preferred Skills

• MongoDB and Oracle Databases

• LangGraph AI Agent Testing

• Linux and Shell Scripting

• OpenShift Container Platforms

• Security Adversarial Testing

• Knowledge of Vector Databases and Embedding Validation

• Experience with AI observability and monitoring tools

Other Details

Actual compensation within the range will be dependent upon the individual's skills, experience, performance, and internal equity. Benefits/perks listed below may vary depending on the nature of your employment with LTIMindtree ("LTIM").

Benefits and Perks

• Comprehensive Medical Plan

• Covering Medical, Dental, Vision, Short Term and Long-Term Disability Coverage

• 401(k) Plan with Company match

• Life Insurance

• Vacation Time, Sick Leave, Paid Holidays

• Paid Paternity and Maternity Leave

Disclaimer

The compensation and benefits information provided herein is accurate as of the date of this posting. LTIMindtree is an equal opportunity employer that is committed to diversity in the workplace. Our employment decisions are made without regard to race, color, creed, religion, sex (including pregnancy, childbirth or related medical conditions), gender identity or expression, national origin, ancestry, age, family-care status, veteran status, marital status, civil union status, domestic partnership status, military service, handicap or disability or history of handicap or disability, genetic information, atypical hereditary cellular or blood trait, union affiliation, affectional or sexual orientation or preference, or any other characteristic protected by applicable federal, state, or local law, except where such considerations are bona fide occupational qualifications permitted by law.