AI Engineer

Job Description

KēSTA I.T. is seeking an experienced AI Agent Developer & Test Engineer to join a cutting-edge team focused on building, testing, and supporting next-generation AI-powered applications. This role combines hands-on AI development with rigorous testing and quality assurance responsibilities, ensuring intelligent agent workflows perform reliably, securely, and consistently throughout the software development lifecycle.

Responsibilities

• Design, develop, and maintain AI agent workflows utilizing modern agent orchestration frameworks.
• Build and execute comprehensive testing strategies for AI agent systems, including unit, integration, regression, and end-to-end testing.
• Create and maintain regression test suites to ensure defects remain resolved over time.
• Implement effective LLM mocking and simulation strategies to support deterministic and repeatable testing.
• Develop and maintain automated end-to-end testing coverage for user-facing applications and workflows.
• Maintain production monitoring, logging, tracing, and observability practices for AI systems.
• Support cloud-based application deployments and ongoing operational maintenance.
• Collaborate closely with software engineers, architects, product owners, and QA teams to deliver high-quality AI solutions.
• Participate in troubleshooting, root-cause analysis, and production support activities.

Required Qualifications

• Professional experience developing AI-powered applications using agent frameworks such as LangGraph and/or LangChain.
• Strong Python development experience.
• Experience with automated testing frameworks including pytest and pytest-asyncio.
• Experience designing and executing unit tests for complex workflow-based applications.
• Experience testing distributed or orchestrated agent workflows from end to end.
• Strong understanding of test isolation, mocking strategies, and simulation techniques.
• Experience managing regression testing programs and quality assurance processes.
• Hands-on experience with Playwright or similar end-to-end testing frameworks.
• Experience producing and interpreting test coverage metrics and reporting.
• Familiarity with structured output validation and JSON schema testing.
• Experience implementing monitoring and observability solutions for production applications.
• Cloud application development and support experience within modern cloud platforms (GCP preferred).
• Strong analytical, troubleshooting, and problem-solving skills.
• Excellent communication and collaboration abilities.

Preferred Qualifications

• Experience with AI evaluation and observability tools such as LangSmith, RAGAS, or similar platforms.
• Experience managing prompt versioning and prompt lifecycle processes.
• Knowledge of adversarial testing, red-team testing, and AI security validation methodologies.
• Experience monitoring token consumption, model utilization, and AI operational costs.
• Experience working with document extraction, OCR, or intelligent document processing solutions.
• Experience supporting applications within regulated industries such as financial services, healthcare, or government.

Preferred Platform & DevOps Experience

• Google Cloud Platform (GCP)
• Cloud Run
• Terraform
• Azure DevOps
• CI/CD pipeline design and automation
• Infrastructure-as-Code (IaC) best practices

Location

Midvale, UT • On-site, Remote

Job Details

Location: Midvale, UT
Job Type: hybrid or office
Posted: Jun 2, 2026
Categories: Software Engineering

Job ID

This position is open and accepting applications. Click the button below to apply directly on the company's website.
Please mention that you found this job on chatgpt-jobs. This helps us get more companies to post here!