Senior Artificial Intelligence Evaluation Engineer

Job Description Ciklum is looking for a Senior AI Evaluation Engineer to join our team full-time in the US . We are a custom product engineering company that supports both multinational organizations and scaling startups to solve their most complex business challenges. With a global team of over 4,000 highly skilled developers, consultants, analysts and product owners, we engineer technology that redefines industries and shapes the way people live. About The Role As a Senior AI Evaluation Engineer, become a part of a cross-functional development team engineering experiences of tomorrow. Responsibilities Build the central eval harness and templates every pod uses; hub-and-spoke: central standards, pod-written testsCreate golden datasets with business and clinical reviewers, including safety and medical-accuracy suitesSet and defend pass thresholds; audit pod evals; report eval pass rates and incidents monthlyStand up production monitoring for drift and regression with the pipeline engineer Requirements 4+ years in ML/LLM evaluation, QA engineering for AI systems, or applied research engineeringHands-on with eval frameworks, LLM-as-judge patterns and their failure modes, statistical rigor on small samplesIndependent spine: the team shipping a thing does not set its own pass bar Desirable Healthcare or safety-critical evaluation experience; red-teaming background What’s in it for you? Strong community: Work alongside top professionals in a friendly, open-door environmentGrowth focus: Take on large-scale projects with a global impact and expand your expertiseTailored learning: Boost your skills with internal events (meetups, conferences, workshops), Udemy access, language courses, and company-paid certificationsEndless opportunities: Explore diverse domains through internal mobility, finding the best fit to gain hands-on experience with cutting-edge technologiesCare: Healthcare, Basic Life Insurance, Short and Long-term disability insurance according to the Company’s Benefit Plans About Us At Ciklum, we are always exploring innovations, empowering each other to achieve more, and engineering solutions that matter. With us, you’ll work with cutting-edge technologies, contribute to impactful projects, and be part of a One Team culture that values collaboration and progress. In the US, Ciklum is growing fast—inviting experienced professionals to lead digital transformation alongside Fortune 500 clients. Be part of a company where innovation and impact go hand in hand. Explore, empower, engineer with Ciklum! Interested already? We would love to get to know you! Submit your application. We can’t wait to see you at Ciklum.