Engineering Expert

This role is for one of our clients We are seeking experienced Engineering Experts to design, develop, and validate complex engineering simulation problems for training and evaluating advanced AI models and coding agents. This role combines deep engineering expertise, scientific computing, simulation-based design, and AI evaluation to create challenging tasks that test an AI system's ability to interpret technical requirements, solve engineering problems, use simulation tools, and produce physically valid solutions. Experts will work across Electrical, Mechanical, and Aerospace Engineering, developing realistic, multi-constraint problems with measurable objectives and objectively verifiable outcomes. The role also involves analyzing AI execution logs, identifying reasoning and tool-use failures, and refining evaluation benchmarks to improve model performance. Job Details Role: Engineering ExpertExperience: 10+ years of hands-on engineering design experienceEducation: Master's degree or PhD in Electrical, Mechanical, Aerospace Engineering, or a closely related disciplineLocations: Bangladesh, India, Indonesia, Egypt, Ghana, Nigeria, Turkey, Brazil, Colombia, and approved US statesEmployment Type: Short-term contractor engagementContract Duration: Up to 24 weeksStart Date: ImmediateCompensation: USD 500 per task, based on an estimated 30-40 hours of work per taskAvailability: Approximately 40 hours per week, with at least 4 hours of overlap with US Pacific Time (PST)Work Arrangement: Remote, with access to a personal computer and reliable high-speed internet Requirements Key Responsibilities2. Simulation and Computational Engineering3. AI Evaluation and Failure Analysis4. Benchmark Development and Refinement Engineering Problem DesignDevelop original, self-contained engineering design problems with realistic technical requirements, competing constraints, and clearly defined optimization objectivesCreate challenging tasks that require AI models to interpret specifications, evaluate design trade-offs, configure simulation environments, and iteratively improve engineering solutionsEstablish validated reference solutions and objective automated grading mechanisms to assess correctness, performance, and compliance with engineering requirementsBuild, configure, execute, and validate simulation-based problem environments using relevant open-source engineering toolsDevelop Python scripts, custom test benches, and supporting utilities to automate simulations and verify expected resultsEnsure that simulation outputs satisfy physical plausibility, dimensional consistency, boundary conditions, convergence requirements, and other domain-specific constraintsTroubleshoot simulation failures, numerical instability, dependency issues, and computational errorsEvaluate the performance of large language models (LLMs) and coding agents across repeated engineering problem-solving trialsAnalyze execution trajectories, tool interactions, simulator feedback, and intermediate outputs to identify systematic reasoning failuresInvestigate issues such as incorrect interpretation of engineering specifications, premature convergence, invalid geometries, inconsistent units, and physically impossible solutionsApply evaluation concepts such as pass@k, failure-mode classification, and analysis of nondeterministic model behaviorCalibrate task difficulty using empirical model performance and evaluation resultsRefine engineering problems to improve their ability to distinguish between successful and unsuccessful AI solutionsEnsure that tasks remain unambiguous, technically rigorous, reproducible, and objectively gradableCollaborate with AI researchers, technical leads, and subject-matter experts to integrate validated benchmarks into AI evaluation workflows Required Qualifications And Skills Master's degree or PhD in Electrical Engineering, Mechanical Engineering, Aerospace Engineering, or a closely related technical disciplineAt least 10 years of practical engineering design experienceStrong Python programming and scripting skillsHands-on proficiency with at least one relevant open-source engineering simulation or design tool, such as:Electrical simulation: ngspice, PySpiceMechanical and computational engineering: OpenFOAM, FEniCSx, CalculiXControl systems: python-control, OpenModelicaCAD and design automation: CadQuery, build123dThermodynamics and meshing: Cantera, GmshExperience developing, executing, debugging, and validating simulation-based engineering workflowsFamiliarity with modern LLMs, AI coding agents, and AI evaluation methodologiesAbility to analyze model execution logs and identify underlying reasoning, planning, and tool-use failuresStrong understanding of physical constraints, units, boundary conditions, numerical convergence, and engineering validationExcellent analytical, problem-solving, and technical documentation skillsAbility to work independently and deliver technically accurate results within defined timelines Preferred Qualifications Experience in AI model evaluation, benchmark development, automated grading, or coding-agent assessmentFamiliarity with repeated-trial evaluation, pass@k metrics, failure taxonomies, and nondeterministic model behaviorExperience building automated testing frameworks and reproducible computational environmentsKnowledge of optimization methods, numerical analysis, simulation automation, and engineering design validationExperience collaborating with multidisciplinary engineering and AI research teams Engineering Domains Candidates should have strong expertise in one or more of the following areas: Electrical EngineeringMechanical EngineeringAerospace Engineering Work Commitment and Contract Terms Expected commitment of approximately 40 hours per weekAt least 4 hours of daily overlap with US Pacific TimeWeekend on-call availability is required; part-time engagement may be acceptable where agreedThe engagement is contractor-based and may last up to 24 weeksCandidates must have a personal desktop or laptop and a stable, high-speed internet connectionCompensation is USD 500 per task, with an estimated 30-40 hours of effort per task. Actual task allocation and payment terms should be confirmed before engagement