Software Engineer | $70/hr Remote

About the role

Execute structured evaluation assignments simulating developer activities using AI-powered tools. Assess and grade technical correctness of outputs within development environments. Design and maintain reproducible test environments with real commit histories and CI workflows. Configure and troubleshoot integrations across multiple software services for seamless connectivity. Investigate and document undocumented product behaviors through systematic testing. Participate in calibration sessions to ensure consistent evaluation methods.

Requirements

• Have strong relevant experience in software engineering with Git/GitHub workflows.
• Proficient with CI/CD systems, ideally GitHub Actions, and authoring custom workflows.
• Advanced scripting abilities in Python or Bash, with understanding of REST APIs and OAuth.
• Fluency with major enterprise SaaS platforms; administrative experience in cloud workspaces is advantageous.
• Experience leveraging AI assistants or agent tools in developer tasks is a plus.
• Background in technical QA, evaluation, or data annotation with consistency in applying rubrics.

Compensation

$50 - $70/hour

Schedule

10-40 hrs/week

Location

Remote