Software Development Engineer - Expert Consultant, AGI - Data Services

Key job responsibilities

• Serve as a trusted domain advisor to cross-functional teams, providing strategic direction and specialized problem-solving support
• Champion domain knowledge sharing across multiple channels and teams to maintain data quality excellence and standardization
• Drive collaborative efforts with science teams to optimize output of complex data collections in your domain expertise, ensuring data excellence through iterative feedback loops
• Foster team excellence through mentorship and motivation of peers and junior team members
• Make informed decisions on behalf of our customers, ensuring that selected code meets industry standards, best practices, and specific client needs
• Collaborate with AI teams to innovate model-in-the-loop and human-in-the-loop approaches, to ensure the collection of high-quality data, safeguarding data privacy and security for LLM training, and more
• Stay abreast of the latest developments in how LLMs and GenAI can be applied to your area of expertise to ensure our evaluations remain cutting-edge
• Develop and write demonstrations to illustrate "what good data looks like" in terms of meeting benchmarks for quality and efficiency
• Provide detailed feedback and explanations for your evaluations, helping to refine and improve the LLM's understanding and output

Basic Qualifications

• 2+ years of non-internship design or architecture (design patterns, reliability and scaling) of new and existing systems experience
• Bachelor's degree in computer science or equivalent
• Experience guiding and coaching other developers
• Previous experience working with or evaluating AI systems

Preferred Qualifications

• 3+ years of full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations experience
• Experience with Cloud platforms (preferably AWS), database systems (SQL and NoSQL), AI tools for development productivity, contributing to open-source projects, and/or version control systems
• Prior experience in defining and creating benchmarks for assessing GenAI model performance
• Excellent written and verbal communication skills, with the ability to explain complex concepts clearly