Computational Linguist with Gen AI experience | Sigma AI

About the role

We’re looking for a versatile Computational Linguist to join our R&D team focused on evaluating and supporting Generative and Agentic AI systems. This role combines linguistic expertise, data analysis, and hands-on experimentation with large language models. You’ll help design annotation workflows, create and refine guidelines and internal documentation, prototype task-specific evaluation metrics, configure annotation tools, and analyze annotator, model and system performance using real-world data, contributing to papers and articles as needed.

Responsibilities

• Help design annotation workflows
• Create and refine guidelines and internal documentation
• Prototype task-specific evaluation metrics
• Configure annotation tools
• Analyze annotator, model and system performance using real-world data
• Contribute to papers and articles as needed

Requirements

• Demonstrate technical leadership in driving complex projects from concept to delivery
• Move between qualitative language analysis and quantitative evaluation
• Work cross-functionally with researchers and annotators to design innovative, rigorous, and scalable evaluation processes for LLM-powered workflows

Qualifications

• Master’s degree (or equivalent experience) in Computational Linguistics, NLP, Linguistics, or a related field
• 2+ years of experience in NLP or AI projects (industry or research)
• At least one year of experience with Gen AI and/or Agentic AI
• Experience using and fine-tuning transformer-based language models (e.g., BERT)
• Proficiency in Python programming
• Proficient with NLP and data science libraries: pandas, numpy, scikit-learn, NLTK
• Experience with generative AI SDKs and frameworks (e.g., OpenAI, Google, Anthropic, LangChain)
• Comfortable with Linux environments and Bash scripting
• Experience working with public datasets (e.g. Hugging Face, Kaggle)
• Familiarity with LLM behavior, prompt-based evaluation, and generative model outputs
• Experience using Git for version control and collaborative development
• Understanding of model evaluation methodologies, including human-AI comparison and red teaming
• Strong written communication skills for documenting experiments and results
• Experience working in cross-functional or research-oriented teams
• Fluent in English

Preferred Qualifications

• Strong interest in and understanding of current trends and techniques in generative AI
• Experience with annotation tools (e.g., Label Studio, Prodigy) and quality metrics for human data
• Experience creating and curating bespoke datasets
• Familiarity with evaluation challenges in creative or subjective NLP tasks
• Understanding of linguistic typology, multilingual NLP, or sociolinguistic variation
• Experience working in WSL environments
• Experience collaborating with annotation teams and working with QA processes