Senior AI Data Engineer

About the role

We offer a flexible working policy that supports a healthy balance between personal and professional well-being. This role requires in-office presence on Tuesdays & Thursdays to collaborate, connect, and learn from peers - while also maintaining the flexibility for meaningful work-life balance.

Responsibilities

• Designing, developing and maintaining scalable pipelines in MSFT Azure to ingest and transform large volumes of text data from multiple sources
• Designing automated workflows for text normalization, deduplication, language identification, PII redaction and metadata enrichment
• Designing automated data validation processes to ensure accuracy and consistency
• Supporting model fine-tuning, semantic search and Gen AI evaluations tuning through dataset curation, prompt dataset preparation, labeling coordination, and text quality validation
• Partnering with the Applied AI team to gather data requirements and build data interfaces for developing and maintaining machine learning systems
• Maintaining data lineage and following data privacy, security and governance best practices
• Implementing data versioning and lineage tracking for machine learning experiments

Requirements

• A Bachelor’s degree or higher in Computer Science, Data Engineering, Applied Mathematics, Computational Linguistics, or a quantitative related field.
• 4+ years of data engineering experience, with at least 2 years working with unstructured data in a business setting.
• Strong proficiency in Python, PySpark, and data manipulation for large unstructured text datasets.
• Strong understanding of NLP concepts such as tokenization, embeddings, semantic search, and experience with standard text libraries such as SpaCy, HuggingFace Datasets, NLTK.
• Solid dataOps knowledge and experience orchestrating advanced NLP data pipelines using cloud based data infrastructure.
• Proficiency with Git and collaborative development frameworks.
• A passion for enabling AI capabilities through scalable, reliable data architecture.
• Problem solving, creativity, curiosity, and a collaborative mindset.

Qualifications

• A Bachelor’s degree or higher in Computer Science, Data Engineering, Applied Mathematics, Computational Linguistics, or a quantitative related field.
• 4+ years of data engineering experience, with at least 2 years working with unstructured data in a business setting.
• Strong proficiency in Python, PySpark, and data manipulation for large unstructured text datasets.
• Strong understanding of NLP concepts such as tokenization, embeddings, semantic search, and experience with standard text libraries such as SpaCy, HuggingFace Datasets, NLTK.
• Solid dataOps knowledge and experience orchestrating advanced NLP data pipelines using cloud based data infrastructure.
• Proficiency with Git and collaborative development frameworks.
• A passion for enabling AI capabilities through scalable, reliable data architecture.
• Problem solving, creativity, curiosity, and a collaborative mindset.

Skills

• Exposure to Microsoft Azure Services such as Fabric, ADLS, AI Foundry, Azure ML, MLflow
• Experience with knowledge graph implementation for NLP applications
• Experience working with data for the legal domain
• Experience designing architectures for large-scale text corpora

Benefits

• Flexible work hours that allow you to balance your 'me time' with your work commitments.
• A supportive, experienced team with an inclusive, encouraging, and vibrant culture.
• A modern open plan workspace, with a gaming area, free snacks, drinks and regular social events.
• Focus on impactful work, solving complex, real challenges utilizing the latest technologies and protocols.
• Ownership of your career path with our internal development framework.
• Access to unlimited courses in LinkedIn Learning.
• An innovative, industry leading SaaS company that is continuing to grow & scale!

Pay

The overall US annual base salary range for this position is $120,000–$170,000. Individual compensation for each candidate depends on factors such as qualifications, experience, and candidate location. This range does not include additional forms of compensation, such as bonuses, or benefits.

Schedule

This role requires in-office presence on Tuesdays & Thursdays to collaborate, connect, and learn from peers - while also maintaining the flexibility for meaningful work-life balance.