Senior Data Engineer

Overview

Are you a Data Engineer passionate about being part of an Agentic AI transformation journey and playing a meaningful, hands-on role? We are looking for a Senior AI Data Engineer to design and build scalable, secure, and high-performance data platforms that power next-generation AI applications.

Key Responsibilities

• Build Data Ingestion Pipelines: Develop robust pipelines to extract data from various sources (databases, APIs, flat files) supporting SpecLink AI and related platforms
• Data Transformation & Processing: Implement scalable transformation and data quality processes to make raw data usable for AI workloads
• Data Loading & Indexing: Design and manage pipelines that load processed data into storage systems, search indices, and vector stores
• Real-Time & Incremental Processing: Enable streaming and near real-time data pipelines where required by AI systems
• Pipeline Orchestration & Automation: Implement scheduling, orchestration, and monitoring of data workflows
• Data Integration & APIs: Develop integration components and APIs to support real-time and on-demand data access
• Testing, Validation & Observability: Implement data quality checks, monitoring, and alerting for pipeline reliability
• Performance Optimization: Continuously optimize pipelines for performance, scalability, and cost efficiency
• Collaboration: Partner with global Data Engineering teams, R&D, and AI architects to implement RAG and AI data solutions
• Documentation & Maintainability: Document data models, pipelines, dependencies, and operational processes
• Operational Ownership: Monitor pipelines in production and handle post go-live maintenance and improvements

Required Experience & Skills

• 6–10+ years of experience in Data Engineering or related roles
• Strong hands-on experience building end-to-end, production-grade data pipelines
• AI Tooling Exposure: Hands-on exposure to tools such as GitHub Copilot, Claude Code, Cursor, or similar AI-assisted development tools
• Data Engineering Fundamentals: SQL, data modelling (dimensional, normalized, Lakehouse)
• Azure Data Platform: Synapse, Data Factory / Fabric, Azure SQL, Cosmos DB
• Understanding of Modern Data Architecture: Lakehouse, Medallion architecture, Delta / Iceberg, batch + streaming
• AI Data Infrastructure: Vector databases, embeddings, RAG indexing, unstructured data pipelines
• Real-Time & Event-Driven Systems: Streaming, event-driven pipelines, asynchronous workflows
• Data Quality & Observability: Data freshness, retrieval quality, monitoring, SLAs, and cost tracking
• AI Evaluation Pipelines: Golden datasets, benchmarking, regression datasets
• Strong experience with MS SQL, Azure SQL, or Cosmos DB
• Exposure to building pipelines for AI/ML or Agentic AI use cases
• Solid understanding of incremental loads, CDC, and performance tuning
• Knowledge of Azure security (Managed Identity, RBAC, Key Vault)

Soft Skills

• Strong communication and stakeholder collaboration
• Ability to work in a fast-paced, cross-functional environment
• Ownership mindset with focus on business impact and delivery

Nice to Have

• Experience with event-driven architecture and asynchronous processing
• Familiarity with CI/CD pipelines, DevOps practices, and Git-based workflows