Azure Data / Search Engineer

Roles and Responsibilities

• Resource will be responsible to design, build, and support client, our enterprise search and intelligent matching platform.
• This role will develop scalable APIs, high-performance search capabilities, event-driven data pipelines, and cloud-native services supporting millions of transactions daily.
• The consultant will work across Azure, Elasticsearch, Kafka, Databricks, Python, and distributed systems to build highly available, low-latency applications.
• The ideal candidate has strong backend engineering experience with cloud-native architectures, search technologies, REST APIs, event streaming, and large-scale data processing.
• Design and develop scalable backend services using Python.
• Build and maintain REST APIs supporting enterprise applications.
• Design and optimize Elasticsearch indexes, queries, mappings, and search relevance.
• Develop intelligent search and matching capabilities for large-scale datasets.
• Build event-driven architectures using Kafka and Azure Event Hub.
• Design high-throughput data ingestion and indexing pipelines.
• Integrate applications with Databricks and enterprise data platforms.
• Develop secure cloud-native applications on Microsoft Azure.
• Implement CI/CD pipelines and automated deployment processes.
• Maintain application health, logging, observability, and production performance.
• Optimize API response time, indexing performance, and infrastructure utilization.
• Participate in architecture, design reviews, code reviews, and production support.
• Collaborate with product managers, architects, data scientists, and engineering teams.

Educational Qualifications

• Engineering Degree BE/ME/BTech/MTech/BSc/MSc.
• Technical certification in multiple technologies is desirable.

Skills

• Mandatory Skills
• Programming Python (required)
• SQL
• REST APIs
• JSON
• Async Python
• Cloud Microsoft Azure
• Azure App Services
• Azure Linux VMs
• Azure Container Apps / AKS (preferred)
• Azure Managed Identity
• Azure Key Vault
• Azure Storage
• Azure Monitor
• Azure Event Hub
• Search Technologies Elasticsearch
• Query DSL
• Relevance tuning
• Mappings design
• Aggregations
• Multi-search (msearch)
• Bulk indexing
• Search optimization
• Index lifecycle management
• Event Streaming Apache Kafka
• Confluent Cloud Producers / Consumers
• Kafka Connect Schema Registry
• Event-driven architecture
• Data Engineering Databricks PySpark
• Delta Lake
• Data ingestion pipelines ETL/ELT
• DevOps Git
• Azure DevOps
• Azure Pipelines
• CI/CD Docker
• Infrastructure as Code (Terraform or Pulumi preferred)
• Databases PostgreSQL SQL Server Relational Databases
• Preferred Qualifications
• Experience with AI-assisted search or intelligent matching platforms.
• Experience with Databricks and Delta Lake.
• Experience with machine learning integration.
• Experience with semantic search or vector search.
• Experience with Azure OpenAI or LLM integration.
• Healthcare or supply chain experience.
• Experience using GitHub Copilot or Codex.

Required Experience

• Min 8+ years of software engineering experience.
• Strong Python backend development experience.
• Experience designing and operating production REST APIs.
• Strong experience with Elasticsearch in production environments.
• Experience building scalable search platforms.
• Experience with Kafka or enterprise event-streaming platforms.
• Experience deploying applications on Microsoft Azure.
• Experience with CI/CD pipelines and automated deployments.
• Experience troubleshooting production systems and performance issues.
• Strong understanding of distributed systems and microservices.