Data & Analytics Platform Architect - Hybrid

Key Responsibilities

• Enterprise Data Platform Architecture & Engineering
• Build, and continuously improve the enterprise data platform, ensuring reliability, scalability, and maintainability across core business processes and analytics use cases.
• Own the full data platform lifecycle — from schema design and pipeline architecture to monitoring, performance tuning, and incident response.
• Establish and enforce data modeling standards, naming conventions, and governance frameworks across all environments.
• Implement policy enforcement points and access controls (data catalogs, encryption, RBAC) to ensure compliance, privacy, and data protection.
• Analytical Data Modeling & Schema Design
• Design, build, and evolve dimensional data models — including star schemas on Azure Databricks — optimized for analytics and reporting.
• Develop and refine medallion architecture (bronze-silver-gold layers) for efficient data ingestion, transformation, and consumption.
• Balance model simplicity, flexibility, and performance while minimizing redundancy across analytical datasets.
• Cloud & Big Data Architecture
• Lead the design and evolution of the Databricks intelligent data platform, enabling scalable big data processing and laying the foundation for AI/ML capabilities.
• Architect and manage Azure-based infrastructure including Azure SQL, Azure Data Factory, Azure Synapse Analytics, Data Lake, and related services.
• Apply data mesh principles across multiple data repositories to enable decentralized, domain-oriented data ownership.
• Pipeline Optimization & Automation
• Ensure ETL/ELT processes and pipeline tools (Azure Data Factory, Databricks/Spark) run efficiently to deliver timely, high-quality data for analytics, BI, and AI/ML.
• Design and implement automation to significantly reduce recurring DBA and operational tasks, minimizing manual intervention.
• Develop monitoring, alerting, and self-healing mechanisms to proactively maintain platform health and SLA adherence.
• Identify and resolve bottlenecks, continuously tuning for performance and scalability.
• Domain-Specific Solutions
• Design and implement algorithms supporting availability guarantees, contractual agreement calculations, regulatory reporting (e.g., GADS), and other domain-specific requirements.
• Serve as Subject Matter Expert (SME) for performance engineering — profiling, tuning, and resolving issues across database and pipeline layers.
• AI & Agentic Capabilities Integration
• Incorporate AI and agentic capabilities as complementary components of the data platform.
• Design and evolve secure LLM integration patterns using enterprise LLM gateways to centralize model access, routing, governance, and cost controls.
• Leverage frameworks like Model Context Protocol (MCP) to connect AI applications and agents with enterprise data sources in a secure, governed manner — enabling intelligent, agent-driven data workflows.

Required Qualifications

• Bachelor’s degree in Computer Science, Data Engineering, Data Science, or a related field.
• 10+ years of experience designing and evolving large-scale data analytics platforms, with deep expertise in data integration (ETL/ELT), medallion-tier pipelines, cloud data services, and MLOps.
• Deep expertise in SQL — query optimization, schema design, indexing strategies, stored procedures, and performance tuning across platforms such as Microsoft SQL Server or Azure SQL.
• Hands-on experience with Microsoft Azure data services (Azure SQL, Azure Data Factory, Azure Synapse Analytics, Azure Data Lake, Blob Storage).
• Proven experience designing and building on Databricks, including Delta Lake, Spark jobs, and cluster management.
• Strong familiarity with data lakehouse architecture and applying data mesh principles across enterprise environments.
• Proven experience with: Azure Data Services (Synapse, Data Factory, Data Lake), Delta Lake, Azure Databricks.
• Solid understanding of enterprise data governance, security (access controls, data privacy), and data quality best practices.
• Demonstrated success automating DBA and data operations tasks to significantly reduce manual workload.
• Experience working with contractual or regulatory reporting requirements in data-intensive industries (e.g., energy, utilities, or finance).
• Strong communication and interpersonal skills; ability to work effectively with both technical and non-technical stakeholders across multiple concurrent priorities.

Preferred Qualifications

• Experience integrating AI/ML or LLM solutions into data platforms (e.g., Azure Cognitive Services, LLM gateways for multi-provider model integration, or context frameworks like MCP for AI-driven data products).
• Experience with energy sector data systems, including solar forecasting, GADS reporting, or availability guarantee frameworks.
• Experience with DevOps practices for data: CI/CD pipelines for database deployments, infrastructure-as-code (Terraform, Bicep, ARM templates).
• Knowledge of data security, encryption at rest/in transit, and RBAC in cloud environments.
• Microsoft Certified: Azure Data Engineer Associate or Databricks Certified Data Engineer Professional.
• Experience partnering with third-party data vendors and managing vendor-delivered integrations.
• Master’s degree in a relevant field.