Azure Data Analytics Architect

Job Overview

Job ID: J52732
Location: Houston, TX
Duration: 20 Months + Extension
Hourly Rate: Depending on Experience (DOE)
Work Authorization: US Citizen, Green Card, OPT-EAD, CPT, H-1B, H4-EAD, L2-EAD, GC-EAD
Client: To Be Discussed Later
Employment Type: W-2, 1099, C2C

Responsibilities

• Lead the architecture, design, and implementation of advanced analytics solutions using Azure Databricks/Fabric.
• Work closely with business stakeholders and other IT teams to understand requirements and deliver effective solutions.
• Oversee the end-to-end implementation of data solutions, ensuring alignment with business requirements and best practices.
• Lead the development of data pipelines and ETL processes using Azure Databricks, PySpark, and other relevant tools.
• Integrate Azure Databricks with other Azure services (e.g., Azure Data Lake, Azure Synapse, Azure Data Factory) and on-premise systems.
• Provide technical leadership and mentorship to the data engineering team, fostering a culture of continuous learning and improvement.
• Ensure proper documentation of architecture, processes, and data flows, while ensuring compliance with security and governance standards.
• Ensure best practices are followed in terms of code quality, data security, and scalability.
• Stay updated with the latest developments in Databricks and associated technologies to drive innovation.

Essential Skills

• Strong experience with Azure Databricks, including cluster management, notebook development, and Delta Lake.
• Proficiency in big data technologies (e.g., Hadoop, Spark) and data processing frameworks (e.g., PySpark).
• Deep understanding of Azure services like Azure Data Lake, Azure Synapse, and Azure Data Factory.
• Experience with ETL/ELT processes, data warehousing, and building data lakes.
• Strong SQL skills and familiarity with NoSQL databases.
• Experience with CI/CD pipelines and version control systems like Git.
• Knowledge of cloud security best practices.

Soft Skills

• Excellent communication skills, with the ability to explain complex technical concepts to non-technical stakeholders.
• Strong problem-solving skills and a proactive approach to identifying and resolving issues.
• Leadership skills with the ability to manage and mentor a team of data engineers.

Nice to have Skills

• Power BI for dashboarding and reporting.
• Microsoft Fabric for analytics and integration tasks.
• Spark Streaming for processing real-time data streams.
• Familiarity with Azure Resource Manager (ARM) templates for infrastructure as code (IaC) practices.

Experience

• Demonstrated expertise of 8+ years in developing data ingestion and transformation pipelines using Databricks/Synapse notebooks and Azure Data Factory.
• Solid understanding and hands-on experience with Delta tables, Delta Lake, and Azure Data Lake Storage Gen2.
• Experience in efficiently using Auto Loader and Delta Live tables for seamless data ingestion and transformation.
• Proficiency in building and optimizing query layers using Databricks SQL.
• Demonstrated experience integrating Databricks with Azure Synapse, ADLS Gen2, and Power BI for end-to-end analytics solutions.
• Prior experience in developing, optimizing, and deploying Power BI reports.
• Familiarity with modern CI/CD practices, especially in the context of Databricks and cloud-native solutions.