Azure Databricks
Prabhav Services Inc. · Dallas, TX · 1 mo ago
OTHRFull-time
Responsibilities
- Data Engineering: Design and build scalable ETL pipelines using Azure Databricks.
- Data Migration: Migrate data from on-premises or legacy systems into Azure cloud environments.
- Big Data Processing: Handle large datasets using Spark, Delta Lake, and Databricks notebooks.
- Integration: Connect Databricks with Azure Data Lake, Synapse, Power BI, and other services.
- Optimization: Tune Spark jobs for performance and cost efficiency.
- Collaboration: Work with data scientists, analysts, and business stakeholders to deliver insights.
Requirements
- Azure Databricks: Strong hands-on experience with Databricks notebooks, clusters, and Delta Lake.
- Apache Spark: Expertise in PySpark/Scala for distributed data processing.
- Azure Ecosystem: Familiarity with Azure Data Lake, Synapse Analytics, Event Hub, and Power BI.
- Programming: Proficiency in Python, Scala, or SQL.
- Cloud Architecture: Understanding of cloud-native design, CI/CD, and DevOps practices.
- Data Governance: Knowledge of security, compliance, and role-based access in Azure.
Skills
- Azure Databricks
- Apache Spark
- Azure Ecosystem
- Programming
- Cloud Architecture
- Data Governance