Databricks DE with Azure
About the role
We are seeking an experienced Azure Databricks Data Engineer to design, develop, and maintain scalable cloud-based data solutions. The ideal candidate will have strong expertise in Azure Data Engineering technologies, Databricks, data pipelines, data warehousing, and big data processing. This role involves collaborating with business stakeholders, architects, data scientists, and application teams to deliver high-quality data platforms that support analytics and AI initiatives.
Fractal is a strategic AI partner to Fortune 500 companies with a vision to power every human decision in the enterprise. We are building a world where individual choices, freedom, and diversity are the greatest assets; an ecosystem where human imagination is at the heart of every decision. No possibility is written off, only challenged to get better.
Fractal has been featured as a Great Place to Work by The Economic Times and recognized as a ‘Cool Vendor’ and a ‘Vendor to Watch’ by Gartner. Learn more about Fractal.
Please Note: We are not offering sponsorship at this time.
Responsibilities
- Design, build, and optimize scalable data pipelines using Azure Databricks.
- Develop ETL/ELT workflows for large-scale structured and unstructured datasets.
- Implement batch and real-time data processing solutions using PySpark and Spark SQL.
- Build and maintain data ingestion frameworks from multiple data sources.
- Develop and manage data lake solutions using Azure Data Lake Storage (ADLS Gen2).
- Integrate data from cloud and on-premises systems into enterprise data platforms.
- Implement data quality, validation, and monitoring frameworks.
- Optimize Spark jobs for cost, scalability, and performance.
- Design dimensional models and support data warehouse initiatives.
- Collaborate with business users to understand reporting and analytics requirements.
- Automate deployment processes using CI/CD pipelines.
- Participate in architecture discussions and provide technical leadership to junior engineers.
- Ensure adherence to security, governance, and compliance standards.
Requirements
- Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or related field.
- 6-10 years of experience in Data Engineering and Analytics.
- Minimum 3+ years of hands-on experience with Azure Databricks.
- Strong experience with PySpark, SQL, and cloud-native data platforms.
- Experience building enterprise-scale data lakes and data warehouses.
- Knowledge of performance tuning and optimization techniques in Spark.
- Experience working in Agile delivery environments.
Skills
Azure Services
- Azure Databricks
- Azure Data Factory (ADF)
- Azure Data Lake Storage Gen2 (ADLS)
- Azure Synapse Analytics
- Azure Key Vault
- Azure Monitor
- Azure DevOps
Data Engineering
- PySpark
- Spark SQL
- Delta Lake
- ETL/ELT Pipeline Development
- Data Modeling
- Data Warehousing Concepts
- Data Governance and Data Quality
Programming
- Python
- SQL
- Scala (Preferred)
DevOps & Automation
- Azure DevOps
- Git
- CI/CD Pipelines
- Infrastructure as Code (Terraform preferred)
Preferred Qualifications
- Azure Data Engineer Associate Certification (DP-203).
- Experience with Unity Catalog and Databricks Workflows.
- Knowledge of Delta Live Tables (DLT).
- Experience supporting AI/ML workloads and data science teams.
- Exposure to Snowflake, Power BI, or Microsoft Fabric.
- Healthcare, Life Sciences, or Pharma domain experience is a plus.
Pay
The wage range for this role takes into account factors such as skill sets, experience, training, licensure, certifications, and business needs. The disclosed range estimate has not been adjusted for geographic differentials. A reasonable estimate of the current range is $120,000 - $140,000. You may also be eligible for a discretionary bonus.
Benefits
- Health, dental, vision, life insurance, and disability plans (eligible from the first day of employment).
- 401(k) Plan eligibility after 30 days of employment.
- 11 paid holidays.
- 12 weeks of Parental Leave.
- Flexible PTO policy for sick time and vacation.