Databricks Solution Architect

About the role

We are looking for a highly skilled and passionate Databricks Solution Architect to design end-to-end data solutions, define architecture patterns, guide implementation teams, ensure integration with enterprise systems, and promote best practices in data engineering and machine learning workflows.

Responsibilities

• Work closely with Data Scientists, Business Teams, and Engineers to understand business requirements, translate those requirements into technical architectures, and ensure the effective integration of Databricks with other enterprise systems.
• Recommend best practices around data engineering, machine learning, and analytics workflows, as well as guide teams in the adoption of efficient architecture patterns.
• Define project scope, set technical direction, and troubleshoot complex problems.
• Lead the development of proof-of-concept solutions, evaluate new platform features, and ensure that security and compliance standards are met.
• Plan and execute migration strategies to transition on-prem data warehouses and data lakes into cloud-based Databricks.
• Develop reusable frameworks for Data Engineering and Machine Learning Data Pipelines.
• Explore AI and Gen AI solutions for business requirements and provide solutions to adopt them.

Requirements

• Proven expertise in implementing Lakehouse and Delta Lake using Databricks.
• Strong PySpark and Python experience.
• Databricks (AWS) Platform Architect Certification.
• Familiarity with ML Ops/LLM Ops and distributed systems.
• Experience with Big Data platforms like Cloudera Hadoop and cloud platforms like AWS, GCP.
• Solid understanding of system design patterns, scalability, observability, and performance tuning.
• Strong analytical and problem-solving skills.
• Passion for exploring and building with emerging technologies.

Skills

• AWS EKS Experience
• Dockers and Containers

Pay

$125,000–$140,000 a year