Sr. Databricks Solutions Architect

Responsibilities

• Lead customer engagements to design, build, and optimize Databricks-based architectures for advanced analytics, data engineering, and machine learning workloads.
• Develop scalable ETL/ELT pipelines and integrate with cloud platforms (AWS, Azure, or GCP).
• Guide customers on data governance, security, and compliance best practices within Databricks environments.
• Consult on architecture, reference implementations, and best practices for leveraging Delta Lake, Unity Catalog, MLflow, and related Databricks capabilities.
• Aid customers with productionalizing data pipelines, machine learning workflows, and AI-driven applications.
• Provide escalated technical support for customer operational issues and help troubleshoot complex platform or workflow challenges.
• Collaborate with internal and Databricks teams, including Engineers, Architects, Project Managers, and Customer Success teams, to ensure engagement goals are met.
• Document technical designs, architecture patterns, deployment procedures, and lessons learned.
• Stay current on Databricks platform features, distributed computing trends, and emerging big data technologies.
• Deliver solutions that improve performance, scalability, and operational efficiency while meeting customer business objectives.
• Support Professional Services and Managed Services initiatives as needed, ensuring billable deliverables meet customer expectations.

Requirements

• US Top Secret Clearance required.
• 7+ years of experience in Data Engineering.
• 10+ years of consulting experience, preferably in data platform or analytics-focused engagements.
• Completion of 6-8 hands-on projects with Databricks in production environments.
• Proven experience with Databricks, including Spark, Delta Lake, MLflow, and cloud integration.
• Strong proficiency in Python and/or SQL for data engineering and analytics.
• Deep understanding of distributed computing concepts and Apache Spark runtime internals.
• Hands-on experience designing and deploying end-to-end big data and machine learning solutions.
• Familiarity with data modeling, performance tuning, and production-grade pipeline design.
• Experience working directly with customers in a consulting or professional services capacity.
• Ability to manage technical scope, timelines, and delivery while maintaining excellent customer communication.
• Bachelor’s degree in Computer Science, Information Systems, Engineering, or equivalent professional experience.
• Willingness to travel up to 30% for customer engagements.

Desired Skills

• Master’s or PhD in Computer Science, Data Science, or related field.
• Experience implementing MLOps pipelines and productionizing machine learning workflows.
• Knowledge of CI/CD, version control (Git), and infrastructure-as-code tools (Terraform, ARM, CloudFormation).
• Exposure to streaming data technologies (Kafka, Kinesis, Event Hubs).
• Familiarity with data visualization tools (Tableau, Power BI, Looker).
• Experience with regulatory compliance frameworks (HIPAA, FedRAMP, SOC2).
• Prior consulting experience with technical project delivery in enterprise environments.
• Strong documentation, whiteboarding, and customer presentation skills.