DATABRICKS SOLUTION ARCHITECT

Skills

• Understanding how application consume data
• Need someone who can come in look at the architecture and code and offer optimization solution
• Focus on performance, cost, security and governance
• Deliver an optimization roadmap
• DataBricks Solutions architect on the application side.
• Someone who has experience with consumption of data

Responsibilities

• Architecture assessment
  • Review the existing Databricks architecture
  • Understand how applications consume Databricks data
  • Review data flows, integrations, APIs, SQL Warehouses, clusters, Delta tables, etc.
  • Identify architectural bottlenecks and unnecessary complexity
• Code review
  • This is critical. They should be comfortable getting into the actual:
  • PySpark code
  • Spark SQL
  • Databricks notebooks
  • SQL queries
  • ETL/ELT pipelines
  • Delta Lake implementation
  • Streaming code,