Senior Data Architect

Primary Responsibilities

• Lead the Design, development, and implementing of Lakehouse & data warehouses architecture in a cloud based data environment that integrates with an overall data architecture.
• Work closely with business stakeholders to interpret their data needs and translate them into technical requirements.
• Have a good understanding of data governance and develop & implement data governance policies and procedures.
• Ensure data quality and integrity by implementing data testing and validation processes.
• Deploying and debugging cloud data initiatives as needed in accordance with best practices throughout the development lifecycle.
• Managing cloud data environments in accordance with company security guidelines.
• Lead the design and build of reusable, repeatable solutions and components for future use.
• Lead and mentor a team of data engineers to implement your solution designs.
• Stay abreast of current and emerging trends and technologies in the data and AI/ML field.
• Educate delivery teams on the implementation of new cloud-based data analytics initiatives, providing associated training as required.
• Partner with Delivery, and Support teams to find opportunities to reduce manual effort needed to complete deployments.
• Consult on Professional Services Engagements to help our customers design and implement data warehouse solutions.
• Lead and develop best practices for the larger data analytics delivery team.
• Provide client presentations to review project design, outcomes and recommendations.
• Employ exceptional problem-solving skills, with the ability to see and solve issues before they snowball into problems.
• Lead the orchestration and automation cloud-based data platforms.

Preferred Skills And Experience

• 7+ years experience in Data Architecture, data engineering & analytics in areas such as performance tuning, pipeline integration & infrastructure configuration.
• 10+ years of consulting experience.
• Completed Databricks Data Engineering Professional/Associate certification OR Cloud Certification (Azure, AWS, GCP).
• Working knowledge of two or more common Cloud ecosystems (AWS, Azure, GCP) with deep expertise in at least one.
• Deep experience with distributed computing with Spark with knowledge of Spark runtime internals & Spark Structured Streaming.
• Working knowledge of MLOps with a strong understanding of essential components of MLOps Architecture to build, train and deploy models.
• Current knowledge across the breadth of Databricks product and platform features.
• Familiarity with optimizations for performance and scalability.
• Data Pipelining Experience leveraging Databricks Delta Live Tables and Data Built Tool (DBT).
• Experience with terraform, Git, CI/CD tools as well as Automation and Integration testing.
• Thorough understanding of Databricks Delta, Iceberg, and Hudi.
• Understanding of Constraints, Expectations, CDC, CDF, SCD Type 1/Type 2.
• Understanding of Unity Catalog & DBX Governance/Security Models.
• Familiarity with leveraging Databricks REST APIs - Testing and Deployment - SCIM API & DBX CLI.