Databricks Architect

Responsibilities

• Design and implement scalable, end-to-end data solutions for ETL, real-time streaming, and machine learning workloads on the Databricks platform
• Provide technical leadership and strategic guidance on Databricks adoption, including developing reference architectures and best practices
• Ensure optimal performance by tuning Spark jobs, optimizing data pipelines, and resolving complex technical issues
• Architect, develop, and debug PySpark jobs and data models, integrating Databricks with cloud services such as AWS S3, EMR, and Lambda
• Implement and enforce data governance, security, and access control policies (e.g., encryption, auditing) on the Databricks environment

Requirements

• 12+ years of experience in Architecture, design and data engineering development of large-scale data ecosystems
• Databricks Design and Architecture: Design and architect scalable and secure Databricks solutions platform, optimizing performance, cost, and reliability
• Hands on Databricks/Dataops Architect that can help reduce consumption, and improve performance issues. Can help build monitoring tools
• Good exposure to AWS. Strong comm and leadership skills and Self driven and work independently
• Experience in Performance initiatives and drive recommendations (identify tuning opportunities, creation of performance environment etc)
• Experience with hadoop environment

Core Skills

• Databricks platform knowledge, Unity Catalog migration implementation exposure, Spark Streaming pipeline design exposure

Benefits

Significant career development opportunities exist as the company grows. The position offers a unique opportunity to be part of a small, fast-growing, challenging, and entrepreneurial environment, with a high degree of individual responsibility.

Tiger Analytics provides equal employment opportunities to applicants and employees without regard to race, color, religion, age, sex, sexual orientation, gender identity/expression, pregnancy, national origin, ancestry, marital status, protected veteran status, disability status, or any other basis as protected by federal, state, or local law.