Databricks on AWS Platform Architect

Location: Dallas, TX

Responsibilities

• Architect and manage a secure, scalable, and highly available Databricks Lakehouse Platform on AWS.
• Design data platforms leveraging AWS S3, Delta Lake, Unity Catalog, and Databricks Workspaces for enterprise analytics and AI.
• Define networking architecture including VPC, PrivateLink, IAM roles, security groups, and encryption standards.
• Establish multi‑environment strategies (Dev, Test, UAT, Prod) with automated provisioning through Terraform and Infrastructure as Code (IaC).
• Implement enterprise‑wide data governance, lineage, metadata management, and compliance controls using Unity Catalog.
• Design high‑performance data ingestion and processing architectures supporting batch, streaming, and real‑time analytics workloads.
• Enable AI and GenAI capabilities through Mosaic AI, Vector Search, Model Serving, AI/BI Dashboards, and Genie Spaces.
• Define CI/CD, monitoring, logging, and observability frameworks using GitHub, Jenkins, CloudWatch, and Databricks Workflows.
• Optimize platform performance, reliability, scalability, and cloud costs through FinOps and workload optimization practices.
• Provide architectural leadership and best practices for Lakehouse modernization, advanced analytics, AI governance, and enterprise data platform adoption.

Core Technologies

• Databricks
• AWS S3
• Delta Lake
• Unity Catalog
• Spark / PySpark
• Terraform
• GitHub
• CI/CD pipelines
• Amazon CloudWatch
• Mosaic AI
• Genie
• Vector Search
• MLflow
• Kafka
• Airflow
• Kubernetes

Certifications / Requirements

• Databricks Platform Administrator
• Databricks Certified Data Engineer
• AWS Certification (DevOps or Solutions Architect)
• Terraform expertise