Data/Information Architect

Mandatory Skills

• Azure Databricks
• Azure data lake
• Ansible
• SQL
• Terraform
• Kafka
• Azure CLI
• Databricks CLI
• Powershell and/or Bash

What Could Set You Apart

• Designed the E2E architecture of unified data platform covering all the aspects of data lifecycle
• Experience in handling the key activities for the Enterprise Architecture practice for a specific sector
• Managing and mentoring architecture talent in the respective sector
• Delivered architecture initiatives and showcased clear business efficiency in line with business strategies
• Took end-end responsibility & accountability in terms of architecture
• Delivered architecture roadmaps and develop delivery blueprints for the technology design
• Excellent communication skills and customer management skills
• Delivered architectural initiatives in the space of digital, cloud, open-source technologies
• Performed a similar role & capacity for large transformational engagements
• Hands on development Design and Develop applications using Databricks
• Experience with Azure, AWS, or other cloud technologies
• In depth understanding of Spark Architecture including Spark Core, Spark SQL, Data Frames, Spark Streaming, RDD caching, Spark MLib
• Good to have any programming language experience with Python/ SQL or Spark/Scala
• Strong understanding of Data Modeling and defining conceptual logical and physical data models
• Maintains close awareness of new and emerging technologies and their potential application for service offerings and products
• Experience in architecting and designing technical solutions for cloud-centric solutions based on industry standards using IaaS, PaaS, and SaaS capabilities
• Experience building and supporting mission-critical technology components with DR capabilities
• Experience with multi-tier system and service design & development for large enterprises
• Exposure to infrastructure and application security technologies and approaches
• Familiarity with requirements gathering techniques

Required Qualifications

• Excellent coding skills either Python or Scala, preferably Python
• At least 10+ years of experience in architecture, design, implementation, and analytics solutions with total of 12+ years in Data Engineering domain
• Designed and implemented at least 2-3 projects end-to-end in Databricks
• Worked on various components of Databricks such as Delta lake, Connectdb API 2.0, SQL Endpoint, Photon engine, Unity Catalog, Databricks workflows orchestration, Security management, Platform governance, Data Security
• Followed various architectural principles to design best suited per problem
• Strong understanding of Databricks Lakehouse concept and its implementation in enterprise environments
• Strong understanding of Data warehousing and various governance and security standards around Databricks
• Strong understanding of cluster optimization and its integration with various cloud services
• Good understanding to create complex data pipelines
• Strong performance optimization skills to improve efficiency and reduce cost
• Extensive knowledge of Spark and Hive data processing framework
• Extensive knowledge of cloud (Azure, AWS, GCP) and common services like ADLS/S3, ADF/Lambda, Cloud databases
• Strong in writing unit test case and integration test
• Strong communication skills and have worked with cross-platform teams
• Great attitude towards learning new skills and upskilling the existing skills
• Responsible for setting best practices around Databricks CI/CD
• Understanding composable architecture to take full advantage of Databricks capabilities
• Experience with machine learning tools such as mlFlow, Databricks AI/ML, Azure ML, AWS sagemaker, etc.
• Experience in distilling complex technical challenges to actionable decisions for stakeholders and guiding project teams by building consensus and mediating compromises when necessary
• Experience coordinating the intersection of complex system dependencies and interactions
• Experience in solution delivery using common methodologies especially SAFe Agile but also Waterfall, Iterative, etc.

Preferred Qualifications

• Good to have Rest API knowledge
• Good to have understanding around cost distribution
• Good to have knowledge on migration project to build Unified data platform
• Good to have knowledge of DBT
• Experience around DevSecOps including docker and Kubernetes
• Knowledge of programming and scripting languages such as JavaScript, PowerShell, Bash etc.
• Experience with data ingestion technologies such as Azure Data Factory, SSIS, Pentaho, Alteryx
• Experience with visualization tools such as Tableau, Power BI
• Demonstrated knowledge of relevant industry trends and standards

Circle K is an Equal Opportunity Employer