Snowflake Data Engineer

Data Engineering

• Design and implement enterprise-grade data pipelines using Snowflake, including ingestion and transformation
• Must be strong in both Core and Semantic aspects
• Develop complex SQL transformations, stored procedures and Dynamic tables inside Snowflake to enable near real-time and batch processing
• Implement Snowflake data sharing, data marketplace integration
• Engineer Snowpipe and Kafka-to-Snowflake streaming ingestion pipelines also handling high throughput event data at scale
• Optimize Snowflake cluster performance – virtual warehouse sizing, query profiling, clustering keys
• Architecture, design aspects, performance tuning, time travel, warehouse concepts - scaling, clustering, micro-partitioning

Data Integration

• Experience with Apache Airflow for designing and maintaining end-to-end ETL/ELT pipelines
• Experience in building reusable parameterized data ingestion pipelines/frameworks is beneficial
• Thorough on data quality checks

AI and Data Science

• Integrate AI/LLMs with data pipelines via Python UDFs or API callouts – enabling text analytics, semantic search and GEN-AI augmented workflow
• Experience with Python based frameworks – scikit learn, PyTorch, TensorFlow
• Experience with NLP and text-mining techniques on unstructured data to identify actionable information
• Time-series forecasting, anomaly detection and propensity modeling

Data Visualization

• Hands-on experience with writing Complex queries using – Joins, Self Joins, Views, Materialized Views, Cursor also Recursive, use of GROUP BY, PARTITION BY functions / SQL Performance tuning
• Hands-on experience with ETL and Dimensional Data Modelling – Slowly Changing Dimensions (SCD – Type 1, 2, 3)
• Good understanding of concepts like schema types, table types - fact-dimension etc. like how to design a dimension vs fact, design considerations factored etc.

Programming and Automation

• Proficiency in Python scripting/programming – using Pandas, PyParsing, Airflow
• Pandas, Tableau server modules, Numpy, Datetime, Apache Airflow related modules, API
• Data Pipeline automation
• Strong Python programming skills

Collaboration and Business Understanding

• Actively participating in discussions with business to understand requirements, perform thorough impact analysis and provide suitable solutions.