Data Engineer

Responsibilities

• Understand a wide range of different types of data systems across the organization, including but not limited to SQL Server, Teradata, Azure SQL Data Warehouse and Snowflake environments
• Interpret data models both logical and physical
• Ingest data into data lake and data warehouses on technologies such as Snowflake, Teradata, and Azure Synapse
• Build scalable and reusable data cleansing algorithms
• Transform and merge data using data processing tools like Databricks, spark
• Understand, comply, and build security features (Encryption, masking, and tokenization) on data technology platforms
• Design simplified data access patterns for data consumers e.g. building API’s, enabling self-serve etc.
• Ensure continuous integration and continuous deployment practices are followed
• Perform peer code and performance reviews
• Collaborate with architects and other software engineers on design
• Collaborate/Partner with other teams including data science, business, and business intelligence on successful deliveries

Skill Requirements

• Advanced skills on following categories of tools
• - Cloud Platforms – Azure, AWS (Anyone)
• - Data Tools and Workload Processors – Databricks, Snowflake
• - Agile Frameworks – Kanban, Scrum
• - Data Lake, Data Lakehouse and Data Warehouse concepts and tools preferable on the cloud e.g. Azure Data Lake (ADLS), S3, Snowflake, Databricks Lake-house etc.
• - Building data pipelines via tools like Apache Nifi, Azure Data Factory, Apache Spark
• - Programming Languages – Java, Scala, or Python (any 2)
• - Knowledge of data architecture and design patterns
• - Strong interpersonal skills
• - Excellent presentation, communication, and organizational skills
• - Strong attention to detail
• - Ability to work in a fast-paced environment
• - Ability to work both independently and as part of a team