Data Engineer III

Key Responsibilities

• Design, develop and implement reliable and effective data solutions based on business requirements.
• Maintain process design artifacts like data flow diagrams, end user process maps and technical design documents.
• Find trends in data sets and develop algorithms to help make raw data more useful to IEHP.
• Create and maintain optimal data pipeline architecture that meet security standards.
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability.
• Develop best practices for database design and development activities.
• Create complex functions, scripts, and services to support the Data Services team.
• Ensure all data solutions meet company and performance requirements.
• Work under minimal supervision with wide latitude for independent judgment.
• Conduct code reviews.
• Direct and mentor some level I and II engineers.
• Serve as a subject matter expert in key business projects.
• Recommend improvements to existing Data Services processes as necessary.
• Provide detailed analysis of data issues; data mapping; and the process for automation and enhancement of data quality.
• Analyze and integrate new technologies with existing applications to improve the design and functionality of applications.
• Maintain proficient programming skills in REST web services, C#.NET, TSQL, XML, JSON, and other relevant languages and/or frameworks.
• Develop and automate solutions to consume data from multiple data sources, including external API
• Program and modify code in languages like Java, Json, Python, and Spark to support and implement Data Warehouse solutions.
• Design and deploy enterprise-scale cloud infrastructure solutions.
• Research, analyze, recommend and select technical approaches for solving difficult and meaningful development and integration problems.
• Work closely with the Data and Engineering teams to design best in class Azure implementations.
• Clearly and regularly communicate with management, colleagues, and domain units.

Qualifications

• Minimum of eight (8) years of experience in provisioning, configuring, and developing solutions in Azure Data Lake, Azure Data Factory, Azure SQL Data Warehouse, Azure Synapse and Cosmos DB.
• Eight (8) years implementing software development methodologies.
• Eight (8) years working with relational databases. Experience building and optimizing big data pipelines, architectures, and data sets.
• Experience performing root cause analysis on internal and external data and processes. Experience using Source Control and management tools such as Azure DevOps and Gitlab/GitHub. Experience transforming requirements into Design Concepts and ERDs using Visio and similar tools.
• Of the total eight, a minimum of five (5) years of hands-on experience with cloud orchestration and automation tools and CI/CD pipeline creation is required.
• Bachelor’s degree in a quantitative discipline such as Computer Science, Statistics, Mathematics or Engineering from an accredited institution required.