Data Engineer IV

Core Responsibilities

• Create and maintain scalable, reliable, consistent and repeatable systems that support data operations for Reporting, Analytics, Applications, and Data Science
• Gather and process raw data at scale (including writing scripts, web scraping, calling APIs, write SQL queries, etc.)
• Use ETL processes in order to maintain, improve, clean, and manipulate data
• Profile data to measure quality, integrity, accuracy, and completeness
• Develop and implement tools, scripts, queries, and applications for ETL/ELT and data operations
• Design, build, and automate Machine Learning Data Pipeline
• Deliver solutions by developing, testing, and implementing code and scripts. Produce reports and uphold data delivery schedules
• Manage life cycle of multiple data sources
• Work closely with stakeholders on the data demand side (analysts and data scientists)
• Increase speed to delivery by implementing workload/workflow automation solutions
• Perform other duties as assigned

Requirements

• Bachelor's degree in an engineering discipline or computer science
• 3+ years of Linux/Unix/CentOS system admin experience
• 5+ years of Hands-on working experience with RDBMS, SQL, scripting, and coding
• Extensive experience with Apache Airflow
• Extensive coding/scripting experience using Python, R, shell scripts
• Extensive experience with SQL, Tableau, ML Pipeline techniques, and ETL techniques
• Extensive background in Linux/Unix/CentOS installation and administration; Windows experience preferred
• Extensive knowledge in data storage that demonstrates knowledge of when to use a file system, relational database, or NoSQL variant
• Extensive experience with Spark, and Hadoop/Hive
• Extensive familiarity with JavaScript API, Rest API or Data Extract APIs
• Extensive experience receiving, converting, and cleansing big data
• Extensive experience with visualization or BI tools, such as Tableau
• Extensive experience with data virtualization concepts, and software (Denodo, Teiid, JBoss)
• Extensive experience with data workflow/data prep platforms, such as Infomatica, Pentaho, or Talend
• Able to identify and resolve end-to-end performance, network, server, and platform issues
• Effective attention to detail with the ability to effectively prioritize and execute multiple tasks
• Expert knowledge of best practices and IT operations in an always-up, always-available service
• Extensive experience receiving, converting, and cleansing data
• Ability to use a wide variety of open-source technologies and cloud services
• Ability to create proof of concept experiments for analytics, machine learning, or visualization tools that include hypothesis, test

Benefits

• Medical, Dental, & Vision Insurance Plans
• Employee Stock Ownership Plan
• 401K offered