Lead Data Engineer

About the role

The Lead Data Engineer is responsible for the design, architecture and support of systems, services and applications required for the collection, storage, processing, and analysis of all forms of data to enable data-driven decisions and outcomes across the enterprise.

Responsibilities

• Work collaboratively with other engineers, data scientists, analytics teams, and business product owners in an agile environment
• Architect, build, and support the operation of Cloud and On-Premises enterprise data infrastructure and tools
• Design robust, reusable and scalable data driven solutions and data pipeline frameworks to automate the ingestion, processing and delivery of both structured and unstructured batch and real-time streaming data
• Lead the development of data APIs and data delivery services to support critical operational processes, analytical models and machine learning applications
• Aid the selection and integration of data related tools, frameworks and applications required to expand platform capabilities
• Understand and implement best practices in management of enterprise data, including master data, reference data, metadata, data quality and lineage

Qualifications

• Bachelor’s degree in computer science, math, engineering, or relevant technical field
• Five years of collective experience in the application of data engineering, data analytics, data warehousing, business intelligence, database administration and data integration concepts and methodologies
• Four years of experience architecting, building, and administering big data and real-time streaming analytics architectures in on-premises and cloud environments
• Three years of experience architecting, building and administering large-scale distributed applications
• Three years of experience with Linux operations and development, including basic commands and shell scripting
• Three years of experience executing DevOps methodologies and continuous integration/continuous delivery
• Demonstrated skills in delivery of high-quality technical project solutions
• Expertise in SQL for data profiling, analysis, and extraction
• Familiarity with data science techniques and frameworks
• Creative thinker with strong analytical skills
• Results oriented with a strong customer focus
• Ability to work in a team environment
• Strong technical communication skills
• Ability to prioritize work to meet tight deadlines
• Ability to learn and keep pace with the latest technology advances and quickly grasp new technologies to support the environment and contribute to project deliverables

Preferred Qualifications

• Master’s degree in a technical field (e.g. computer science, math, engineering)
• Understanding of big data and real time streaming analytics processing architecture and ecosystems
• Experience with data warehousing architecture and implementation, including hands on experience with source to target mappings and developing ETL code
• Experience with advanced analytics and machine learning concepts and technology implementations
• Experience with data analysis and using data visualization tools to describe data
• Relevant technology or platform certification (AWS, Microsoft, etc.)
• Software development experience in relevant programming languages (e.g. Java, Python, Scala, Node.js)
• Experience with Snowflake