Sr. Data Engineer

Primary Responsibilities & Essential Functions

• Design, implement, and maintain the data pipeline architecture for our business
• Assemble large, complex data sets that meet functional / non-functional business requirements
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and GCP or AWS 'big data' technologies
• Build analytics tools that utilize the data pipeline to provide actionable insights into sensor behavior, application usage, operational efficiencies, and other key business performance metrics
• Work across vertical stakeholders to assist with data-related technical issues as well as supporting their evolving data infrastructure needs
• Create the data tools and frameworks for our analytics and data science team that assist them in building industry-leading insights and solutions

Skills & Qualifications

• Minimum 5+ years of experience in a data engineer (or similar) role
• Advanced experience working with both relational & non-relational databases, including Postgres, BigQuery, and Firebase
• Experience with big data tools: Hadoop, Spark, Kafka, etc.
• Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
• Experience with stream-processing systems: Storm, Spark-Streaming, etc.
• Experience with object-oriented/object function scripting languages: Python, Kotlin, etc.
• Experience with machine learning frameworks (like Keras or PyTorch) and libraries (like scikit-learn)
• Advanced working SQL knowledge and query authoring
• Experience building and optimizing 'big data' data pipelines, architectures and data sets
• Strong track record of manipulating, processing and extracting value from large, disconnected datasets
• Working knowledge of message queuing, stream processing, and highly scalable 'big data' data stores
• Strong project management and organizational skills
• Experience supporting and working with cross-functional teams in a dynamic environment