Senior Data Engineer

About the role

We are seeking an experienced Senior Data Engineer to design, develop, test, and maintain scalable data solutions that support business and customer needs. This role is responsible for designing data architectures, building robust data pipelines, optimizing large-scale data systems, and collaborating with cross-functional teams throughout the software development lifecycle.

Responsibilities

• Design, develop, test, and maintain scalable software applications and data solutions.
• Engage with stakeholders to understand business requirements and translate them into effective technical solutions.
• Design and implement scalable data architectures, including data warehouses, data lakes, and streaming platforms.
• Develop, maintain, and optimize ETL/ELT pipelines for processing large volumes of data from multiple sources.
• Assess application requirements and determine appropriate technology stacks, deployment strategies, and integration methods.
• Create high-level software architecture designs, including system components and interfaces.
• Collaborate with product owners, architects, designers, and cross-functional teams.
• Define and implement software testing strategies, policies, and processes.
• Improve application performance and development efficiency by adopting new technologies and best practices.
• Apply modern software engineering practices, including test-driven development, continuous integration, and continuous delivery.
• Support security practices to protect data through encryption and anonymization.
• Develop user-friendly interfaces where applicable.
• Develop and maintain backend services, APIs, and microservices.
• Evaluate emerging technologies and contribute to continuous improvement initiatives.
• Ensure data integrity, quality, scalability, and performance across all data pipelines.

Requirements

• Bachelor's degree.
• 5+ years of progressive experience in Data Engineering or a related field with a strong focus on large-scale data systems.
• Expert proficiency in Python or Scala.
• Advanced SQL skills, including complex querying, data manipulation, and database design.
• Experience with big data technologies such as Apache Spark, Hadoop, Kafka, and Flink.
• Experience with cloud platforms including AWS, Azure, or Google Cloud Platform (GCP) and their associated data services.
• Strong understanding of data warehousing concepts, data modeling (dimensional and relational), and ETL/ELT processes.
• Experience with NoSQL databases.
• Experience with Kubernetes, Docker, and cloud infrastructure technologies.
• Experience with version control systems such as Git.
• Strong analytical, problem-solving, and critical thinking skills.
• Excellent communication and interpersonal skills with the ability to present technical information clearly.

Preferred Skills

• GitHub
• Machine Learning

Technical Skills

• Python
• Java
• SQL
• Apache Spark
• Apache Hadoop
• Kafka
• BigQuery
• Google Cloud Platform (GCP)
• Kubernetes
• Docker
• NoSQL
• Cloud Architecture
• Data Architecture
• Cloud Infrastructure
• Big Data
• Data Management

Benefits

• Group Health (Medical, Dental, and Vision)
• Paid Time Off
• Paid Holidays
• 401(k) matching
• Group Life and Disability insurance
• Professional Development opportunities
• Wellness programs
• Variety of other perks