Senior Data Engineer

Primary Responsibilities

• Participate actively in Agile/Scrum ceremonies and deliver assigned user stories and features in line with defined timelines
• Collaborate closely with cross-functional teams, including architects and product stakeholders, to understand requirements and contribute to solution design
• Support ongoing product operations by addressing production issues, performing root cause analysis, and ensuring timely resolution
• Develop a strong understanding of product architecture and features, and proactively suggest enhancements, improvements, and proof of concepts (POCs)
• Design, build, and maintain scalable data pipelines using Azure Data Factory and Azure Databricks
• Perform data analysis activities including source data analysis, data profiling, and data mapping to support data integration and transformation needs
• Develop and optimize complex SQL queries to support data processing and reporting requirements
• Contribute to the development and maintenance of large-scale data warehouse solutions
• Support data migration initiatives, including transitioning from legacy/on-premises systems to cloud-based platforms (e.g., Azure)
• Build and support applications using Microsoft Power Apps where required
• Continuously learn and adapt to new data technologies and frameworks to enhance team capability and solution delivery

Required Qualifications

• High School Diploma or equivalent experience (Bachelor's degree preferred in Computer Science, Engineering, or related field)
• 6+ years of experience in data engineering, including DevOps practices, Big Data technologies, Apache Spark, and Azure Cloud
• 6+ years of hands-on experience with large-scale data processing using PySpark within the Azure ecosystem
• Programming Languages: Python, PySpark
• Cloud Technologies: Azure (ADF, Databricks, Synapse, SQL Server, Web Apps, Key Vault, Functions, Logic Apps, Azure ML, DevOps)
• Machine Learning & AI: ML models, GPT, NLP algorithms
• 5+ years of experience designing and implementing ETL frameworks for self-service analytics platforms using PySpark on Azure
• 5+ years of experience building and maintaining scalable data pipelines using Azure Data Factory (ADF) and Azure Databricks
• 5+ years of hands-on experience working with large-scale data warehouse platforms
• 5+ years of experience with Azure data processing tools, including Azure Data Factory and Azure Databricks
• 4+ years of experience building and deploying machine learning models, including familiarity with ML lifecycle management
• 4+ years of experience in data migration projects, including transitioning from on-premises systems to cloud platforms (Azure preferred)
• 3+ years of experience or working knowledge of Generative AI concepts, including LLMs, RAG architectures, GPT, and NLP algorithms

Pay

Pay is based on several factors including but not limited to local labor markets, education, work experience, certifications, etc. In addition to your salary, we offer benefits such as, a comprehensive benefits package, incentive and recognition programs, equity stock purchase and 401k contribution (all benefits are subject to eligibility requirements). No matter where or when you begin a career with us, you'll find a far-reaching choice of benefits and incentives. The salary for this role will range from $91,700 to $163,700 annually based on full-time employment.