Senior Data Engineer

Responsibilities

• Design, develop, and maintain real-time or batch data pipelines to process and analyze large volumes of data.
• Develops and maintains scalable data pipelines, ensuring data quality, and deploys machine learning models to production.
• Collaborates with business teams to improve data models that feed business intelligence tools, increasing data accessibility and fostering data-driven decision-making across the organization.
• Builds real-time and batch pipelines to handle large volumes of data efficiently.
• Collaborates with cross-functional teams and translates business requirements into scalable data solutions.

Requirements

• Bachelor’s degree in computer science, information systems, data science, management information systems, mathematics, physics, engineering, statistics, economics, and/or a related field required.
• Master’s degree in computer science, information systems, data science, management information systems, mathematics, physics, engineering, statistics, economics, and/or a related field preferred.
• Minimum of eight (8) years of experience as a data engineer with full-stack capabilities.
• Minimum of ten (10) years of experience in programming.
• Minimum of five (5) years in Cloud technologies like Azure, Aws or Google.
• Strong SQL Knowledge.
• Experience in ML and ML Pipeline a plus.
• Experience in real-time integration, developing intelligent apps and data products.
• Proficiency in Python and experience with CI/CD practices.
• Strong background in IAAS platforms and infrastructure.
• Hands-on experience with Databricks, Spark, Fabric, or similar technologies.
• Experience in Agile methodologies.
• Hands-on experience in the design and development of data pipelines and data products.
• Experience in developing data ingestion, data processing, and analytical pipelines for big data, NoSQL, and data warehouse solutions.
• Hands-on experience implementing data migration and data processing using Azure services: ADLS, Azure Data Factory, Event Hub, IoT Hub, Azure Stream Analytics, Azure Analysis Service, HDInsight, Databricks Azure Data Catalog, Cosmo Db, ML Studio, AI/ML, etc.
• Extensive experience in Big Data technologies such as Apache Spark and streaming technologies such as Kafka, EventHub, etc.
• Extensive experience in designing data applications in a cloud environment.
• Intermediate experience in RESTful APIs, messaging systems, and AWS or Microsoft Azure.
• Extensive experience in Data Architecture and data modeling.
• Knowledgeable with BI tools such as Power BI and Tableau.

Qualifications

May occasionally work evenings and/or weekends.

Pay

The anticipated pay range/scale for this position is $121,116.00 to $151,395.00 Annually. Actual starting base pay within this range will depend on factors including geographic location, education, training, skills, and relevant experience.

Benefits

This position is eligible to receive a discretionary annual bonus. Perks and Benefits include medical, dental and vision insurance; flexible spending accounts and/or health savings accounts; dependent savings accounts; 401(k) with company matching contributions; employee stock purchase plan; and a tuition reimbursement program. The Company provides 9 paid holidays per year, and, upon hire, new employees will accrue paid time off (PTO) at a rate of 0.0577 hours of PTO per hour worked, up to a maximum of 120 hours per year.