Data Engineer

About the role

This role is responsible for designing, building, and maintaining data flows between corporate IT systems and ProMach's data lake. Leveraging tools such as Apache Spark and Delta Lake, you will manage data ingestion and transformation, integrate with cloud services, ensure data quality, and collaborate with analysts to deliver data-driven insights. Success in this position requires expertise in data modeling, SQL, Python, and CI/CD pipelines.

Responsibilities

• Data Pipeline Development: Design, build, and maintain scalable data pipelines using Databricks, Apache Spark, and Delta Lake.
• ETL/ELT Processes: Develop and manage robust ETL/ELT workflows for data extraction, transformation, and loading from diverse source systems. Monitor and resolve any issues with scheduled jobs that move data.
• Data Architecture and Modeling: Contribute to designing and implementing data models and architecture for data lakes and data warehouses. Provide guidance on architecture and platforms that facilitate reporting and analytics needs.
• Cloud Integration: Integrate data solutions with cloud platforms (Azure, AWS, or GCP) and related services.
• Data Quality and Governance: Implement data quality checks, monitoring, and data governance best practices to ensure consistency, accuracy, and reliability.
• Automation: Implement and maintain CI/CD pipelines to automate deployment and streamline workflows.
• Collaboration: Work closely with business analysts and stakeholders to define data requirements and deliver actionable insights.

Qualifications

• Proven experience in data engineering and cloud-based development.
• 3–5 years' experience in a similar role.
• Expertise in Databricks and the broader Databricks ecosystem (Databricks SQL, Delta Lake, Databricks Runtime).
• Proficiency in SQL, Python, and Apache Spark.
• Experience with cloud platforms such as Azure, AWS, or GCP.
• Strong understanding of data modeling, ETL/ELT, and data warehousing concepts.
• Experience with CI/CD tools and practices.
• Excellent problem-solving, analytical, and communication skills.
• Eager to learn new skills and step in to deliver where needed.
• Bachelor's Degree in Information Systems, Computer Science, or related field, or equivalent work experience.

Preferred experience

• Experience with ERP systems (SAP, Infor) and Salesforce CRM.
• Familiarity with BI tools such as Tableau.
• Manufacturing industry experience.
• Motivated, committed and energetic self-starter dedicated to providing high quality and responsive IT service.

Benefits

• Comprehensive medical and dental programs
• Life insurance
• Generous paid time off program
• Retirement savings plan with a company match
• No waiting period for benefits – eligible on your first day of employment