Sr. Data Engineer

Job Duties

• Aid and lead the design, development, and maintenance of scalable, high-performance data solutions supporting analytics, AI, and business intelligence.
• Design and document technical requirements for data flows between operational systems and data warehouse.
• Support end-to-end development of ETL/ELT processes using Apache Spark on Databricks.
• Ingest data into data lakes and warehouses; implement Delta Lake for transactional integrity; orchestrate batch and streaming data pipelines.
• Collaborate with cross-functional technology teams to extract, transform, and load data from various sources, build and optimize data models, and maintain software applications.
• Explore and integrate emerging AWS technologies to enhance data engineering capabilities.
• Write complex SQL queries for data validation and transformation; manage Databricks workspaces, clusters, and jobs.
• Support data quality initiatives and ensure compliance with governance and security policies.
• Participate in development and operationalization of machine learning and generative AI applications.
• Troubleshoot production workflows, refactor legacy systems, participate in code reviews, and mentor junior engineers.

Requirements

• Master’s Degree in Computer Science or related field and seven (7) years of experience in the job offered or a related occupation.
• Experience must include seven (7) years in the following skills:
• Data engineering with Databricks and AWS
• Apache Spark, SQL, and Python or Scala
• Delta Lake, Unity Catalog, and cloud data warehouses (e.g., Redshift)
• Data modeling, ETL/ELT processes, and data governance
• Orchestration tools including Gitlab or Bitbucket and CI/CD practices
• Collaborating with cross-functional teams to deliver data solutions using Python and Databricks in Agile environments
• Designing and optimizing scalable data pipelines with Python and Spark in Databricks
• Developing modular, reusable components in Databricks notebooks and workflows
• Implementing data validation frameworks to ensure pipeline accuracy and consistency

Pay

$141,336 - $184,400/year

Contact

To apply, please send resumes to [email protected]. Reference #: 002115.