Data Engineer

About the role

UST is seeking a skilled Data Engineer with 6+ years of experience in designing, developing, and supporting scalable data pipelines and enterprise data platforms. The ideal candidate will have strong expertise in Google Cloud Platform (GCP), Apache Airflow DAGs, Java/Spark development, and SQL-based data engineering to support large-scale analytics and data transformation initiatives.

Responsibilities

• Develop reusable data processing frameworks and components ensuring data quality, consistency, and reliability across data platforms.
• Optimize Spark jobs for performance, scalability, and cost efficiency.
• Code reviews, testing, deployment, and production support activities of monitoring & troubleshooting airflow workflow and production pipelines.
• Design, develop, and maintain scalable ETL/ELT pipelines to ingest, transform, and load data from multiple sources into data warehouses and data lakes.
• Build and optimize data solutions using SQL, Python, Spark, and cloud technologies such as Azure, AWS, or GCP.
• Develop and manage data integration workflows using tools like Azure Data Factory, Databricks, Airflow, or Informatica.
• Ensure data quality, integrity, security, and governance through validation, monitoring, and compliance best practices.
• Collaborate with business stakeholders, data analysts, and data scientists to deliver reliable, analytics-ready datasets.
• Maintain and optimize data pipelines and database performance to support high-volume data processing and reporting requirements.

Requirements

• Experience range: 7-10 Years
• Technical Skills: Mandatory: BigQuery, Apache Airflow, Spark/Java/ PySpark
• Nice to have: GCP, GCS, Relational Databases, Data Modeling, Kafka, Terraform
• Skills and experiences: Should have 6+ years of experience in Data Engineering with demonstrated hands-on experience of 4+ years of experience in Design, develop, and maintain scalable data pipelines on Google Cloud Platform (GCP)
• 3+ years of recent experience in Develop, schedule, and manage workflows using Apache Airflow DAGs
• 3+ years of experience in Build and optimize batch and streaming data processing solutions using Java and Apache Spark.
• 4+ years of experience with writing complex and optimized SQL queries for data extraction, transformation and reporting
• Implement ETL/ELT processes for ingesting, transforming, and loading large-scale datasets.

Qualifications

• Experience in high-volume data integration and migration initiatives.
• Experience in query performance through indexing, portioning and data model optimization.
• Experience in building and optimizing data solutions using SQL, Python, Spark, and cloud technologies such as Azure, AWS, or GCP.
• Experience in developing and managing data integration workflows using tools like Azure Data Factory, Databricks, Airflow, or Informatica.
• Experience in ensuring data quality, integrity, security, and governance through validation, monitoring, and compliance best practices.
• Experience in collaborating with business stakeholders, data analysts, and data scientists to deliver reliable, analytics-ready datasets.
• Experience in monitoring, troubleshooting, and optimizing data pipelines and database performance to support high-volume data processing and reporting requirements.

Skills and experiences

• Should have 6+ years of experience in Data Engineering with demonstrated hands-on experience of 4+ years of experience in Design, develop, and maintain scalable data pipelines on Google Cloud Platform (GCP)
• 3+ years of recent experience in Develop, schedule, and manage workflows using Apache Airflow DAGs
• 3+ years of experience in Build and optimize batch and streaming data processing solutions using Java and Apache Spark.
• 4+ years of experience with writing complex and optimized SQL queries for data extraction, transformation and reporting
• Implement ETL/ELT processes for ingesting, transforming, and loading large-scale datasets.

Benefits

• Full-time, regular employees accrue a minimum of 10 days of paid vacation per year, receive 6 days of paid sick leave each year (pro-rated for new hires throughout the year), 10 paid holidays, and are eligible for paid bereavement leave and jury duty.
• Eligible to participate in the Company’s 401(k) Retirement Plan with employer matching.
• Eligible to participate in medical, dental, and vision insurance, as well as the following Company-paid Employee Only benefits: basic life insurance, accidental death and disability insurance, and short- and long-term disability benefits.
• Eligible to purchase additional voluntary short-term disability benefits, and participate in a Health Savings Account (HSA) as well as a Flexible Spending Account (FSA) for healthcare, dependent child care, and/or commuting expenses as allowable under IRS guidelines.