Data Engineer

About the role

We are seeking a highly skilled GCP Data Engineer with strong Python expertise to design, build, and optimize scalable data solutions on Google Cloud Platform (GCP). The ideal candidate will have hands-on experience developing batch and real-time data pipelines, working with large-scale datasets, and enabling analytics and AI/ML use cases.

Responsibilities

• Data Engineering & Pipeline Development
  • Develop and optimize ETL/ELT workflows for structured and unstructured data processing using GCP services such as Dataflow, Dataproc, and Pub/Sub
  • Implement event-driven data processing using Cloud Functions and Pub/Sub
  • Build and manage data ingestion frameworks for streaming and batch data sources
• Data Storage & Processing
  • Design and optimize data lakes and data warehouses using BigQuery and Cloud Storage
  • Develop efficient data models to support analytics, reporting, and machine learning workloads
  • Optimize performance and cost of data pipelines and queries
• Development & Automation
  • Develop solutions using Python
  • Automate workflows and orchestration using Cloud Composer (Airflow)
  • Implement CI/CD pipelines and deployment automation
• Collaboration & Support
  • Collaborate with analytics, AI/ML, and business teams for data consumption needs
  • Troubleshoot data issues and perform root cause analysis
  • Continuously improve pipeline reliability, scalability, and performance

Required Technical Skills

• GCP Services: BigQuery, Dataflow, Dataproc, Pub/Sub, Cloud Functions (Gen2), Cloud Composer (Airflow), Cloud Storage, Cloud SQL
• Programming: Python (advanced)
• Query Language: SQL (advanced)
• Data Processing: Batch & Streaming architectures
• Scripting: Bash/Shell scripting
• Concepts: Data Warehousing, ETL/ELT, Data Lake / Lakehouse architectures

Preferred Skills & Experience

• Insurance Domain Experience or Certification
• Google Cloud Platform (GCP) services: Vertex AI, Cloud Run, BigQuery, Cloud Storage, IAM, Cloud Logging, Cloud Monitoring
• AI & Machine Learning concepts: Large Language Models (LLMs), Gemini Models, Agentic AI Frameworks, Prompt Engineering, RAG Architectures, Vector Search Concepts, Semantic Retrieval, AI Evaluation Frameworks
• Data & Analytics: SQL, BigQuery, Metadata Modeling, Data Pipelines

Required Experience

• 5+ years of overall data engineering or software engineering experience
• 2+ years of hands-on Google Cloud Platform experience
• 2+ years of Python development
• 2+ years of experience building data pipelines (batch and streaming)

Preferred Qualifications

• Experience with Dataproc (Spark/PySpark) for large-scale processing
• Familiarity with event-driven architectures
• Knowledge of Terraform or Infrastructure as Code
• Understanding of cost optimization (FinOps)
• Google Cloud Professional Data Engineer Certification
• Experience supporting AI/ML data pipelines

Pay

The base compensation range for this role in the posted location is: $75,000–$96,000. Actual compensation may vary based on geographic location, education, certifications, relevant experience, seniority, performance, market conditions, internal equity, and applicable laws.