Jobs · Engineering · New York

Staff / Principal Data Engineer

AppGate · New York, NY · 1 mo ago
HybridEngineeringFull-time

Key Responsibilities

  • Own the design, build and operation of the data lake and ingestion platform end-to-end, from architecture through production reliability
  • Build low-latency batch and streaming pipelines that ingest signals from internal and external sources, normalize them to a common schema, enrich them with context and serve model-ready data to the layers above
  • Establish data quality, freshness, completeness, lineage and observability so the platform is trustworthy enough to automate on top of
  • Own deployment, CI/CD and operational reliability of the platform on Kubernetes
  • Partner with data science, product and architecture to turn the platform into a shared foundation across 360 Fraud Protection

Required Qualifications

  • Extensive experience building and operating large-scale data platforms and data lakes, with comfort working at high data volumes
  • Deep, hands-on expertise with Apache Spark, Apache Flink and modern big-data systems
  • Proven command of best practices for building and maintaining data pipelines in both batch and streaming modes
  • Strong production engineering skills across the full delivery lifecycle, including Kubernetes and CI/CD tooling, with the ability to ship end-to-end
  • A track record of owning data infrastructure end-to-end with limited supervision

Preferred Qualifications

  • Experience with generative AI and embedding models, including embedding pipelines, vector databases and retrieval
  • A cybersecurity or threat intelligence background, with hands-on exposure to threat types such as phishing, mobile threats and malware
  • Familiarity with transaction data and transaction fraud signals

Similar jobs

Principal Data Engineer

MidAmerican Energy CompanyDes Moines, IA· 1 mo ago
Information Technologyapply on fa-essf-saasfaprod1.fa.ocs.oraclecloud.com

Principal Data Engineer

SovrnBoulder, CO· 3 wk ago
Information Technology$200k–$240k/yrapply on job-boards.greenhouse.io

Principal Data Engineer

Fitch RatingsChicago, IL· 2 wk ago
Information Technology$140k–$160k/yrapply on careers.fitch.group

Principal Data Engineer

KemperDowners Grove, IL· 1 wk ago
Information Technology$112k–$187k/yrapply on kemper.wd5.myworkdayjobs.com