Staff / Principal Data Engineer
AppGate · New York, NY · 1 mo ago
HybridEngineeringFull-time
Key Responsibilities
- Own the design, build and operation of the data lake and ingestion platform end-to-end, from architecture through production reliability
- Build low-latency batch and streaming pipelines that ingest signals from internal and external sources, normalize them to a common schema, enrich them with context and serve model-ready data to the layers above
- Establish data quality, freshness, completeness, lineage and observability so the platform is trustworthy enough to automate on top of
- Own deployment, CI/CD and operational reliability of the platform on Kubernetes
- Partner with data science, product and architecture to turn the platform into a shared foundation across 360 Fraud Protection
Required Qualifications
- Extensive experience building and operating large-scale data platforms and data lakes, with comfort working at high data volumes
- Deep, hands-on expertise with Apache Spark, Apache Flink and modern big-data systems
- Proven command of best practices for building and maintaining data pipelines in both batch and streaming modes
- Strong production engineering skills across the full delivery lifecycle, including Kubernetes and CI/CD tooling, with the ability to ship end-to-end
- A track record of owning data infrastructure end-to-end with limited supervision
Preferred Qualifications
- Experience with generative AI and embedding models, including embedding pipelines, vector databases and retrieval
- A cybersecurity or threat intelligence background, with hands-on exposure to threat types such as phishing, mobile threats and malware
- Familiarity with transaction data and transaction fraud signals