Senior Software Engineer, Data Infrastructure
Decagon · New York, NY · 3 wk ago
On-siteEngineering$200k–$400k/yrFull-time
About the role
We're hiring a Senior Data Infrastructure Engineer to design, build, and operate the data systems that power Decagon's AI products. You'll own critical data pipelines and storage layers end-to-end, improve reliability and performance, and create paved paths that let every Decagon engineer work confidently with data at scale.
Responsibilities
- Design and implement high-throughput data pipelines and streaming systems with strong SLOs, clear runbooks, and actionable telemetry.
- Build and operate real-time and batch ingestion infrastructure using tools like Kafka, Flink, and Airflow.
- Own our analytical data layer — schema design, query performance, and cost optimization across ClickHouse, BigQuery, or similar.
- Partner with research and product teams to architect data solutions, evaluate performance, and scale new features.
- Tune pipeline and query latencies: optimize data paths, apply smart caching/partitioning, and hit tight p95/p99 targets.
- Lead infrastructure-as-code (Terraform) and GitOps practices for data systems; reduce drift with reusable modules and policy-as-code.
- Participate in on-call and drive down toil through automation and elimination of recurring data issues.
Requirements
- 5+ years building and operating production data infrastructure at scale.
- Hands-on experience with Tier 1 data technologies: ClickHouse, Kafka (or MSK/Pub-Sub/RabbitMQ), and Flink or dbt.
- Proven track record meeting high availability and low latency targets across streaming and batch workloads.
- Excellent observability chops (OpenTelemetry, Prometheus/Grafana, Datadog) and strong incident response discipline.
- Clear written communication and the ability to turn ambiguous data requirements into simple, reliable designs.
- Experience with CDC tooling (Debezium) and orchestration frameworks (Airflow, Dagster, or Prefect).
- Familiarity with Spark or Dask for large-scale data processing.
- Experience with cloud data warehouses (Snowflake, BigQuery, Redshift, Databricks).
- Experience being an early data/platform/infrastructure engineer at another company.
- Strong Kubernetes experience (GKE/EKS/AKS) and multi-cloud exposure (GCP, AWS, Azure).
Qualifications
- Experience with customer-managed deployments.
- Compensation range: $200K – $400K + offers equity.
- Final compensation may vary based on location within the United States.
Benefits
- Take what you need vacation policy (subject to local requirements; UK employees receive 25 days of statutory leave).
- Medical, Dental, and Vision benefits for you and your family.
- Life Insurance and Disability Benefits.
- Retail Plan (e.g., 401K, pension).
- Parental Leave.
- Fertility and family building benefits through Carrot.
- Daily lunches and snacks in the office to keep you at your best.