Senior Staff Data Engineer

About the role

The Data Platform team builds and operates BILL’s core data infrastructure, providing the end-to-end foundation that collects, stores, processes, governs, and serves data so every team at BILL can use it. We own the full data stack: inbound and outbound data lake, real-time streaming pipelines, batch processing, and data access layers including a Starburst query engine, Databricks Feature Store, Neo4j Knowledge Graph, and OpenSearch. Some capabilities require real-time access with strict low-latency SLAs. Our charter is to simplify the data landscape and power AI at BILL. To achieve this, we focus on building scalable platform capabilities rather than creating one-off pipelines or analytics reports. Engineers here work at the systems level: designing architectures, incubating new capabilities, setting standards, and enabling the rest of BILL to self-serve. The team sits within the CTO organization.

Responsibilities

• Own the outcome for our data architectural, driving platform-wide technical decisions and mentoring the team.
• Own and evolve critical infrastructure across the full data lifecycle, spanning ingest (batch and streaming), store, enrich, query, and serve.
• Architect and own critical data platform capabilities end-to-end, from inbound ingestion through data lake storage to downstream serving, including the feature store, query engine, knowledge graph, and search.
• Define technical direction for the team’s most complex, cross-cutting problems, such as streaming versus batch trade-offs, schema contracts, data access patterns, and real-time serving architectures.
• Drive design and delivery of new capabilities from inception to GA, including reference implementations, SLAs, and clear ownership handoff models.
• Establish and maintain architectural standards and engineering patterns adopted across the organization.
• Lead multi-phase technical migrations at enterprise scale, including compute platform upgrades, warehouse-to-lake migrations, and infrastructure modernization.
• Partner with engineering teams across BILL, such as ML/AI, Risk, Payments, and Analytics, to translate diverse data needs into durable platform solutions.
• Mentor senior and staff engineers, actively shaping the technical culture and engineering quality bar of the team.
• Own and continuously improve critical production systems with a focus on reliability, cost efficiency, and a self-serve developer experience.

Requirements

• Bachelor’s degree in Computer Science, Engineering, Mathematics, or a related field, or equivalent practical experience.
• 8+ years of experience in data engineering, including at least 5 years building and operating modern data platforms or data infrastructure.
• Demonstrated expertise designing, architecting, and operating large-scale distributed data platforms, with a deep understanding of streaming, batch, and real-time data processing architectures and their associated trade-offs.
• Strong experience designing and operating cloud-native data platforms on AWS, including storage, compute, networking, security, and automation, with the ability to make architectural decisions that balance scalability, reliability, performance, security, and cost.
• Hands-on experience building data ingestion platforms using event streaming technologies such as Kafka, Flink, Spark Streaming, or equivalent, including Change Data Capture (CDC) patterns and event-driven architectures.
• Strong proficiency with large-scale batch processing frameworks such as Spark, Airflow, AWS Glue, dbt, or equivalent technologies.
• Experience designing and implementing modern data lake architectures using open table formats such as Apache Iceberg, Delta Lake, or equivalent technologies.
• Experience with data serving and consumption technologies, including distributed SQL query engines (Trino/Starburst, Presto)
• Expert-level SQL and strong Python programming skills, supported by solid software engineering practices including testing, CI/CD, code quality, version control, observability, and operational excellence.
• Strong experience implementing Infrastructure as Code using Terraform, AWS CDK, or equivalent technologies to provision, manage, and evolve cloud infrastructure.