Software Engineer, Data Infrastructure (Staff)

Lightfield is an AI-native CRM that assembles itself from your email, calendar, and meetings. It captures every interaction and turns it into organized context: accounts, tasks, follow-ups, and insights, so nothing slips through the cracks. We’re rethinking CRM from first principles—building a platform that learns from how companies actually work, adapting, automating, and surfacing insights that drive growth. Over 5,000 companies have used Lightfield since launch, making us the fastest-adopted product in our category. Backed by Greylock, Lightspeed, and Coatue, our team previously built Tome (used by 25M+ people) and worked on Llama, Instagram, Facebook Messenger, Pinterest, Google, and Salesforce.

About the role

We're building the infrastructure foundation for a fast-growing AI product company serving thousands of customers. Our Postgres fleet serves 5B+ queries a month (2,500 QPS steady state, spikes past 25,000 QPS), and database workload more than doubled last month. Redis sustains ~50,000 commands per second behind a job platform executing 10M+ background jobs daily across ~170 queues. We ingest tens of millions of emails and calendar events monthly.

We're hiring a staff-level engineer who excels in data infrastructure but is excited to work across backend systems, infrastructure, and product-facing data problems. The role is close to the product, customers, and production. You’ll build the next generation of our data systems, evolving our pragmatic foundation into best-practice data architecture: change data capture, event modeling, schema design, query performance, freshness guarantees, and the boundary between transactional and analytical workloads.

Our system of record is unusual: customers define their own objects, attributes, and relationships at runtime, creating a schema-flexible, graph-shaped store with typed edges, versioned attribute values, and relationship history. This makes schema design, indexing, and query performance genuinely hard problems. The surface area includes customer-facing dashboards, historical/audit data, datasets for pipeline-generation products, and evaluation data for AI agents.

This role can be based in San Francisco (HQ with founders and most of engineering) or Cambridge (new infrastructure-focused site in Kendall Square).

Responsibilities

• Scale the analytics engine behind customer-facing dashboards, tackling query performance under row-level security, workload isolation, read architecture, and observability as data volume grows.
• Design ingestion paths, event models, schemas, and query patterns to move data from transactional writes into search, dashboards, and history—with guarantees around freshness, correctness, replay, and failure recovery.
• Evolve our schema-flexible, graph-shaped data model so customer-defined objects, attributes, and relationships remain fast to query as their size and complexity grow.
• Build foundations for historical reporting and auditability, including attribute versioning, relationship history, and change capture.
• Build reliable data systems for usage metering, pipeline generation, and AI evaluation, where errors have direct customer, product, or financial consequences.
• Decide when existing architecture remains the right foundation and when new analytical, streaming, or workflow systems earn their added complexity.
• Set technical direction, abstractions, ownership boundaries, and engineering practices for data systems as the company grows.

What your first year looks like

The anchor project is scaling the analytics serving path behind customer-facing dashboards, but the work stays close to the product. Current projects include:

• Zero-downtime schema migrations for an 18-collection Typesense search deployment.
• A usage-metering pipeline for consumption billing.
• Historical and audit data modeling.
• Evaluation data infrastructure for our AI agents.

Expect a mix of foundational data systems and product-shaped projects, with scope to own the technical direction for how data is modeled, moved, and served across Lightfield—and to shape the architecture and team as the company scales.

Requirements

• 7+ years of experience.
• Strong software engineering fundamentals.
• Experience owning production data systems where query plans, replication lag, backfills, data freshness, schema evolution, or data correctness had real user-facing consequences.
• Comfort debugging across multiple layers of the stack.
• Good judgment about when to make tactical fixes vs. invest in durable platform/architecture changes.
• Product orientation: care about how data infrastructure decisions affect customers, users, and engineering velocity.
• Clear communication, strong ownership, and a bias toward practical tradeoffs.

Skills

Helpful but not required:

• ClickHouse, OLAP systems, event pipelines, data warehouses, or analytical infrastructure.
• Kafka, Flink, Spark, Iceberg, or similar streaming and lakehouse systems.
• Postgres at scale, and the boundary between OLTP and OLAP systems.
• APIs, queues, workflow systems, and distributed systems.
• Observability, incident response, service ownership, and production debugging.
• Data for ML/AI systems: enrichment pipelines, eval harnesses, or data-quality tooling.
• Experience in a high-growth product environment.

Why this role is interesting

You’d be building our analytical data architecture from close to the beginning—the foundations are deliberately simple, and the architecture that scales them is yours to shape. Customer-facing data products are on the roadmap, database workload more than doubled last month, and the foundations you build will carry the company for years.

Benefits

• Competitive salary and meaningful early equity.
• Health insurance (medical, dental, vision).
• 3 weeks of PTO + 11 paid company holidays + winter holiday break.
• 3 months of paid family leave.
• Wednesdays work from home.
• Regular team dinners, events, offsites, and retreats.
• 401k plan.
• Commuter and lunch stipend.

Pay

Compensation range: $180K - $300K.