Jobs · Information Technology · California

Engineering Manager, Data Knowledge Platform

Airwallex · San Francisco, CA · 2 wk ago
HybridInformation Technology$200k–$288k/yrFull-time

About the role

The Knowledge Platform team is at the heart of our company's data and AI strategy. We are building the foundational infrastructure that empowers the entire company to leverage data, AI, and ML into business impact. This includes managing our data infrastructure (Databricks, Spark, Kafka, etc.), the technology to serve that data to our users (RAG, MCP, etc.), and the platform to host and govern these AI/ML models.

In 2026, our team’s overarching mission is to evolve our full data ecosystem—encompassing both platform and models—into a fully AI agent-ready infrastructure; we will empower customers to engage directly with the data platform to extract actionable value through capabilities like analytics and natural language querying, while also upgrading the platform to deliver robust, real-time performance for instant, data-driven decision-making.

Responsibilities

  • Provide visionary technical leadership and define a clear 1-3 year strategic roadmap for the Realtime Data Platform.
  • Lead the multi-year effort to modernize our core data platform, introducing real-time analytical processing at petabyte scale.
  • Partner with product teams to enable new data-driven features, such as AI-powered applications or real-time dashboards, by ensuring the underlying platform capabilities are in place.
  • Successfully scale and structure the engineering teams, including hiring new talent and mentoring managers and senior individual contributors.
  • Cultivate and maintain strong relationships with product and other engineering teams, serving as a trusted technical advisor on all things Data and AI.

Requirements

A minimum of 8 years of experience in data or software engineering, with at least 3 years in a leadership or management role.

Proven experience successfully managing and scaling an engineering team of 10+ people, including managers and senior individual contributors.

Demonstrated ability to define and execute a technical strategy that led to a significant business outcome or operational improvement.

Strong command of the data and software engineering domains, with a focus on architecture and strategy.

Deep understanding of distributed data processing technologies (e.g., Apache Spark, Databricks) and event streaming (e.g., Kafka).

Solid knowledge of modern data storage and serving technologies, such as CubeJS, ElasticSearch, and Clickhouse, and more.

Familiarity with observability tooling such as Splunk, Grafana, and Prometheus.

Qualifications

  • Experience within the financial domain.
  • Hands-on design experience in crafting data processing patterns for a modern Lakehouse architecture.
  • Excellent written and verbal communication skills tailored for diverse audiences (leadership, users, company-wide).
  • Ability to rapidly evaluate various technologies and conduct proof of concepts to drive architecture design.
  • Experience thriving in a complex environment.

Similar jobs