Principal Software Engineer, Tech Lead - Platform

About the Role

Cardlytics is seeking a Principal Software Engineer, Tech Lead - Platform to join the team, reporting to the VP of Engineering, Cloud & Data Infrastructure. We are looking for an experienced technical leader to own and evolve our data compute environment end-to-end on AWS, including defining the next-generation platform direction. In this role, you will make architectural, design, and operational decisions that shape how analysts, engineers, and business stakeholders interact with data at Cardlytics.

As data consumption increasingly shifts toward AI and agentic applications, you will also shape the platform's evolution into an AI-ready foundation — spanning the semantic layer, feature store, and data ontology that these systems depend on. You will combine deep expertise in industry-leading data platforms with the ability to provide technical leadership, define platform standards, and act as the primary technical partner for diverse user groups across the organization.

Responsibilities

• Architecture and design: Design and evolve the Cardlytics next-generation architecture; establish reference architectures and standards for batch, streaming, ML, and AI/agentic workloads.
• AI-ready data platform: Build out the semantic layer, feature store, and data ontology that make Cardlytics data consumable by AI/agentic applications; enable governed, self-service access for both human and machine (LLM/agent) consumers.
• User enablement: Serve as the primary technical partner for analysts, engineers, and business users; develop self-service guides, onboarding materials, and best-practice documentation to reduce friction across all user groups.
• Technical leadership: Set technical direction for the platform and mentor engineers across the data and platform teams; drive engineering excellence and best practices, and manage the vendor relationship.
• Performance and reliability: Define monitoring standards and SLAs, lead performance tuning for Spark jobs, open table formats (e.g. Delta, Hudi), and SQL query engines; own incident response and root-cause analysis.
• Governance and security: Drive adoption of a unified governance layer (e.g., Databricks Unity Catalog) for centralized RBAC, column-level security, data lineage, and metadata management; implement and maintain security best practices including encryption and audit logging.
• Platform standards and governance: Define the operating model and guardrails for the managed lakehouse platforms (including Databricks on AWS) — compute policies, access model, catalog structure, and cost governance — with day-to-day administration executed by the platform and domain teams working within those standards.
• Infrastructure as code: Own the CI/CD strategy for lakehouse and data platform deployments (Terraform, GitHub Actions, and platform-native packaging such as Asset Bundles); enforce DevOps practices across engineering teams.

Requirements

• 7+ years of experience in data engineering or data platform roles, with at least 2 years in a technical lead or architecture capacity.
• Deep, hands-on expertise with the leading industry platforms, frameworks, and tools: Databricks, AWS EMR or similar, Delta Lake, Unity Catalog, Spark (PySpark), and SQL; with a solid understanding of compute/cluster management on AWS.
• Proven experience designing and delivering Lakehouse architectures and large-scale ELT/ETL pipelines in production environments.
• Proficiency in Infrastructure as Code tools, such as Terraform.
• Demonstrated experience supporting diverse stakeholder groups including analysts, engineers (including ML engineers), and non-technical business users.
• Strong understanding of data governance, data modeling, and cloud security best practices.
• Excellent communication and collaboration skills, with the ability to translate technical concepts for non-technical audiences.

Nice to Have

• Experience designing or operating a semantic layer, feature store, or data ontology / knowledge graph in a production environment.
• Familiarity with managing data for ML systems and AI/agentic applications, such as training pipelines or inference for LLM-powered and agentic tools.

Technical Environment

We primarily use macOS and Google Workspace (Docs, Sheets, Slides). While our preferred platform is macOS, we support both macOS and Windows. Familiarity with or willingness to work within this environment is required.

Core Values

Our shared values are the driving force behind everything we do. Across all roles, we are looking for teammates who embody these values:

• Customer and partner first
• Act with urgency and focus
• Integrity with our partners and data
• Accountability even when challenged
• Empowerment over hierarchy
• Growth over comfort

Benefits

• Flexible paid time off plus company holidays
• Medical, dental, and vision insurance begins on your first day
• 401(k) retirement plan with company match, plan also includes a student loan debt repayment option
• Employee Stock Purchase Plan
• Educational assistance for continuing education
• Lifestyle Spending Account for physical, emotional, and financial wellness (like gym memberships, home down payments, art classes, park passes, and more!)
• Complimentary Calm app subscriptions to support employee mental health and wellbeing