Senior Data Platform Engineer — Kafka & PostgreSQL
for each heading. Use - for any list of duties/requirements/benefits, and
for prose. Do NOT repeat job title as heading. Do NOT put Job ID/requisition numbers as headings. Preserve all substantive info. Remove boilerplate, navigation, legal, EEO, etc. Also remove application instructions. The posting includes: company description paragraph at top (Mirantis...). Then "Job Description" heading then description. Then "Main Responsibilities" bullet list. Then "Qualifications" with Must Have and Nice to Have. Then "Education And Experience". Then "Additional Information" about benefits.
We need to decide which sections to include based on content. We have About the role (maybe the company description and job description summary). Responsibilities (Main Responsibilities). Requirements (maybe Must Have). Qualifications (maybe Nice to Have? Actually Qualifications section could contain both Must Have and Nice to Have? The instruction: Use headings for section headings (e.g. About the role, Responsibilities, Requirements, Qualifications, Skills, Benefits, Pay, Schedule). So we need to map content accordingly.
We have: About the role: could be the introductory paragraph describing Mirantis and the role purpose. Could combine the first paragraph and the "Job Description" paragraph.
Responsibilities: the Main Responsibilities bullet list.
Requirements: maybe the "Must Have" bullet list (the required qualifications). Qualifications: maybe the "Nice to Have" bullet list? Or we could treat Qualifications as the "Education And Experience" plus maybe other qualifications. But we have both Requirements and Qualifications headings; we need to decide.
The instruction: "Prefer a few substantive sections over many thin ones." So we might combine similar content. But we need to include sections only when posting has real, specific content for it. We have content for About the role, Responsibilities, Requirements (Must Have), Qualifications (Nice to Have), Benefits (Additional Information). No explicit Pay or Schedule info, so we should NOT include those sections.
Skills: Could be extracted from qualifications? But we already have Requirements and Qualifications; maybe we can omit Skills if not distinct. The instruction says use headings e.g. About the role, Responsibilities, Requirements, Qualifications, Skills, Benefits, Pay, Schedule. It's not mandatory to include all; only include if real content.
Thus we will output:
About the role
Mirantis, an IREN company, is the Kubernetes-native AI infrastructure company, enabling organizations to build and operate scalable, secure, and sovereign infrastructure for modern AI, machine learning, and data-intensive applications. By combining open source innovation with deep expertise in Kubernetes orchestration, Mirantis empowers platform engineering teams to deliver composable, production-ready developer platforms across any environment—on-premises, in the cloud, at the edge, or in sovereign data centers. As enterprises navigate the growing complexity of AI-driven workloads, Mirantis delivers the automation, GPU orchestration, and policy-driven control needed to manage infrastructure with confidence and agility. Committed to open standards and freedom from lock-in, Mirantis ensures that customers retain full control of their infrastructure strategy.
We are looking for an experienced Senior Data Platform Engineer to own the event-streaming, transactional database, and custom connectivity backbone driving the k0rdent-ai platform — our multi-tenant control plane for enterprise GPU infrastructure. Every cluster provisioned, every GPU-hour consumed, and every tenant action produces high-cardinality metadata that must be transactionally recorded and securely isolated. You will architect the core pipeline connecting our streaming data plane (Apache Kafka via Strimzi) to our relational data tier (Open Source PostgreSQL via CloudNativePG), which runs across both elastic Kubernetes clusters and high-performance bare metal hardware. Beyond standard administration, a substantial focus of this role is custom ecosystem engineering: deep Go (Golang) systems programming to build custom open-source clients and connectors, design specialized integration libraries, and develop proprietary middleware that bridges our database multi-tenancy with enterprise identity planes like Active Directory (AD)/LDAP. Because this system is the transactional record of every tenant action across the platform, this role also owns its disaster-recovery posture — designing how the platform survives a full region or cluster loss, not just a single-node failure.
Responsibilities
- Hybrid Database Architecture: Design, deploy, and operate high-availability Open Source PostgreSQL topologies (via CloudNativePG on Kubernetes and native Patroni-style topologies on bare metal) and distributed Apache Kafka clusters (via Strimzi), across both containerized Kubernetes environments and high-performance bare metal hardware.
- Custom Client & Connector Development: Write custom, enterprise-grade open-source Kafka clients, standalone connector services, and utility libraries from scratch using preferably Go (Golang) to extend data capabilities where off-the-shelf tooling falls short.
- Enterprise Identity & Data Integration: Architect and build system integrations connecting PostgreSQL authentication and row-level security (RLS) policy evaluation to enterprise Active Directory (AD), LDAP, and OIDC identity providers — via role-mapping and session-context layers (e.g., mapped Postgres roles, JWT claims consumed by RLS policies) rather than a direct connection.
- Transactional Architecture & CDC: Scale high-throughput Change Data Capture (CDC) pipelines via Debezium and Kafka Connect. Implement resilient architectural patterns to maintain absolute data integrity between databases and topics without dual-write risk.
- Cross-Region Resilience & Disaster Recovery: Design and operate cross-region failover and disaster-recovery orchestration for both PostgreSQL and Kafka — including replication topology (sync vs. async trade-offs), split-brain prevention via quorum/witness mechanisms, and explicit RPO/RTO targets for a full region or cluster loss, not just single-node HA.
- PostgreSQL & Infra Tuning: Optimize PostgreSQL instances for heavy ingestion and zero-downtime operations. Tune Write-Ahead Logs (WAL), logical replication streams, connection pooling (PgBouncer), and configure underlying Kubernetes infrastructure primitives (CSI storage volumes and CNI network paths) to eliminate replication lag and unnecessary cross-node latency.
- GitOps & Self-Service Platforming: Maintain a strictly declarative infrastructure-as-code (IaC) culture using Terraform and ArgoCD, creating self-service workflows so internal product teams can securely provision databases, topics, schemas, and ACLs through code.
- Security & Isolation: Implement strict multi-tenant isolation, combining database-level row-level security (RLS) with CNI network policies, mTLS, and Kafka topic-level RBAC.
Requirements
- Platform Seniority: 10+ years in systems, software, data, or platform engineering, with a track record of owning large-scale, business-critical data infrastructure.
- Programming: Go (Golang) or similar engineering skills — comfortable writing clean, concurrent systems code, building custom client libraries, and interacting directly with database/network APIs.
- Open Source PostgreSQL Expertise: 5+ years managing native PostgreSQL. Deep knowledge of database internals (WAL streams, logical replication, publications/subscriptions, and connection architectures) across both Kubernetes stateful environments and bare metal physical nodes.
- Production PostgreSQL-on-Kubernetes: Hands-on experience running PostgreSQL on Kubernetes via a mature Postgres operator — CloudNativePG (CNPG) preferred, but equivalent experience with Zalando's postgres-operator or a comparable operator is acceptable. What matters is depth with declarative cluster lifecycle, failover, and backup/PITR management, not the specific product name.
- Production Kafka: Strong experience managing Apache Kafka at scale on Kubernetes via a declarative operator — Strimzi preferred, but equivalent experience with Confluent for Kubernetes, Koperator, or a comparable operator is acceptable. What matters is depth with operator-managed broker lifecycle and scaling, not the specific product name.
- Custom Data Movement & CDC: Proven experience tuning Kafka Connect and Debezium pipelines. Experience writing or contributing to open-source database connectors, Kafka client libraries, or integration frameworks.
- Enterprise Identity Planes: Hands-on experience building integrations connecting distributed systems to an enterprise IAM/SSO provider — Active Directory, LDAP, Okta, Azure AD, or a generic OIDC provider all count; the specific product matters less than real experience integrating application-level authorization with an enterprise identity system.
- Kubernetes & Cloud Infrastructure: Strong grasp of K8s primitives (Pod lifecycles, Operators, CSI storage layers, and CNI overlay routing), backed by production experience in AWS or GCP.
Qualifications
- Nice to Have: Experience designing and operating cross-region disaster recovery and failover orchestration for stateful systems (Postgres and/or Kafka), including split-brain prevention and RPO/RTO trade-off decisions.
- Nice to Have: Active contributions to open-source Kafka connectors, PostgreSQL operators (e.g., CloudNativePG, Zalando), or Go-based database libraries.
- Nice to Have: Experience with schema governance for event streams (e.g., Karapace or Confluent Schema Registry) and compatibility-mode management for evolving event contracts.
- Nice to Have: Familiarity with high-cardinality telemetry ingestion using Apache Flink or Kafka Streams.
- Nice to Have: Familiarity with US enterprise compliance benchmarks (SOC 2, ISO 27001).
- Nice to Have: Hands-on knowledge of open-source Redis (caching, pub/sub, or as a lightweight data store) — a huge plus.
- Nice to Have: Hands-on knowledge of open-source MongoDB (document modeling, replication, sharding) — a huge plus.
Benefits
- Work with an established Silicon Valley leader in the cloud infrastructure industry;
- Work with exceptionally passionate, talented and engaging colleagues, helping Fortune 500 and Global 2000 customers implement next-generation cloud technologies;
- Be a part of cutting-edge, open-source innovation;
- Thrive in the high-energy environment of a young company where openness, collaboration, risk-taking, and continuous growth are valued;
- Professional development and training;
- Attend conferences and working groups;
- Company outings, happy hours, hackathons, and tech talks;
- Receive a competitive compensation package with a strong benefits plan.
- We are a Leader for Container Management in G2 (#2 after AWS)!
We should not include Pay or Schedule sections as no info.
Check for any boilerplate removed: The initial "https://www.mirantis.com/" maybe considered navigation? It's a URL; we can keep as part of description? It's not substantive; maybe drop. The instruction: remove cookie/navigation/legal/EEO/equal-opportunity/scam-warning boilerplate, duplicated lines, and application instructions. The URL is not boilerplate but could be considered navigation; but it's just a link. Might be okay to keep? Probably safe to drop as it's not substantive. We'll omit.
Also we included "Education And Experience" section? Not included as separate heading; we could incorporate into Requirements or Qualifications. The posting had "Education And Experience Bachelor's degree in Computer Science & Engineering (or a related field), or 10+ years of equivalent related experience." This is a requirement; we could add to Requirements list. Let's add a bullet under Requirements.
Add: "Education And Experience: Bachelor's degree in Computer Science & Engineering (or a related field), or 10+ years of equivalent related experience."
Now ensure we didn't repeat job title as heading. Good.
Check for any duplicated lines
for prose. Do NOT repeat job title as heading. Do NOT put Job ID/requisition numbers as headings. Preserve all substantive info. Remove boilerplate, navigation, legal, EEO, etc. Also remove application instructions. The posting includes: company description paragraph at top (Mirantis...). Then "Job Description" heading then description. Then "Main Responsibilities" bullet list. Then "Qualifications" with Must Have and Nice to Have. Then "Education And Experience". Then "Additional Information" about benefits. We need to decide which sections to include based on content. We have About the role (maybe the company description and job description summary). Responsibilities (Main Responsibilities). Requirements (maybe Must Have). Qualifications (maybe Nice to Have? Actually Qualifications section could contain both Must Have and Nice to Have? The instruction: Use headings for section headings (e.g. About the role, Responsibilities, Requirements, Qualifications, Skills, Benefits, Pay, Schedule). So we need to map content accordingly. We have: About the role: could be the introductory paragraph describing Mirantis and the role purpose. Could combine the first paragraph and the "Job Description" paragraph. Responsibilities: the Main Responsibilities bullet list. Requirements: maybe the "Must Have" bullet list (the required qualifications). Qualifications: maybe the "Nice to Have" bullet list? Or we could treat Qualifications as the "Education And Experience" plus maybe other qualifications. But we have both Requirements and Qualifications headings; we need to decide. The instruction: "Prefer a few substantive sections over many thin ones." So we might combine similar content. But we need to include sections only when posting has real, specific content for it. We have content for About the role, Responsibilities, Requirements (Must Have), Qualifications (Nice to Have), Benefits (Additional Information). No explicit Pay or Schedule info, so we should NOT include those sections. Skills: Could be extracted from qualifications? But we already have Requirements and Qualifications; maybe we can omit Skills if not distinct. The instruction says use headings e.g. About the role, Responsibilities, Requirements, Qualifications, Skills, Benefits, Pay, Schedule. It's not mandatory to include all; only include if real content. Thus we will output:
About the role
Mirantis, an IREN company, is the Kubernetes-native AI infrastructure company, enabling organizations to build and operate scalable, secure, and sovereign infrastructure for modern AI, machine learning, and data-intensive applications. By combining open source innovation with deep expertise in Kubernetes orchestration, Mirantis empowers platform engineering teams to deliver composable, production-ready developer platforms across any environment—on-premises, in the cloud, at the edge, or in sovereign data centers. As enterprises navigate the growing complexity of AI-driven workloads, Mirantis delivers the automation, GPU orchestration, and policy-driven control needed to manage infrastructure with confidence and agility. Committed to open standards and freedom from lock-in, Mirantis ensures that customers retain full control of their infrastructure strategy.
We are looking for an experienced Senior Data Platform Engineer to own the event-streaming, transactional database, and custom connectivity backbone driving the k0rdent-ai platform — our multi-tenant control plane for enterprise GPU infrastructure. Every cluster provisioned, every GPU-hour consumed, and every tenant action produces high-cardinality metadata that must be transactionally recorded and securely isolated. You will architect the core pipeline connecting our streaming data plane (Apache Kafka via Strimzi) to our relational data tier (Open Source PostgreSQL via CloudNativePG), which runs across both elastic Kubernetes clusters and high-performance bare metal hardware. Beyond standard administration, a substantial focus of this role is custom ecosystem engineering: deep Go (Golang) systems programming to build custom open-source clients and connectors, design specialized integration libraries, and develop proprietary middleware that bridges our database multi-tenancy with enterprise identity planes like Active Directory (AD)/LDAP. Because this system is the transactional record of every tenant action across the platform, this role also owns its disaster-recovery posture — designing how the platform survives a full region or cluster loss, not just a single-node failure.
Responsibilities
- Hybrid Database Architecture: Design, deploy, and operate high-availability Open Source PostgreSQL topologies (via CloudNativePG on Kubernetes and native Patroni-style topologies on bare metal) and distributed Apache Kafka clusters (via Strimzi), across both containerized Kubernetes environments and high-performance bare metal hardware.
- Custom Client & Connector Development: Write custom, enterprise-grade open-source Kafka clients, standalone connector services, and utility libraries from scratch using preferably Go (Golang) to extend data capabilities where off-the-shelf tooling falls short.
- Enterprise Identity & Data Integration: Architect and build system integrations connecting PostgreSQL authentication and row-level security (RLS) policy evaluation to enterprise Active Directory (AD), LDAP, and OIDC identity providers — via role-mapping and session-context layers (e.g., mapped Postgres roles, JWT claims consumed by RLS policies) rather than a direct connection.
- Transactional Architecture & CDC: Scale high-throughput Change Data Capture (CDC) pipelines via Debezium and Kafka Connect. Implement resilient architectural patterns to maintain absolute data integrity between databases and topics without dual-write risk.
- Cross-Region Resilience & Disaster Recovery: Design and operate cross-region failover and disaster-recovery orchestration for both PostgreSQL and Kafka — including replication topology (sync vs. async trade-offs), split-brain prevention via quorum/witness mechanisms, and explicit RPO/RTO targets for a full region or cluster loss, not just single-node HA.
- PostgreSQL & Infra Tuning: Optimize PostgreSQL instances for heavy ingestion and zero-downtime operations. Tune Write-Ahead Logs (WAL), logical replication streams, connection pooling (PgBouncer), and configure underlying Kubernetes infrastructure primitives (CSI storage volumes and CNI network paths) to eliminate replication lag and unnecessary cross-node latency.
- GitOps & Self-Service Platforming: Maintain a strictly declarative infrastructure-as-code (IaC) culture using Terraform and ArgoCD, creating self-service workflows so internal product teams can securely provision databases, topics, schemas, and ACLs through code.
- Security & Isolation: Implement strict multi-tenant isolation, combining database-level row-level security (RLS) with CNI network policies, mTLS, and Kafka topic-level RBAC.
Requirements
- Platform Seniority: 10+ years in systems, software, data, or platform engineering, with a track record of owning large-scale, business-critical data infrastructure.
- Programming: Go (Golang) or similar engineering skills — comfortable writing clean, concurrent systems code, building custom client libraries, and interacting directly with database/network APIs.
- Open Source PostgreSQL Expertise: 5+ years managing native PostgreSQL. Deep knowledge of database internals (WAL streams, logical replication, publications/subscriptions, and connection architectures) across both Kubernetes stateful environments and bare metal physical nodes.
- Production PostgreSQL-on-Kubernetes: Hands-on experience running PostgreSQL on Kubernetes via a mature Postgres operator — CloudNativePG (CNPG) preferred, but equivalent experience with Zalando's postgres-operator or a comparable operator is acceptable. What matters is depth with declarative cluster lifecycle, failover, and backup/PITR management, not the specific product name.
- Production Kafka: Strong experience managing Apache Kafka at scale on Kubernetes via a declarative operator — Strimzi preferred, but equivalent experience with Confluent for Kubernetes, Koperator, or a comparable operator is acceptable. What matters is depth with operator-managed broker lifecycle and scaling, not the specific product name.
- Custom Data Movement & CDC: Proven experience tuning Kafka Connect and Debezium pipelines. Experience writing or contributing to open-source database connectors, Kafka client libraries, or integration frameworks.
- Enterprise Identity Planes: Hands-on experience building integrations connecting distributed systems to an enterprise IAM/SSO provider — Active Directory, LDAP, Okta, Azure AD, or a generic OIDC provider all count; the specific product matters less than real experience integrating application-level authorization with an enterprise identity system.
- Kubernetes & Cloud Infrastructure: Strong grasp of K8s primitives (Pod lifecycles, Operators, CSI storage layers, and CNI overlay routing), backed by production experience in AWS or GCP.
Qualifications
- Nice to Have: Experience designing and operating cross-region disaster recovery and failover orchestration for stateful systems (Postgres and/or Kafka), including split-brain prevention and RPO/RTO trade-off decisions.
- Nice to Have: Active contributions to open-source Kafka connectors, PostgreSQL operators (e.g., CloudNativePG, Zalando), or Go-based database libraries.
- Nice to Have: Experience with schema governance for event streams (e.g., Karapace or Confluent Schema Registry) and compatibility-mode management for evolving event contracts.
- Nice to Have: Familiarity with high-cardinality telemetry ingestion using Apache Flink or Kafka Streams.
- Nice to Have: Familiarity with US enterprise compliance benchmarks (SOC 2, ISO 27001).
- Nice to Have: Hands-on knowledge of open-source Redis (caching, pub/sub, or as a lightweight data store) — a huge plus.
- Nice to Have: Hands-on knowledge of open-source MongoDB (document modeling, replication, sharding) — a huge plus.
Benefits
- Work with an established Silicon Valley leader in the cloud infrastructure industry;
- Work with exceptionally passionate, talented and engaging colleagues, helping Fortune 500 and Global 2000 customers implement next-generation cloud technologies;
- Be a part of cutting-edge, open-source innovation;
- Thrive in the high-energy environment of a young company where openness, collaboration, risk-taking, and continuous growth are valued;
- Professional development and training;
- Attend conferences and working groups;
- Company outings, happy hours, hackathons, and tech talks;
- Receive a competitive compensation package with a strong benefits plan.
- We are a Leader for Container Management in G2 (#2 after AWS)!