Senior Kafka Platform Engineer

Key Responsibilities

• Build and maintain internal applications, APIs, and automation to streamline Kafka cluster provisioning, access controls, topic management, and operational processes.
• Design and manage production Kafka environments (self-hosted or managed services such as Confluent or cloud-native equivalents), including upgrades, scaling strategies, disaster recovery planning, and performance optimization.
• Deploy and operate Kafka workloads in Kubernetes environments using tools like Helm, operators, and GitOps workflows to ensure consistent, repeatable infrastructure delivery.
• Implement and support components such as Kafka Connect, schema management services, and cross-cluster replication technologies; standardize connector usage and promote reusable patterns.
• Strengthen platform reliability through service level objectives (SLOs), incident management practices, automated recovery mechanisms, and clear operational documentation.
• Develop comprehensive observability solutions, including metrics, logging, tracing, lag monitoring, and capacity reporting dashboards.
• Ensure platform security through encryption, authentication mechanisms, access controls, and compliance-focused automation practices.
• Promote best practices for event streaming, including topic structure, partitioning strategies, schema evolution, message ordering, and fault-tolerant processing approaches.
• Collaborate closely with application engineers, data teams, and infrastructure groups to improve adoption and usability of the platform.
• Contribute to technical leadership by mentoring team members and shaping platform direction, standards, and roadmap.

Required Qualifications

• Strong collaboration and communication skills when working with engineering and platform teams.
• Significant hands-on experience operating Kafka at scale in production (including cluster internals, replication, partitioning, and recovery processes).
• Solid software engineering background, with a track record of delivering reliable production systems.
• Experience running stateful workloads in Kubernetes environments.
• Proficiency in infrastructure automation using Terraform, Helm, GitOps tools (e.g., Argo CD or Flux), and CI/CD pipelines.
• Programming experience in at least one of the following: Python, Go, or Java, along with command-line scripting and Linux fundamentals.
• Experience implementing monitoring, alerting, and performance tuning solutions for distributed systems.
• Familiarity with security practices including encryption, identity/authentication protocols, and secrets management solutions.
• Knowledge of Kafka ecosystem tools such as connectors, schema services, and replication frameworks.
• Experience working in public cloud environments and understanding networking and access control concepts.
• Proven experience in incident response, operational readiness, and driving improvements through post-incident reviews.

Preferred Qualifications

• Background in building developer platforms or self-service infrastructure solutions.
• Experience with stream-processing technologies such as Kafka Streams, Flink, or Spark Streaming.
• Familiarity with Kubernetes-native Kafka solutions (e.g., operator-based deployments).
• Knowledge of change data capture (CDC) tools and scalable database integration patterns.
• Experience designing multi-region architectures and implementing disaster recovery strategies.