Kafka Engineer

About the role

Kafka Engineer - Remote position at Bright Vision Technologies, a technology consulting and software development company delivering cloud, AI, data, and enterprise solutions across the United States. This is a 100% remote (U.S.) full-time, direct W2 role. The company welcomes U.S. Citizens, Green Card Holders, EAD Holders, and H-1B transfer candidates; note that new H-1B sponsorship is not available for this position.

Responsibilities

• Architect, deploy, and operate large-scale Apache Kafka or Confluent Platform clusters across on-prem and cloud environments
• Design partitioning, replication, and topic strategies that balance throughput, durability, and operational simplicity
• Implement strong security on Kafka clusters using SASL, mTLS, ACLs, RBAC, and integration with corporate IdPs
• Operate Schema Registry, Kafka Connect, KSQL/ksqlDB, and Kafka Streams in production
• Build and operate Kafka Connect pipelines integrating sources and sinks across enterprise systems
• Design HA/DR strategies for Kafka, including MirrorMaker 2, Cluster Linking, and multi-region active-active patterns
• Build CI/CD pipelines for Kafka topic, ACL, and connector configurations using GitOps patterns
• Implement comprehensive observability using Prometheus, Grafana, Datadog, or Confluent Control Center
• Drive Kafka cost and capacity optimization through right-sizing and storage tiering
• Onboard application teams to Kafka with clear patterns, templates, and best practices
• Lead incident response and post-incident reviews for streaming workloads, applying disciplined engineering practices and partnering closely with stakeholders
• Mentor and coach junior and mid-level engineers through code review, design review, pair programming, and structured knowledge sharing
• Maintain comprehensive, current technical documentation — including architecture diagrams, design decisions, configuration references, runbooks, and operational procedures
• Continuously evaluate emerging streaming technologies (Pulsar, Redpanda, AWS MSK, Azure Event Hubs)

Requirements

• Bachelor’s degree in Computer Science, Engineering, or a related technical discipline
• Five or more years of experience operating Apache Kafka or Confluent Platform in production
• Deep, hands-on knowledge of Kafka internals (partitions, replication, ISRs, consumer groups)
• Strong experience with Kafka security (SASL, mTLS, ACLs, RBAC)
• Hands-on experience with Kafka Connect, Schema Registry, and either Kafka Streams or ksqlDB
• Experience with HA/DR strategies for Kafka
• Strong scripting skills in Python, Bash, or Go
• Hands-on experience with infrastructure-as-code (Terraform, Ansible)
• Working knowledge of observability tooling for Kafka
• Excellent troubleshooting, communication, and documentation skills

Qualifications

• Confluent Certified Administrator or Developer credentials
• Experience operating Kafka on Kubernetes (Strimzi, Confluent Operator)
• Exposure to managed Kafka services (AWS MSK, Azure Event Hubs Kafka API)
• Familiarity with stream processing frameworks (Flink, Spark Streaming)
• Experience with data governance and lineage for streaming data

Pay

$100,000–$150,000 annually

Schedule

Full-time