Kafka Engineer

About the role

The Kafka Engineer / Administrator / Developer is a key member of the program technical team, supporting large-scale data streaming, system integration, and platform modernization initiatives. This role is responsible for designing, developing, administering, and optimizing Apache Kafka clusters and event-driven architectures that support high-volume, mission-critical data flows.

Responsibilities

• Design, build, administer, and maintain Kafka clusters across development, test, and production environments.

• Manage Kafka topics, partitions, brokers, replication, retention policies, and access controls.

• Monitor Kafka performance, availability, throughput, and latency; proactively identify and resolve issues.

• Implement and maintain high availability and fault-tolerant Kafka configurations.

• Develop and support event streaming pipelines using Kafka for real-time and near-real-time data processing.

• Integrate Kafka with API Gateway (APIGW)–based microservices and downstream backend systems.

• Design and implement Kafka producers, consumers, and connectors (e.g., Kafka Connect) to support system integrations and ETL/data movement needs.

• Collaborate with application teams to define event schemas, topics, and data contracts.

• Ensure reliable message delivery, data integrity, and error handling across streaming workflows.

• Implement Kafka security best practices, including authentication, authorization, encryption in transit, and auditing.

• Support DevSecOps practices, CI/CD pipelines, and infrastructure-as-code approaches where applicable.

• Participate in incident response, root cause analysis, and operational readiness activities.

Requirements

• Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related field.

• 3+ years of experience developing, administering, and supporting Apache Kafka in enterprise environments.

• Hands-on experience managing Kafka clusters, topics, partitions, and event streaming pipelines.

• Experience integrating Kafka with microservices, API Gateways (APIGW), and backend systems.

• Strong understanding of event-driven architectures, messaging patterns, and data streaming concepts.

• Experience with Linux-based environments and command-line administration.

• Strong troubleshooting and performance tuning skills.

• Ability to clearly communicate technical concepts to both technical and non-technical stakeholders.

Qualifications

• Experience supporting federal healthcare programs.

• Experience working in Agile, Scrum, and/or DevSecOps environments.

• Familiarity with cloud-based Kafka deployments (AWS MSK or similar managed Kafka services).

• Experience with CI/CD pipelines and automation tools.

• Knowledge of cloud security concepts and secure data transmission.

• Experience with monitoring tools and observability platforms for Kafka (e.g., Prometheus, Grafana, CloudWatch).

• Knowledge of schema management tools (e.g., Schema Registry).

• Familiarity with containerized environments and orchestration tools (Docker, Kubernetes) is a plus.

Skills

• Linux-based environments and command-line administration.

• Event-driven architectures, messaging patterns, and data streaming concepts.

• CI/CD pipelines and automation tools.

• Cloud security concepts and secure data transmission.

• Monitoring tools and observability platforms for Kafka (e.g., Prometheus, Grafana, CloudWatch).

• Schema management tools (e.g., Schema Registry).

• Containerized environments and orchestration tools (Docker, Kubernetes).

Benefits

N/A

Pay

$100,000 – $130,000 (commensurate with experience)

Schedule

40 hours/week, availability during core business hours