Kafka Administrator
Smart Folks Inc · Dallas, TX · 3 wk ago
OTHRFull-time
Role Descriptions
A Kafka Administrator is responsible for installing| configuring| maintaining| and monitoring Apache Kafka clusters to ensure high availability| security| and optimal performance. They manage topics| partitions| and replication| troubleshoot issues| implement security protocols (SSLTLS| SASL| ACLs)| and automate tasks using tools like Ansible or Terraform.Roles and Responsibilities
- Cluster Management
- Installing| configuring| and upgrading Kafka brokers| ZookeeperKRaft| Connect| and Schema Registry.
- Maintaining topics| partitions| and replication strategies.
- Monitoring cluster health| broker performance| consumer lag| and producer latency using tools like Prometheus and Grafana to ensure high throughput.
- Performance Tuning
- Implementing high availability designs for disaster recovery.
- Ensuring broker redundancy.
- Topic Partition Management
- Managing topic configuration| partitioning strategies| replication factors| and data retention policies.
- Security Access Control
- Implementing security measures such as SSLTLS encryption| Kerberos authentication| and ACLs (Access Control Lists) to secure data.
- Troubleshooting Support
- Providing 3rd-level support to resolve broker| producer| or consumer issues and performing root cause analysis (RCA).
- Automation
- Developing scripts to automate routine tasks| such as provisioning and monitoring| using Ansible| Puppet| or Kubernetes.
- Collaboration
- Working with developers to design data pipelines| optimize producer consumer configurations| and onboard new applications.
Required Skills
- In-depth knowledge of Apache Kafka, Confluent Kafka.
- Strong Linux system administration skills.
- Experience with monitoring tools (Prometheus| Grafana| JMX).
- Scripting proficiency in Python| Bash| or Java.
- Experience with containerization tools (Docker| Kubernetes OpenShift).