Hadoop Administrator

Responsibilities

• Deploying, maintaining the Hadoop cluster, adding and removing nodes using cluster monitoring tools like Ganglia, Nagios, or Cloudera Manager, configuring the Name Node high availability
• Keeping a track of all the running Hadoop jobs
• Implementing, managing, and administering the overall Hadoop infrastructure
• Takes care of the day-to-day running of Hadoop clusters
• Works closely with the database team, network team, BI team, and application teams to ensure all big data applications are highly available and performing as expected
• Knowledge on Apache Distribution, manually setting up all configurations such as CoreSite, HDFS Site, YARN Site, and Map Red Site
• Aware of configuration setup using Hadoop distributions like Hortonworks, Cloudera, or MapR
• Capacity planning and estimating requirements for lowering or increasing the capacity of the Hadoop cluster, deciding the size of the Hadoop cluster based on the data to be stored in HDFS
• Ensures the Hadoop cluster is up and running at all times, monitoring the cluster connectivity and performance
• Manages and reviews Hadoop log files, performs backup and recovery tasks, manages resources and security

Qualifications

• Bachelor's degree in Computer Science, Information Technology, or a related field
• Minimum 5 years of experience in Hadoop administration
• Experience with cluster monitoring tools like Ganglia, Nagios, or Cloudera Manager
• Hands-on experience with Apache Distribution and manual setup of configurations
• Strong understanding of Hadoop architecture, including HDFS, YARN, and MapReduce
• Experience with capacity planning and resource management
• Excellent troubleshooting skills and ability to resolve complex technical issues