Hadoop/Bigdata Developer

Job Duties

• Update high-level Infrastructure design documents.
• Install, upgrade, and maintain BDA/Hadoop (CDH), Kafka, Nifi, ODI clusters.
• Set up and troubleshoot user access in Hadoop clusters.
• Create ACL roles for Hive databases for different access groups.
• Support data ingestion using Oracle Golden Gate/Confluent Kafka and Hadoop (Big data).
• Set up and manage data replication between multiple cluster environments (Prod – DR, DR-UAT).
• Support migration between multiple environments.
• Code tune long-running jobs in MapReduce/Spark using Hive/Spark/Anaconda.
• Add nodes and respective services to Hadoop and Kafka clusters.
• Coordinate with multiple teams and vendors to stabilize big data environments.
• Integrate ETL (ODI/NIFI/UC4) applications with big data technologies for data processing and scheduling.
• Renew server certificates for Hadoop and Kafka clusters.
• Generate and manage encrypt/decrypt keys for internal and external communication.
• Create universal scripts to monitor server health for Hadoop, Kafka, Nifi, ODI, and other big data applications.
• Fix issues related to network connectivity between clusters.
• Participate in daily sync-up meetings with the project team.
• Enhance existing frameworks for deployment into production and maintenance.

Requirements

• Bachelor’s Degree in computer science, Computer Information Systems, Information Technology, or a closely related technical field, or a combination of education and experience equating to the U.S. equivalent of a bachelor's degree in one of the aforementioned subjects.