Hadoop Engineer

Location: Chicago, IL / Charlotte, NC (Hybrid)
Duration: 3 months

About the role

We are seeking a Hadoop Engineer to support NextGen Platforms built around Big Data technologies. The ideal candidate will have experience in the cluster management of vendor-based Hadoop and Data Science products. This role involves participation in the full life cycle of an application within an agile development process and requires the ability to develop, engineer, and communicate at a high technical level with clients, development teams, and partners.

Responsibilities

• Provide subject matter expertise in managing Hadoop and Data Science Platform operations, with a focus on Cloudera Hadoop, Jupyter Notebook, OpenShift, and Docker-Container Cluster Management.
• Manage platform operations across all environments, including upgrades, bug fixes, deployments, and disaster recovery.
• Handle incident, problem, and capacity management, including metrics and monitoring for resolution and forecasting.
• Integrate solutions with other applications and platforms.
• Serve as a technical liaison between client partners and vendors.
• Perform L1, L2, and L3 level support for the Hadoop Platform.
• Conduct Yarn, Spark, and Impala job debugging and troubleshooting.
• Address issues related to Name node, HDFS space, and file/folder permission backups.
• Perform query performance and resource utilization tuning.

Requirements

• Experience with Hadoop, Kafka, Spark, Impala, Hive, HBase, and Ozone.
• Strong knowledge of Hadoop Architecture, HDFS, and cluster administration.
• Knowledge of fully integrated AD/Kerberos authentication.
• Experience setting up optimum cluster configurations and debugging YARN.
• Hands-on experience analyzing Hadoop log files, compression, encoding, and file formats.
• Expert-level knowledge of Cloudera Hadoop components such as HDFS, Sentry, HBase, Kafka, Impala, SOLR, Hue, Spark, Hive, YARN, Zookeeper, and Postgres.
• Experience with monitoring, alerting, and job scheduling systems like Autosys.
• Proficiency with automation and DevOps tools such as Ansible, Jenkins, SVN, and Bitbucket.

Skills

• Strong technical knowledge of Unix/Linux.
• Proficiency with databases (Sybase/SQL/Oracle), data warehousing, and SQL.
• Skilled in Java, Python, Perl, and Shell scripting.

Preferred Qualifications

• Experience working on Big Data Technologies.
• Cloudera Admin / Dev Certification.
• Certification in Cloud, Docker-Container, or OpenShift Technologies.
• Experience with Tableau, including building dashboards connected to Hadoop-Hive Tables.

Benefits

• Supplemental medical, dental, vision, life, and disability insurance plans.
• Employee Stock Purchase Program (ESPP).
• 401K program with company match after 12 months of tenure.
• Health Savings Account (HSA) on the HDHP plan.
• SupportLinc Employee Assistance Program (EAP) with up to 8 free counseling sessions.
• Corporate discount savings program and other discounts.
• On-demand training program and access to certification prep, technical and leadership courses/books/seminars after 6+ months of tenure.
• Certification discounts and perks for associations including CompTIA and IIBA.
• Dedicated customer service team and certified Career Coach for consultants.