Big Data Developer

Job Duties And Responsibilities

• Build processes that support data transformation, data structures, metadata, dependency and workload management
• Build and Implement data ingestion and curation process developed using Big data tools such as Spark (Scala/python), Hive, Spark, HDFS, Sqoop, Hbase, Kerberos, Sentry and Impala etc.
• Handle ingestion of huge volumes data from various platforms for Analytics needs and writing high-performance, reliable and maintainable ETL code
• Show strong SQL knowledge and data analysis skills for data anomaly detection and data quality assurance
• Write shell scripts
• Create complex SQL queries, Hadoop commands and Git
• Create Database, Schemas, Hive tables (External and Managed) with various file formats (Orc, Parquet, Avro and Text etc.)
• Maintain performance of production jobs and advise any necessary infrastructure changes
• Work on Code version using Bitbucket and CI/CD pipeline

Requirements

• Bachelor’s degree in Computer Science, IT, Engg or related with at least 60 months of experience

Equal Opportunity Employer.