Hadoop and PySpark Developer

About the role

You will contribute to the full software development lifecycle, from requirements elicitation through implementation and support. The role involves designing, coding, testing, and maintaining data solutions with a focus on Hadoop and PySpark.

Responsibilities

• Document assigned business requirements and facilitate design discussions to guide the technical team.
• Develop, integrate, and maintain new features or updates in existing applications while ensuring system stability.
• Conduct code reviews, implement changes, and maintain code repositories.
• Implement test strategies, analyze results, and coordinate bug fixes to uphold software quality standards.
• Develop user training programs, documentation, and support frameworks for smooth software adoption.
• Actively resolve production issues and recommend preventive strategies to enhance system reliability.
• Maintain detailed records of code, testing techniques, and support activities to enrich the knowledge base.

Requirements

• Experience in Hadoop, Python, and Spark.
• Good experience in end-to-end implementation of data warehouses and data marts, including PySpark development.
• Knowledge and hands-on experience in SQL and Unix shell scripting.
• Experience working with Big Data implementations in a production environment.
• Familiarity with Big Data technologies such as Hadoop, Hive, Spark, and Python.
• Collaborative spirit and excellent communication skills.
• Ability to handle end-to-end SDLC phases from requirement gathering to implementation.
• Passion for design and hands-on coding experience.
• Proactive approach to testing, troubleshooting, and refining applications.
• Ability to work with cross-functional teams and perform software integration.

Preferred Qualifications

• Understanding of Agile methodologies and technologies.
• Able to explore and apply evolving tools within the Hadoop ecosystem to solve relevant problems.