Hadoop and PySpark Data Engineer

About the role

Join the Infosys Financial Services unit, a global leader in driving digital transformation for financial institutions. We specialize in leveraging advanced technologies such as AI, cloud, and data-led innovation to help clients accelerate growth and unlock business value. As a Technology Consultant 2, you'll work with cutting-edge AI technologies, collaborate with industry experts, and contribute to transformative projects across banking, wealth management, and insurance sectors.

Responsibilities

• Contribute to the requirements elicitation process by documenting assigned parts of business requirements, in line with guidance provided.
• Facilitate software application design discussions and document design decisions to guide the technical team toward building software solutions.
• Participate in coding and integrate new features or updates into existing applications, focusing on maintaining system stability.
• Conduct code reviews, make changes to the codebase, and maintain code repositories.
• Implement test strategies, analyze results, and coordinate bug fixes to uphold software quality standards.
• Develop user training programs, documentation, and support frameworks to ensure a smooth transition to new software applications.
• Actively participate in resolving production issues and recommend preventive strategies to enhance system reliability.
• Maintain detailed records of code, testing techniques, and support activities to enrich the knowledge base and assist similar projects.

Requirements

• Experience in data warehousing architectural approaches.
• Sound understanding and experience with the Hadoop ecosystem (Cloudera).
• Experience working with a Big Data implementation in a production environment.
• Experience with Big Data technologies like Hadoop, Hive, Spark, Python, etc.
• Experience in Python and Unix shell scripting.
• Experience with orchestration tools like Autosys or Airflow.
• Sound knowledge of relational databases (SQL) and experience with large SQL-based systems.
• Understanding of Agile methodologies and technologies.

Qualifications

• Bachelor’s degree or foreign equivalent from an accredited institution. Three years of progressive experience in the specialty may substitute for each year of education.
• This position may require relocation and/or travel to work/project location.
• Candidates authorized to work for any employer in the United States without employer-based visa sponsorship are welcome to apply. Infosys is unable to provide immigration sponsorship for this role now or in the future.

Preferred Skills

• Ability to understand and explore constantly evolving tools within the Hadoop ecosystem and apply them appropriately to relevant problems.
• Knowledge and experience with Cloud and containerization technologies.
• Experience with data visualization tools like Tableau.

Contribution to the Team

• A collaborative spirit and excellent communication skills.
• Ability to handle end-to-end SDLC phases from requirement gathering to implementation.
• A knack for translating complex requirements into actionable development tasks.
• A passion for design and hands-on coding experience.
• A proactive approach to testing, troubleshooting, and refining applications.
• Ability to work with cross-functional teams and perform software integration.

Pay

Estimated annual compensation range for a candidate based in Jersey City, NJ: $82,419 to $126,600.

Benefits

• Medical, Dental, Vision, and Life Insurance.
• Long-term and Short-term Disability.
• Health and Dependent Care Reimbursement Accounts.
• Accident, Critical Illness, Hospital Indemnity, and Legal Insurance.
• 401(k) plan with contributions dependent on salary level.
• Paid holidays plus Paid Time Off.