Python Developer
Job Summary
Design, build, and maintain high-performance ETL/ELT data pipelines using Python and PySpark. Apply expert-level coding skills to develop and manage data processing jobs leveraging PySpark for distributed computing across large-scale datasets. Take full ownership of the data workflow, including getting data from multiple sources, scrubbing, and validating data to ensure the highest quality. Write and optimize complex, performant SQL queries for data extraction, integrity checks, and performance tuning. Contribute to platform modernization by exploring and increasing the adoption of AI/ML, including using tools like Copilot and Claude for acceleration, and building models to fill data gaps or improve systems. Collaborate with data architects by proposing ideas and great questions, taking ownership as the expert on data, pipelines, and systems. Implement DevOps practices for the automated deployment and orchestration of Python applications and data pipelines (e.g., using Docker, Jenkins, Terraform).
Key Responsibilities
- Design, build, and maintain high-performance ETL/ELT data pipelines using Python and PySpark.
- Apply expert-level coding skills to develop and manage data processing jobs leveraging PySpark for distributed computing across large-scale datasets.
- Take full ownership of the data workflow, including getting data from multiple sources, scrubbing, and validating data to ensure the highest quality.
- Write and optimize complex, performant SQL queries for data extraction, integrity checks, and performance tuning.
- Contribute to platform modernization by exploring and increasing the adoption of AI/ML, including using tools like Copilot and Claude for acceleration, and building models to fill data gaps or improve systems.
- Collaborate with data architects by proposing ideas and great questions, taking ownership as the expert on data, pipelines, and systems.
- Implement DevOps practices for the automated deployment and orchestration of Python applications and data pipelines (e.g., using Docker, Jenkins, Terraform).
Qualifications
- Technical Skills:
- Programming: Expert-level proficiency in Python, including libraries like Pandas and NumPy.
- Designing: Designing data pipelines for the data coming from multiple sources.
- Data Processing: Solid hands-on experience with PySpark for building scalable data workflows.
- Data Querying: Expert-level knowledge of writing complex SQL queries (Oracle or Snowflake), with proven ability to perform performance tuning on large datasets and complex database code.
- Cloud Platform: Robust experience with AWS cloud services and associated data services, specifically: AWS Glue (ETL), S3, Lambda, Redshift, DynamoDB, Athena, ECS, EventBridge, OpenSearch, RDS.
- ETL & Data Management: Robust proficiency in ETL/ELT methodologies and tools, as well as Data Quality, Data Validation, and Anomaly Detection techniques.
- Scripting: Working experience with scripting and automation using Unix and Python.
- Desired Skills & Professional Attributes:
- Familiarity with AI/ML and Large Language Model (LLM) approaches to data analysis and validation.
- Knowledge of data warehousing concepts and data modeling techniques.
- Experience with DevOps, Continuous Integration, and Continuous Delivery (e.g., Jenkins, GitHub).
- Experience with BI Reporting tools such as Power BI or Tableau.
- Robust preference for candidates with prior experience in the investment data domain.
- Ability to work independently through complex data challenges and robust analytical and problem-solving skills.