Data Engineer - Python, dbt & Oracle

Position Overview

Our Client is seeking an experienced Data Engineer to support a major data infrastructure modernization initiative. This project involves assessing and modernizing approximately 700–800 legacy Perl scripts and RunMyJobs/PL/SQL scheduling workflows. The selected candidate will help migrate these legacy processes to a modular, observable, and testable data platform using Python, dbt, Apache Airflow, and Oracle 19c.

Responsibilities

• Assess, redesign, and migrate legacy Perl scripts, RunMyJobs workflows, and PL/SQL scheduling logic to Python scripts and Apache Airflow DAGs
• Design, build, and deploy production-grade Airflow DAGs for complex, interdependent enterprise batch workflows
• Develop custom Airflow operators and sensors for Oracle 19c, REST APIs, SFTP/FTP transfers, and external triggers
• Build and maintain data transformation workflows using dbt
• Write and optimize complex Oracle SQL and PL/SQL queries
• Improve the performance and reliability of data pipelines operating across on-premises systems
• Implement error handling, retry processes, SLA alerts, execution logs, audit trails, and data-lineage monitoring
• Troubleshoot failed or partially completed jobs across application logs, databases, and Linux/Unix filesystems
• Work closely with DBAs and application teams to document job dependencies, identify critical processing paths, and improve data quality and recoverability
• Support parameter-driven automation and reusable pipeline-development standards

Required Qualifications

• Strong hands-on experience with dbt and Oracle 19c. Both are mandatory
• At least five years of Python development experience, particularly in automation, scripting, and enterprise ETL pipelines
• At least two years of experience developing and deploying production-grade Apache Airflow DAGs, custom operators, and sensors
• Strong experience with Oracle SQL, PL/SQL development, complex queries, and performance tuning
• Advanced knowledge of Linux/Unix environments and shell-based processing
• Experience modernizing or replacing enterprise job schedulers such as RunMyJobs or AutoSys
• Experience developing REST API integrations and SFTP/FTP file-transfer processes
• Strong understanding of batch processing, job dependencies, failure recovery, and operational monitoring
• Proven ability to troubleshoot issues across application logs, databases, filesystems, and data pipelines

Preferred Qualifications

• Experience modernizing or replacing legacy Perl applications or scripts
• Previous experience in banking, investment management, or financial services
• Familiarity with financial data such as holdings, transactions, net asset values, and tax lots
• Experience with Git or Bitbucket, Jenkins, and CI/CD-based deployment processes
• Knowledge of operational risk, data governance, audit controls, and data lineage within financial systems

Location: Jersey City, NJ — Hybrid, 3 days onsite at 185 Hudson Street, Suite 1150, Jersey City, NJ 07311

Work Authorization: U.S. Citizens and Green Card holders only

Candidate Requirement: Local candidates only

Interview Process: First-round video interview followed by a two-hour onsite final interview