Ab Initio Developer

About This Role

Join the Home Lending Data and Insights team as a Software Engineer 3 - Contingent, where you'll build, run, and modernize enterprise data pipelines in a hybrid landscape—spanning on-prem platforms (Python, Oracle, Teradata and Ab Initio) and a modern Google Cloud stack (BigQuery, Dataplex). You'll focus on dependable batch and near real-time processing, operational excellence, and the transition from legacy scheduling and ETL patterns to cloud-native orchestration with Google Cloud Composer, backed by strong data quality standards using Informatica Data Quality.

In This Role, You Will

• Build and maintain scalable batch and near real-time data pipelines using AB Initio, Python, PySpark, PL/SQL and SQL to ingest, transform, and publish curated datasets across on-prem and Google Cloud platforms.
• Develop and optimize BigQuery transformations and data models, including partitioning, clustering, query optimization, and cost/performance tuning.
• Support modernization/migration from Teradata and Ab Initio workflows to GCP/BigQuery, including logic re-platforming, reconciliation, parallel runs, and controlled cutovers.
• Implement orchestration and scheduling for pipelines using legacy Autosys while driving migration toward Google Cloud Composer (Airflow), including dependency management, retries, SLAs, and backfills.
• Apply data governance and discovery practices using Dataplex: metadata management, dataset organization, classification support, and ensuring data is consumption-ready.
• Build and operationalize data quality controls using Informatica Data Quality: profiling, rule implementation, thresholds, exception handling, and embedding quality gates into pipelines.
• Ensure operational excellence: monitoring, alerting, runbooks, incident triage/root cause analysis, and continuous improvements to reliability and performance.
• Implement secure data engineering practices: least-privilege access, PII handling/masking where required, retention controls, and audit-friendly documentation.
• Partner with product, analytics, and engineering stakeholders to translate requirements into clear data contracts, curated datasets, and maintainable documentation (data dictionaries, reconciliation notes, operational runbooks).
• Use AI-assisted coding tools (e.g., GitHub Copilot, Devin, or similar) to accelerate development while maintaining strong code review discipline, testing, and secure coding standards.
• Closely partner with Product Owners, Architects and Engineers on definition, design, development, integration, testing and support of reliable and reusable Data pipelines.
• Analyze highly complex business requirements; generate technical specifications to design ETL processes.
• Act as an expert technical resource for analysis and provide critical direction to less experienced staff.
• Work with team members to provide insight into solving complex problems with middleware while leveraging enterprise and industry best practices (including scalability, availability, maintainability, and flexibility).

Required Qualifications

• 4+ years of Data Engineering experience, or equivalent demonstrated through one or a combination of the following: work experience, training, military experience, education
• 4+ years PL/SQL and SQL skills with proven experience in Oracle, Teradata, Python and/or BigQuery: complex query development, tuning, and debugging.
• 4+ years Ab Initio skills with proven experience to build complex graphs, Psets, performance tuning.
• 3+ years programming skills in Python; hands-on PySpark for distributed data processing
• 3+ years of ETL/ETL design, data warehousing concepts, and data modeling best practices

Desired Qualifications

• Production operations experience: monitoring, SLAs, incident response, root cause analysis, and performance optimization.
• Experience working in hybrid environments (on-prem + cloud) and supporting data migration/modernization initiatives.
• Experience with scheduling/orchestration in Autosys and Airflow-based orchestration (Cloud Composer direction).
• Experience with Git-based workflows, code reviews, and automated testing practices for data pipelines.
• Experience with Harness, Jenkins and uDeploy based CICD environments.
• Practical experience using AI-assisted coding tools in daily development to improve productivity without compromising quality or security.
• Ab Initio development/maintenance experience and/or hands-on migration of Ab Initio graphs to modern Spark/SQL patterns.
• Experience with Dataplex and broader data governance concepts (metadata, classification, stewardship, lineage practices).
• Experience with Informatica Data Quality implementation patterns (profiling, rules, scorecards/metrics, exception workflows).
• Experience designing near real-time patterns (micro-batch/event-driven concepts) and handling late-arriving/out-of-order data.
• Familiarity with GCP operational practices for data workloads (service accounts/IAM basics, job monitoring, quota/cost controls).