Principal Data Engineer
About the role
We are seeking a highly experienced Principal Data Engineer to provide technical leadership for mission-critical Security/Product Master Data Platforms and other enterprise-scale data platforms that support the entire enterprise.
Responsibilities
Master Data Platforms and other enterprise-scale data platforms, ensuring scalability, reliability, availability, performance, and enterprise-wide reuse.
Establish engineering standards, data architecture patterns, integration patterns.
Define and drive the target-state architecture for enterprise Security/Product, and design principles for mission-critical master data and enterprise-scale data platforms.
Lead system design and modernization roadmaps for high-impact initiatives across security, product, master data, and other enterprise-scale data domains.
Provide technical leadership across multiple teams, not limited to a single project or squad.
Act as a trusted advisor to leadership on technology strategy, trade-offs, and long-term platform evolution.
Drive alignment across engineering, data, and platform teams to ensure consistency and reusability.
Lead the design and development of enterprise-grade Python applications and distributed systems.
Oversee architecture and implementation of data pipelines, APIs, and large-scale data processing frameworks.
Ensure solutions are designed with high availability, fault tolerance, and observability.
Lead end-to-end engineering ownership for mission-critical database and master data platforms, including development, support, maintenance, lifecycle management, performance, reliability, and operational excellence.
Apply advanced database optimization strategies across Oracle and related platforms, including performance tuning, partitioning, query optimization, resiliency, recoverability, and scalability.
Ensure efficient data modeling, storage, and access patterns across platforms.
Lead transition of legacy and operational master data capabilities toward modern cloud data lakehouse architecture using S3, Iceberg, Redshift, and medallion patterns for ingestion, transformation, curation, quality, and governed consumption.
Architect and standardize CI/CD pipelines using Jenkins and modern DevOps practices.
Drive adoption of automation-first principles across build, test, and deployment workflows.
Promote DevSecOps best practices and governance controls.
Lead cloud modernization of enterprise master data capabilities, including transition to cloud data lakehouse architecture and medallion-based bronze, silver, and gold data layers.
Influence AWS-based data architecture decisions, including appropriate use of Spark, AWS EMR, cloud storage, orchestration, data quality controls, and scalable consumption patterns.
Drive modernization while preserving operational continuity, enterprise availability, data trust, cost discipline, security, and compliance expectations.
Establish frameworks for performance engineering, observability, monitoring, SLAs, SLOs, SLIs, and operational readiness for enterprise-wide master data platforms.
Lead root cause analysis of critical production issues and define systemic improvements across platform reliability, data quality, resiliency, recovery, and downstream dependency management.
Ensure the platform meets enterprise-grade resiliency, recovery, security, and availability requirements for mission-critical data distribution.
Demonstrate proficiency in using AI tools and AI-assisted engineering practices to improve individual and team efficiency, productivity, solution quality, and delivery velocity.
Mentor senior engineers and leads, raising the overall technical bar of the organization.
Drive knowledge sharing, standards adoption, and engineering excellence initiatives.
Serve as a role model for engineering best practices and problem-solving.
Qualifications
15+ years of progressive engineering experience, including technical leadership for enterprise-scale data, database, or platform engineering capabilities.
Strong hands-on data engineering experience with Python, Oracle/ODI, Spark, AWS EMR, Redshift, Iceberg, and large-scale data processing patterns.
Experience developing, supporting, maintaining, and modernizing mission-critical database, master data, or enterprise-scale data platforms in production environments.
Strong functional and data understanding of Security/Product Master Data Platforms, with working knowledge of enterprise data domains such as Clients, Accounts, Assets and Liabilities, Trades, and Activities.
Experience with cloud data lakehouse architecture, medallion patterns, data quality controls, lineage, governed consumption, and reusable data product patterns.
Proficiency using AI tools and AI-assisted engineering practices to improve productivity, solution quality, documentation, troubleshooting, and delivery velocity.
Strong communication, technical influence, and cross-functional leadership skills with the ability to engage engineering, architecture, governance, operations, product, and business stakeholders.
Experience in Agile/Scrum at scale, such as SAFe or similar frameworks.
Experience in financial services, wealth management, capital markets, Security/Product Master Data, or enterprise reference data platforms.
Experience leading modernization of enterprise master data or reference data platforms from legacy technologies toward cloud data lakehouse, medallion architecture, or governed data product models.
Experience with enterprise data governance, data quality, lineage, stewardship, metadata management, and enterprise consumption patterns at scale.
Experience applying AI-enabled engineering practices to improve development efficiency, operational productivity, solution quality, and delivery velocity.