Data Engineer

Key Responsibilities

• Design, develop, and support scalable data pipelines and enterprise data integration solutions.
• Build and maintain batch and real-time data ingestion, transformation, and processing frameworks.
• Develop cloud-native data engineering solutions supporting enterprise data lake, warehouse, and lakehouse platforms.
• Implement ETL/ELT processes for structured, semi-structured, and unstructured data sources.
• Support Master Data Management (MDM) initiatives across security, account, client, and reference data domains.
• Collaborate with data architects, business analysts, governance teams, and application teams to support enterprise data initiatives.
• Implement data quality validation, monitoring, metadata management, and lineage processes.
• Support cloud migration and modernization efforts involving legacy and enterprise data platforms.
• Optimize data processing, storage, and pipeline performance for scalability and operational efficiency.
• Ensure compliance with enterprise security, governance, and regulatory standards within financial services environments.
• Support reporting, analytics, and downstream consumption platforms through reliable and trusted data delivery.

Required Skills & Experience

• Strong hands-on experience in Data Engineering and enterprise-scale data integration.
• Proven experience developing scalable ETL/ELT pipelines and distributed data processing solutions.
• Experience working with modern cloud-based data platforms and data ecosystems.
• Hands-on expertise with:
• Strong SQL expertise along with programming/scripting experience in Python, PySpark, or Snowpark.
• Experience with dbt (Data Build Tool) for:
• Data transformation and modeling
• ELT pipeline development within Snowflake/Databricks
• Modular, reusable SQL-based data workflows
• Data testing, documentation, and version control integration
• Experience with cloud platforms such as Azure, AWS, or GCP, including integration with Snowflake and Databricks.
• Solid understanding of data lake, data warehouse, and lakehouse architectures, and their implementation across platforms.
• Experience with orchestration and workflow tools (e.g., Airflow, Databricks Workflows, Snowflake Tasks) for pipeline scheduling and automation.
• Experience supporting Master Data Management (MDM) and enterprise data governance initiatives.
• Familiarity with metadata management, data lineage, data cataloging, and data quality processes.
• Experience integrating diverse data sources, including:
• APIs and microservices
• File-based ingestion (batch)
• Real-time/streaming data (e.g., Kafka, Spark Streaming)
• Knowledge of performance tuning, cost optimization, and scalability techniques across both Spark-based and Snowflake environments.
• Understanding of enterprise security, compliance, and governance standards, including RBAC, data masking, and encryption.
• Experience working in Agile and DevOps environments, including CI/CD for data pipelines.

Preferred Qualifications

• Financial Services or Banking industry experience preferred.
• Experience supporting regulatory, risk, compliance, or operational reporting data environments.
• Exposure to real-time data processing and streaming technologies.
• Familiarity with CI/CD processes and infrastructure automation.
• Strong analytical, troubleshooting, and problem-solving skills.
• Excellent communication and collaboration skills.

Education

• Bachelor's degree in Computer Science, Information Systems, Engineering, or related field.