Senior Data Engineer

About the role

The Senior Data Engineer will help transform our cloud data systems by designing and operating architectures that drive analytical and business value from a wide range of data sources. This role partners closely with analysts, traders, product owners, and IT teams to deliver high-performance, resilient, and automated data pipelines, curated analytical datasets, and governed semantic models.

Responsibilities

• Design and operate Snowflake-centric analytical architectures supporting mixed workloads, including heavy read/query patterns, reporting, downstream applications, and AI/RAG use cases.
• Evaluate and apply the appropriate platform (Snowflake, Databricks, Postgres, ADLS) based on workload requirements, performance characteristics, and cost considerations.
• Build and maintain scalable, automated data ingestion and refresh pipelines at terabyte scale using Azure Data Factory, Azure Functions, Azure Logic Apps, Databricks, Python, and Snowflake.
• Integrate data from external vendors and internal systems using APIs, streams, flat files, event feeds, and relational databases; implement robust incremental and backfill strategies.
• Design and develop analytical data models, including dimensional models (facts, dimensions), conformed dimensions, and SCD patterns that balance usability, performance, and maintainability.
• Build and maintain governed semantic models / semantic layers (business entities, measures, metrics, hierarchies) to ensure consistent data consumption across BI tools, APIs, and AI-driven interfaces.
• Optimize Snowflake performance and cost, including warehouse sizing, query tuning, clustering and pruning strategies, and SQL best practices.
• Own operational readiness for data pipelines, including monitoring, alerting, runbooks, incident response, and ongoing reliability improvements.
• Develop and implement data quality validation and testing frameworks, including schema validation, reconciliation, anomaly detection, and freshness/completeness checks.
• Plan and execute work using agile methodologies, contributing to technical design reviews, documentation, and knowledge sharing.
• Collaborate directly with analysts and business stakeholders to understand data usage, clarify requirements, and translate data needs into actionable technical designs.

Requirements

• Bachelor’s degree in computer science, Engineering, Data Science, or equivalent practical experience.
• Strong, hands-on experience with Snowflake in production environments, including data loading patterns, query optimization, and cost management.
• Advanced SQL expertise (complex ANSI-SQL, window functions, performance tuning) and solid data warehousing fundamentals.
• 6+ years of experience with relational databases (e.g., SQL Server, Postgres, MySQL, Oracle), including schema design and query optimization.
• 6+ years of experience building and operating data ingestion and transformation pipelines on large datasets (batch and incremental).
• 2+ years of experience with Spark or distributed data processing frameworks (Databricks, Hadoop/Cloudera).
• 2+ years of experience with Azure data services, including Azure Data Factory, Azure Functions, Logic Apps, ADLS Gen2, Azure SQL, and CI/CD tooling (Azure DevOps or equivalent).
• Strong experience in data modeling, including dimensional modeling, SCDs, and designing curated “gold” datasets.
• Experience working with modern data file formats and ingestion strategies (Parquet, Avro, JSON; partitioning, compression, schema evolution).
• Proven experience supporting enterprise data quality, governance, and documentation.
• Practical experience applying AI to data platforms, including semantic models, RAG pipelines, or natural-language-to-data solutions.
• Strong Python programming skills for data acquisition, orchestration, and automation.
• Excellent communication skills, with the ability to explain technical concepts clearly to both technical and non-technical stakeholders.
• Demonstrated ownership mindset, strong troubleshooting skills, and commitment to continuous improvement through automation and better platform design.

Qualifications

• Master’s degree in Computer Science, Engineering, Data Science, or related field.
• 6+ years of relevant experience.
• Experience with Snowflake, Databricks, Azure data services, and traditional databases.
• Experience with distributed data processing frameworks like Spark or Hadoop.
• Experience with Azure Data Factory, Azure Functions, Logic Apps, and ADLS Gen2.
• Experience with modern data file formats and ingestion strategies.
• Experience with AI and natural language processing techniques.
• Experience with Python and other scripting languages.
• Experience with Agile methodologies and DevOps practices.

Skills

• Strong understanding of Snowflake, Databricks, and Azure data services.
• Advanced SQL and data warehousing skills.
• Experience with distributed data processing frameworks.
• Experience with modern data file formats and ingestion strategies.
• Experience with AI and natural language processing techniques.
• Strong Python programming skills.

Benefits

Comprehensive benefits package including health insurance, retirement plans, paid time off, and more.

Pay

$160,000 - $190,000

Schedule

Full-time, remote-friendly position.