Senior Data Engineer
CPI Security · Charlotte, NC · 3 wk ago
On-siteEngineeringFull-time
What You'll Do
- Spend approximately 70% of your time coding, building pipelines, and implementing solutions.
- Dedicate 30% to architectural design, standards definition, and technical guidance.
- Work alongside engineers and provide mentorship through code reviews, pair programming, and technical design sessions.
- Provide architectural guidance and hands-on mentorship to engineers.
Data Architecture & Design
- Define and document reference architectures, design patterns, and standards for the enterprise data platform.
- Create technical design documentation, data flow diagrams, and architectural decision records (ADRs).
- Establish data modeling standards, naming conventions, and best practices across the platform.
Architecture Governance
- Establish and maintain data modeling standards, design patterns, and architectural guidelines.
- Review and approve technical designs to ensure alignment with architectural principles and enterprise standards.
- Collaborate with stakeholders to define data governance policies and ensure compliance with security requirements.
Technical Mentorship & Collaboration
- Provide architectural guidance and hands-on mentorship to engineers through code reviews, pair programming, and technical design sessions.
- Share expertise in data vault modeling, dbt development, and cloud data engineering best practices.
- Foster a culture of technical excellence and continuous learning within the team.
Data Vault Implementation
- Design and implement data vault 2.0 modeling patterns to build a scalable, audit-friendly enterprise data platform.
Modern Data Engineering
- Build and maintain automated data pipelines using dbt (Cloud/Core), Python, and Snowflake.
- Implement comprehensive data quality testing frameworks using dbt tests, custom Python validations, and automated monitoring.
Cloud Data Platform Development
- Architect and implement an enterprise data platform on Snowflake, including automated deployment pipelines, data quality frameworks, and monitoring solutions.
- Enable reliable, scalable, and automated data workflows by implementing DataOps best practices for continuous integration, testing, deployment, and monitoring.
- Establish and maintain Snowflake security governance through role-based access control (RBAC).
External Data Integration
- Integrate and operationalize data from external systems such as CRM, ERP, and third-party platforms via secure cloud data sharing, CDC, and APIs.
DataMart & Dimensional Modeling
- Design and build data marts using dimensional modeling techniques (Kimball methodology).
ETL/ELT Pipeline Development
- Design and implement robust data transformation models using dbt, SQL, and Python.
Data Quality & Testing
- Implement comprehensive data quality testing frameworks using dbt tests, custom Python validations, and automated monitoring.
Cloud Migration Support
- Play an integral role in planning, designing, and implementing data migration strategies from legacy on-premises SQL Server systems to our modern Snowflake cloud platform.