Data Engineer - Senior Consultant

About the role

Guidehouse offers a comprehensive, total rewards package that includes competitive compensation and a flexible benefits package that reflects our commitment to creating a diverse and supportive workplace.

Responsibilities

• Develop and maintain scalable data ingestion, transformation, and curation pipelines using Databricks (Delta Lake, Delta Live Tables, Auto Loader) and AWS services, supporting consistent delivery of analytics-ready datasets across the Data Platform.
• Implement standardized batch and near real-time data pipelines that integrate legacy systems and cloud-native data sources, contributing to enterprise-wide data access, reuse, and platform consistency.
• Support full data pipeline lifecycle activities, including intake, requirements analysis, source profiling, and technical implementation, aligning development work with defined intake processes, SLAs, and governance.
• Build production-ready data pipelines that meet defined technical and documentation standards, including participation in validation, testing, and release processes to enable reliable and compliant production deployments.
• Apply data quality checks, validation rules, and observability practices to improve pipeline reliability, support monitoring, and contribute to platform stability and operational performance targets.
• Integrate pipelines with AWS services (e.g., S3, streaming frameworks, APIs) and enterprise data tools to support secure, scalable data movement and interoperability across the ecosystem.
• Contribute to governed data engineering practices by implementing metadata capture, lineage tracking, and supporting access control patterns aligned to enterprise data governance standards.
• Support analytics and reporting use cases by preparing curated datasets and enabling consumption through SQL-based access, dashboards, and enterprise BI tooling.

Requirements

• Bachelor’s degree is required
• Minimum FOUR (4) years of experience in data engineering, with hands-on development of data pipelines in cloud or distributed data environments
• Strong proficiency in Python, PySpark, and SQL for building and maintaining scalable ETL/ELT pipelines
• Experience working with Databricks and Delta Lake to support ingestion, transformation, and curated data layer development
• Working knowledge of AWS data services (e.g., S3, IAM, VPC) and integration patterns supporting secure and scalable data architectures
• Experience implementing data quality checks, monitoring, and basic performance optimization techniques for pipeline efficiency and reliability
• Familiarity with data governance concepts, including metadata, lineage, and access control frameworks in regulated environments
• Experience working within Agile delivery environments and contributing to CI/CD-enabled development workflows

Qualifications

• Experience supporting enterprise data platforms or federal data modernization initiatives, particularly in highly regulated environments
• Exposure to streaming technologies such as Kafka, Kinesis, or EventBridge for near real-time data processing
• Familiarity with Databricks Unity Catalog and governance capabilities (RBAC, data masking, auditing)
• Experience using metadata/catalog tools such as Informatica EDC or similar platforms
• Understanding of DevSecOps practices, including automated testing, deployment, and environment promotion across dev/test/prod
• Exposure to performance optimization techniques (e.g., partitioning, Z-ordering, clustering) for large-scale data processing workloads

Skills

• Python
• PySpark
• SQL
• Databricks
• Delta Lake
• AWS services (S3, IAM, VPC)
• Streaming technologies (Kafka, Kinesis, EventBridge)
• Databricks Unity Catalog
• Metadata/catalog tools (Informatica EDC)
• DevSecOps practices
• Performance optimization techniques (partitioning, Z-ordering, clustering)

Benefits

• Medical, Rx, Dental & Vision Insurance
• Personal and Family Sick Time & Company Paid Holidays
• Parental Leave and Adoption Assistance
• 401(k) Retirement Plan
• Basic Life & Supplemental Life
• Health Savings Account, Dental/Vision & Dependent Care Flexible Spending Accounts
• Short-Term & Long-Term Disability
• Student Loan PayDown
• Tuition Reimbursement, Personal Development & Learning Opportunities
• Skills Development & Certifications
• Employee Referral Program
• Corporate Sponsored Events & Community Outreach
• Emergency Back-Up Childcare Program
• Mobility Stipend

Pay

Details on pay are not specified in this posting.

Schedule

Details on schedule are not specified in this posting.