Data Engineer

About the role

The Data Engineer is responsible for designing, building, and maintaining scalable data pipelines, data platforms, and data warehouse/data lakehouse architectures that support enterprise analytics, reporting, automation, and AI-enabled solutions. This role ensures that data is accurate, reliable, well-governed, secure, and optimized for consumption across Microsoft Fabric, Power BI, generative AI, AI agents, and future technologies. The Data Engineer will enable trusted, AI-ready data foundations and support client- and vendor-facing technology solutions that are critical to business operations, growth, and service delivery.

Essential Functions

• Design, build, and maintain robust ETL/ELT data pipelines to ingest, process, and integrate data from multiple source systems (e.g., CRM platforms, APIs, and external data sources) into Azure and Microsoft Fabric.
• Develop and manage the enterprise data platform, including lakehouse architecture (bronze, silver, gold layers), ensuring data is structured, scalable, and optimized for analytics.
• Ensure the quality, integrity, and consistency of data through validation processes, monitoring, and proactive issue resolution.
• Optimize performance and scalability of data workflows, pipelines, and storage to support efficient data processing and reporting.
• Implement and enforce data governance, security, and compliance standards, including data lineage, access controls, and regulatory requirements.
• Enable AI-ready and agentic enterprise capabilities by preparing governed datasets, metadata, lineage, and integration patterns that allow AI agents, automation, and decision-support tools to operate safely and effectively.
• Support the design, delivery, and ongoing operation of client- and vendor-facing data integrations, reporting assets, APIs, and platform services that require reliable, secure, and scalable data engineering practices.

Responsibilities & Duties

• Build and maintain data pipelines using Azure Data Factory, Fabric Data Pipelines, and other integration tools.
• Develop data transformation logic using SQL, PySpark, or similar technologies to standardize and prepare data for analytics use.
• Design and manage lakehouse structures in Microsoft Fabric, including bronze, silver, and gold data layers.
• Collaborate with BI developers to ensure data is modeled and structured appropriately for Power BI semantic models and reporting.
• Maintain documentation of data pipelines, data models, and system architecture to support transparency, governance, and knowledge sharing.
• Partner with business stakeholders and IT teams to understand data requirements and deliver scalable data solutions.
• Support data governance initiatives, including metadata management, lineage tracking, and data classification (e.g., integration with Purview).
• Prepare and expose curated, governed, and well-documented data assets for AI, generative AI, semantic search, retrieval-augmented generation, and agent-based enterprise workflows.
• Establish monitoring, observability, alerting, and support practices for AI-enabled and externally facing data solutions to ensure reliability, traceability, performance, and timely issue resolution.

Job requirements

• Education & Experience

• Required:
• Bachelor’s degree in Information Systems, Computer Science, Data Engineering, or a related field, or equivalent work experience.
• 5+ years of experience in data engineering, data integration, or data platform development.
• Strong experience with data pipeline development and ETL/ELT processes.
• Proficiency in SQL and data modeling concepts (warehouse/lakehouse).
• Experience working with cloud-based data platforms (e.g., Azure Data Factory, Azure SQL, Microsoft Fabric, or similar).
• Understanding of data governance, data quality, and security principles.
• Experience designing or supporting data assets, integrations, or platform services used by enterprise applications, automation, analytics, or AI-enabled workflows.
• Preferred:
• Experience with Microsoft Fabric, Lakehouse architecture, and Delta Lake concepts.
• Experience with PySpark or distributed data processing frameworks.
• Familiarity with Power BI, semantic models, and analytics workflows.
• Experience with data cataloging and governance tools (e.g., Microsoft Purview).
• Experience working in regulated environments with compliance requirements (e.g., PCI, financial services).
• Familiarity with generative AI, retrieval-augmented generation, semantic search, AI agent concepts, responsible AI practices, or AI-ready data architecture.
• Experience supporting client- or vendor-facing technology solutions, secure data exchanges, APIs, integrations, or externally consumed reporting platforms.
• DP-600, DP-700, AZ-305

Travel

Minimal travel expected; occasional travel may be required based on project needs.

Work Environment

This job operates in a clerical, office setting. This role routinely uses standard office equipment such as computers, phones, photocopiers, filing cabinets and fax machines.

Physical Demands

This is a largely sedentary role; however, some filing may be required. This would require the ability to lift files, open filing cabinets and bend or stand as necessary.

Position Type & Expected Hours of Work

This is a full-time, exempt position. Days and hours of work are Monday through Friday 8:30 a.m. to 5 p.m. Some flexibility in hours is allowed based on management’s approval.

Job Compensation & Benefits

Through our competitive compensation and benefits program, we seek to attract, retain and motivate a high-performing and increasingly diverse Associate population. Our compensation and benefit structure is comprised of many factors, including but not limited the size of our Firm, our financial position, the industry, business objectives, market salary information, benchmark pay levels and practices against the respective peer group (to ensure equal pay for equal work, with allowable pay differences based on factors not prohibited by law), and unique circumstances of the Firm that are subject to change.

Compensation
Minimum: $116,000.00 - Maximum: $135,000.00 USD

Benefits
Heavily subsidized medical insurance options
Generous employer HSA contributions
100% employer-paid Short-Term Disability, Long-Term Disability, Accident, Basic Life Insurance, and Telehealth/Virtual Urgent Care/Whole Mental Health
Dental and Vision plans
Unlimited PTO
Organization-paid holidays, including an extended Winter Break
Safe Harbor 401k Plan
Employer-paid professional dues, licenses, and renewals
Continuing Education resources
Family Planning Assistance Program and Paid Parental Leave
Subsidized fitness programs

Equal Employment Opportunity

Cain Watters & Associates is committed to upholding the principles of the Equal Pay and Opportunity Act (EPOA). If you have any concerns regarding compliance with this policy, we encourage you to report them immediately by contacting HR at hr@cainwatters.com. Your concerns will be handled with the utmost confidentiality and care.