Data & Integration Ops Engineer

About the role

The Data & Integration Ops Engineer is an experienced data engineering professional responsible for the stable, secure, and efficient operation of Focus's data platforms and integration pipelines. This role bridges data engineering and operations - applying DataOps and DevOps best practices to own SLAs, monitor systems proactively, and resolve issues before they impact the business. You'll work closely with data engineers, analytics engineers, and platform teams to ensure data is delivered accurately and on time.

Primary Responsibilities

• Data Pipeline Operations & Reliability: Oversee the end-to-end operation of data pipelines (ELT workflows) across development, UAT, and production environments. Monitor pipeline schedules (e.g., Airflow DAGs) and ensure on-time data delivery to meet or exceed defined SLAs for data availability and quality.
• Integration Pipeline Operations: Monitor and troubleshoot integration workflows across Azure Integration Services (Logic Apps, Event Hub, AKS-based transformation jobs) that move data between source systems (e.g., Salesforce FSC) and downstream targets. Diagnose failures in integration code and coordinate with Infrastructure/Cloud Engineering and Cyber teams when issues trace back to underlying Azure infrastructure.
• Incident Response & Recovery: Act as the primary point of contact for data platform incidents during business hours, diagnosing issues in real-time and coordinating rapid recovery efforts. Lead root cause analysis and implement preventive measures to minimize future disruptions.
• Operational Governance & Compliance: Serve as a steward of the data platform, managing production data access and governance. Administer Snowflake RBAC and access policies, and audit write-access permissions to production datasets and systems to ensure data integrity, security, and compliance with internal policies and industry regulations.
• Troubleshooting & Performance Optimization: Identify and troubleshoot pipeline failures or data quality issues, including root cause diagnosis of failed dbt transformations or upstream data problems. Optimize pipeline performance (e.g., query tuning, resource scaling) across both data pipelines and integration workflows to improve throughput and reduce latency, ensuring robust performance of the data platform.
• Platform Monitoring & Improvement: Implement monitoring, logging, and alerting for data workflows and platforms, using these tools to proactively detect anomalies. Analyze performance metrics and incident patterns to drive continuous improvements, such as enhancing resiliency, refining SLAs, and updating processes to prevent recurring issues.
• Cross-Team Collaboration: Work closely with Data Engineering, Analytics, Infrastructure/Cloud Engineering, Cyber, and IT Ops teams to prioritize and address production data issues. Provide guidance and mentorship on operational best practices to other data team members, fostering a culture of reliability and quality.

Required Skills

• Data Pipeline & Orchestration: Strong hands-on experience with data workflow management systems (especially Apache Airflow/Astro for DAG orchestration) and familiarity with scheduling, monitoring, and maintaining complex DAGs in production.
• Data Transformation & Tools: Proficiency with SQL and data transformation frameworks like dbt (Data Build Tool) for building and troubleshooting data models. Capability to debug SQL queries and pipeline scripts to resolve data quality or performance issues in a timely manner.
• Programming & Scripting: Advanced programming skills in Python (or similar languages) for writing data pipeline jobs and automation scripts. Experience with version control (e.g., Git) and understanding of CI/CD tools/processes for deploying data pipelines and platform changes.
• Monitoring & Incident Response: Experience implementing monitoring and alerting systems (using tools such as logging frameworks, observability dashboards) to track SLAs, runtime metrics, and quickly detect pipeline failures. Skilled in systematic troubleshooting and root cause analysis for complex systems under pressure.
• Data Platforms & Cloud: Solid understanding of Snowflake (RBAC, secure views, dynamic tables, resource monitors) and Astro/Airflow, including their operational aspects (performance tuning, security, monitoring). Strong working knowledge of Azure services relevant to data and integration pipelines (networking basics, APIM, Event Hub, AKS, Logic Apps).

Qualifications

• Education: Bachelor’s degree in Computer Science, Software Engineering, or a related technical field, or equivalent hands-on experience.
• Experience: Typically 5+ years of professional experience in data engineering, data operations (DataOps), or a related field, including substantial experience managing production data pipelines and platforms. Experience applying DevOps, DataOps, or SRE practices to production data systems is highly desirable.
• Expertise: Proven track record of operational excellence in a data-focused environment – e.g., owning and improving SLAs, handling production incidents, and implementing robust automation. Familiarity with industry best practices in DataOps/Data Engineering and data governance standards.