Data Ops Engineer

Responsibilities

• We are seeking a Data Operations Engineer to design, build, and maintain real-time data ingestion pipelines.
• In this role, you will be responsible for the reliable flow of streaming data from a wide range of sources into our data platform, ensuring data quality, observability, and scalability.
• You'll partner closely with data engineers, platform engineers, and analytics teams to deliver trustworthy, low-latency data that supports operational decisions.
• Aid the team in delivering continual data feeds to users and monitoring the status of the health of data quality and overall data ingest.
• Build resilient pipelines with appropriate backpressure, prioritization, retries, and error-handling strategies.
• Employ a variety of data manipulation and visualization tools to effectively convey status and historical trends to leadership, users, and data team.
• Collaborate with platform, software, and other data engineers to (re)configure data ingestion pipelines to be more reliable.
• Support the incident management process to ensure that incidents are documented and resolved quickly. Perform root cause analysis to understand and prevent repeated occurrences of data outages.
• Develop and maintain software to automate monitoring of real-time feeds and alert for timeliness, volume, lineage, and distribution data issues.
• Process learnings and rely on historical data from data pipelines, translating them into actionable steps to improve data ingest.
• Partner with security and governance teams to enforce encryption, authentication, authorization, and data classification.
• Demonstrate proficiency with frequent-used scripting languages (Python, bash) commonly used in data science applications and data analytics.

Qualifications

• U.S. citizenship and an active TS/SCI clearance.
• Bachelor of Science required in Computer Science, Mathematics, EE, Physics, Information Systems, or Information Technology.
• 3+ years of experience in data engineering, data operations, or DevOps roles supporting production data pipelines.
• Tools/Apps/Platforms: NiFi, Kafka, Grafana, Prometheus, Apache Flink/Spark Streaming, Snowflake, Elasticsearch, Kafka, MQTT, JMS.
• Operating Systems: Windows, Linux (RedHat).