Forward Deployed Data Engineer with Security Clearance

Responsibilities

• Implement real-time data pipelines using MQTT and Redpanda for stream processing.
• Implement offline data pipelines using Dagster for batch processing.
• Parse and process binary message formats from various data sources.
• Build data warehouses using PostgreSQL, Apache Iceberg, Parquet, and S3.
• Design data models that support high-performance analytical queries.
• Validate and normalize diverse data sources.
• Improve local development environments and CI/CD workflows using modern tooling and GitHub Actions.

Requirements

• Expertise in time-series data processing and analysis (windowing, resampling, interpolation, etc.).
• Proficiency in Python and Rust for data engineering workflows.
• Experience with binary message parsing.
• Experience with row-based and columnar data formats.
• Experience with OLTP and OLAP databases.
• Knowledge of distributed systems, streaming architectures, and batch processing patterns.
• Hands-on experience with workflow orchestration platforms such as Dagster or Airflow.
• Hands-on experience with streaming platforms such as Redpanda or Kafka.
• Experience working with binary message formats such as Protobuf.