Software Engineer, Data Infrastructure
About the role
Peregrine helps public safety organizations, state and local governments, federal agencies, and private-sector institutions address societal challenges with speed and accuracy. Our AI-enabled platform transforms siloed and disconnected data into operational intelligence, providing mission-critical information to support better, faster decision-making.
Responsibilities
- Design and operate a high-throughput, real-time data integration platform across diverse customer environments
- Architect a scalable open table format layer for reliable data storage at petabyte scale
- Build and optimize distributed data processing pipelines with Apache Spark and adjacent streaming technologies
- Drive performance, reliability, and cost efficiency across the full data infrastructure stack
- Collaborate with platform and product engineering teams to define data contracts, schemas, and integration patterns
- Establish best practices, tooling, and patterns that raise the quality bar for data infrastructure across the organization
Requirements
- Deep passion for data infrastructure
- Experience with open table formats, particularly Apache Iceberg
- Extensive hands-on experience with Apache Spark for batch and streaming data processing at scale
- Experience with real-time data integration and stream processing
- Experience with data pipeline orchestration using Airflow or similar tools
- Strong software engineering fundamentals in Python and/or Scala
- Experience with AWS or comparable cloud platforms, including S3-based data lake architectures
- Experience with Kubernetes and containerized deployment of data workloads
- Commitment to operational excellence
Qualifications
- 2-5 years of experience operating large-scale data infrastructure systems in production environments
- Strong technical vision with the ability to translate complex data requirements into clean, durable infrastructure designs
- Desire to own significant portions of the data stack end-to-end, from ingestion to serving
Skills
- Open table formats, particularly Apache Iceberg
- Apache Spark for batch and streaming data processing
- Real-time data integration and stream processing
- Data pipeline orchestration using Airflow or similar tools
- AWS or comparable cloud platforms
- Kubernetes and containerized deployment of data workloads
Benefits
Not specified
Pay
$160,000 - $220,000 Annually + Benefits + Equity (if applicable) + Bonus (if applicable)
Schedule
Not specified
Benefits
Not specified
Qualifications
Not specified
Skills
Not specified
Benefits
Not specified
Pay
Not specified
Schedule
Not specified
Benefits
Not specified
Qualifications
Not specified
Skills
Not specified
Benefits
Not specified
Pay
Not specified
Schedule
Not specified
Benefits
Not specified
Qualifications
Not specified
Skills
Not specified
Benefits
Not specified
Pay
Not specified
Schedule
Not specified
Benefits
Not specified
Qualifications
Not specified
Skills
Not specified
Benefits
Not specified
Pay
Not specified
Schedule
Not specified
Benefits
Not specified
Qualifications
Not specified
Skills
Not specified
Benefits
Not specified
Pay
Not specified
Schedule
Not specified
Benefits
Not specified
Qualifications
Not specified
Skills
Not specified
Benefits
Not specified
Pay
Not specified
Schedule
Not specified
Benefits
Not specified
Qualifications
Not specified
Skills
Not specified
Benefits
Not specified
Pay
Not specified
Schedule
Not specified
Benefits
Not specified
Qualifications
Not specified
Skills
Not specified
Benefits
Not specified
Pay
Not specified
Schedule
Not specified
Benefits
Not specified
Qualifications
Not specified
Skills
Not specified
Benefits
Not specified
Pay
Not specified
Schedule
Not specified
Benefits
Not specified
Qualifications
Not specified
Skills
Not specified
Benefits
Not specified
Pay
Not specified
Schedule
Not specified
Benefits
Not specified
Qualifications
Not specified
Skills
Not specified
Benefits
Not specified
Pay
Not specified
Schedule
Not specified
Benefits
Not specified
Qualifications
Not specified
Skills
Not specified
Benefits
Not specified
Pay
Not specified
Schedule
Not specified
Benefits
Not specified
Qualifications
Not specified
Skills
Not specified
Benefits
Not specified
Pay
Not specified
Schedule
Not specified
Benefits
Not specified
Qualifications
Not specified
Skills
Not specified
Benefits
Not specified
Pay
Not specified
Schedule
Not specified
Benefits
Not specified
Qualifications
Not specified
Skills
Not specified
Benefits
Not specified
Pay
Not specified
Schedule
Not specified
Benefits
Not specified
Qualifications
Not specified
Skills
Not specified
Benefits
Not specified
Pay
Not specified
Schedule
Not specified
Benefits
Not specified
Qualifications
Not specified
Skills
Not specified
Benefits
Not specified
Pay
Not specified
Schedule
Not specified
Benefits
Not specified
Qualifications
Not specified
Skills
Not specified
Benefits
Not specified
Pay
Not specified
Schedule
Not specified
Benefits
Not specified
Qualifications
Not specified
Skills
Not specified
Benefits
Not specified
Pay
Not specified
Schedule
Not specified
Benefits
Not specified
Qualifications
Not specified
Skills
Not specified
Benefits
Not specified
Pay
Not specified
Schedule
Not specified
Benefits
Not specified
Qualifications
Not specified
Skills
Not specified
Benefits
Not specified
Pay
Not specified
Schedule
Not specified
Benefits
Not specified
Qualifications
Not specified
Skills
Not specified
Benefits
Not specified
Pay
Not specified