Senior Software Engineer, Data Engineering

Responsibilities

• Design, build, and maintain high-throughput streaming and batch data processing services, with a primary focus on raw and bronze-level ingestion.
• Develop and operate stream-based applications responsible for real-time data transformation, enrichment, validation, and routing.
• Own and evolve event schemas and data contracts, including Avro schemas and Schema Registry governance.
• Ensure scalability, fault tolerance, and performance of streaming and ingestion pipelines under heavy load.
• Contribute to platform-level concerns such as deployment automation, observability, operational tooling, and CI/CD.
• Participate in the design and implementation of cloud-native data infrastructure supporting real-time and batch workloads.

Requirements

• 5+ years of professional experience building production software systems, preferably in a distributed or data-intensive environment.
• Strong experience with Java (and/or Scala) as well as Python, in backend or data processing applications.
• Experience designing and operating streaming systems using Kafka or Kafka Streams (or similar).
• Experience working with event-driven architectures, including schema evolution and compatibility.
• Experience building real-time and/or near-real-time data pipelines at scale.
• Solid understanding of distributed systems concepts (partitioning, fault tolerance, backpressure, exactly-once/at-least-once semantics).
• Familiarity with Avro, Protobuf, or similar serialization formats and schema governance practices.

Pluses

• Experience with Databricks, particularly for ingestion, bronze-layer processing, or structured streaming.
• Experience deploying and scaling applications in containerized environments (e.g., Kubernetes, AKS).
• Experience working with artifact repositories (e.g., Artifactory, ProGet, Maven repositories).
• Experience with Infrastructure-as-Code (e.g., Terraform, Databricks Asset Bundles).
• Familiarity with the Microsoft Azure cloud ecosystem.
• Familiarity with Apache Spark.
• Familiarity with CI/CD pipelines, automated testing, and deployment workflows.

Pay

The pay range for this position in Andover, Massachusetts at the start of employment is expected to be between $130,000 - $150,000 USD per year. However, base pay offered is based on market location, and may vary further depending on individualized factors for job candidates, such as job-related knowledge, skills, experience, and other objective business considerations.