Data Integration Engineer :: Remote

Responsibilities

• Design, build, and maintain integration pipelines connecting source systems to downstream targets, using containerized Python services on Azure Container Apps with Service Bus and Event Hub.
• Build and operate flows against Salesforce's Pub/Sub API over gRPC, including Platform Event publish and subscribe, Avro decode, durable replay checkpointing, and reconnection handling for long-lived subscriptions.
• Implement idempotent processing, deduplication, retry policies, and dead-letter consumers so at-least-once delivery does not produce duplicate or lost records in target systems.
• Design and consume REST APIs, including OAuth 2.0 with token caching, rate limit handling, and error semantics that distinguish retryable from terminal failures.
• Build and maintain Python-based transformation and validation services that enforce business rules as data moves between systems.
• Land request and response event streams into Snowflake via Event Hub as a durable parallel stream, so integration activity is available for reporting, reconciliation, and audit.
• Instrument integrations for distributed tracing, structured logging, and correlation across asynchronous hops, and deliver monitoring hooks, alert thresholds, runbooks, and escalation paths to Operations before go-live.
• Write unit and integration tests covering failure and replay paths, maintain CI/CD pipelines (e.g., GitHub Actions) across DEV, SIT, UAT, and PROD, and translate stakeholder requirements into technical designs, including which pattern applies.