Staff Software Engineer - Reporting, Data Platform & Observability

About the role

OneTrust’s mission is to enable innovation through the responsible use of data and AI. The Challenge: OneTrust is seeking a Staff Software Engineer to join the Reporting and Data Platform team. This is a hands‑on individual contributor role focused on designing, building, operating, and improving distributed backend services and data‑processing platforms. You will work across Java microservices and Python/PySpark data pipelines, with a strong focus on reliability, scalability, performance, and observability. You will take complex or ambiguous problems from investigation through production delivery and help improve the systems that power reporting and data‑driven experiences.

Responsibilities

• Own complex features and technical improvements from discovery through production rollout, making substantial hands‑on contributions across backend services and data‑processing pipelines.
• Investigate ambiguous problems, identify root causes, evaluate trade‑offs, and implement pragmatic solutions that improve code quality, maintainability, automated testing, and operational readiness.
• Review code and technical designs, document important implementation decisions and system behavior, and partner with product managers, engineers, and other teams to clarify requirements and deliver outcomes.
• Apply AI‑assisted engineering tools such as Devin, Claude, or similar systems to accelerate delivery while maintaining production‑quality design, code, tests, security, and operational readiness.
• Design and implement production services using Java, Spring Boot, and Maven, including APIs, asynchronous workflows, report generation, aggregation, export, and scheduling capabilities.
• Develop event‑driven functionality using Kafka and related messaging patterns, and work with caching technologies, relational storage, and service‑to‑service integrations.
• Improve service performance, scalability, fault tolerance, and resource efficiency through appropriate patterns for retries, idempotency, caching, backpressure, concurrency, and failure recovery.
• Diagnose issues across services, queues, databases, and downstream dependencies, and modernize established capabilities incrementally while maintaining production stability.
• Build and maintain ingestion and transformation pipelines using Python, PySpark, Azure Databricks, and Delta Lake across batch and streaming workloads.
• Implement schema evolution, checkpoint management, deduplication, replay, late‑arriving‑data handling, and standardized data‑layer patterns.
• Optimize Spark joins, partitioning, Delta operations, cluster utilization, and query performance while troubleshooting failed, delayed, or inefficient Databricks workloads.
• Protect tenant boundaries across joins, aggregations, deduplication, and Delta operations; implement data‑quality controls; and monitor data freshness, completeness, and correctness.
• Work securely with Azure storage, identities, secrets, and encryption mechanisms.
• Improve observability across backend services, event‑driven workflows, and data pipelines using meaningful metrics, structured logs, traces, and business telemetry.
• Build and maintain actionable dashboards, monitors, and alerts using Datadog and Grafana, applying OpenTelemetry, Prometheus, and Micrometer patterns where appropriate.
• Participate in the on‑call rotation and incident‑response workflows, using PagerDuty, Datadog monitors, or equivalent platforms to diagnose production issues and drive sustainable resolution.
• Reduce recurring alerts and operational toil by improving alert quality, eliminating noisy or non‑actionable monitors, creating runbooks and diagnostic tools, and implementing corrective actions from blameless incident reviews.
• Improve end‑to‑end correlation and monitor availability, error rates, latency, ingestion lag, data freshness, event throughput, consumer lag, job health, rejected records, checkpoint health, tenant‑specific failures, data‑quality violations, and Spark resource utilization.

Skills

• Self‑directed, hands‑on engineer who enjoys solving complex problems across distributed services and data platforms.
• Systems thinker who evaluates trade‑offs, takes ownership of production behavior, and uses clear design thinking to simplify solutions and reduce delivery cycles.
• Motivated by building reliable, secure, and maintainable systems and improving them over time.
• Comfortable working across service, platform,