Senior Software Engineer - SRE
OneTrust · Atlanta, GA · 2 wk ago
Engineering$116k–$175k/yrFull-time
The Challenge
- Build and implement application observability and platform monitoring tools to continuously improve the customer experience
- Eliminate toil by automating processes, tuning alerts, and improving code where it is most needed
- Frequently evaluate new ideas and trends to identify potentially useful tools and techniques
- Collaborate with different functional groups to identify gaps, prioritize, and resolve issues
- Define, implement, and maintain Service Level Indicators (SLIs) and Service Level Objectives (SLOs) aligned with customer experience
- Design and instrument SLIs such as latency, error rates, and availability across critical services
- Manage and enforce error budgets to balance system reliability with product feature velocity
- Improve alert quality by reducing noise and focusing on actionable, high-signal alerts
- Embed with product teams to review architectures and catch reliability risks early
- Share your knowledge and experience with the Engineering organization
- Share your findings with technical leadership and senior management
- Create scripts in Python, Bash, Java, or Ruby for operational automation and incident response
Requirements
- Bachelor's degree in computer science, Engineering, or related technical or business field
- 4+ years of application development experience with Java or other equivalent language
- Experience with Spring environment
- Experience in cloud-based infrastructure (Azure, AWS, GCP, etc.)
- A knowledge of the importance of centralizing logging, metrics dashboards, and alerting
- A good awareness of databases (ideally SQL/NoSQL)
- Hands-on experience with observability tools (Datadog, Prometheus, Grafana, etc.)
- Knowledge with CI/CD pipelines and infrastructure-as-code (Terraform, Helm, Jenkins, GitLab)
- Experience with Kubernetes and container orchestration (EKS/AKS/GKE)
- Experience with distributed systems at scale
- Familiarity with service meshes and microservices architectures
- Experience with chaos engineering tools (Gremlin, Chaos Monkey)
- Background in product-facing services with high traffic scale
Qualifications
- Experience with AI-assisted incident response systems (root cause analysis, log summarization, anomaly triage)
- Experience developing or integrating Large Language Models (LLM)-based tools to reduce Mean Time To Repair (MTTR) and improve alert quality
- Experience applying machine learning techniques for anomaly detection, capacity prediction, or failure pattern analysis
- Experience deploying AI systems in production (not just experimentation)
- Knowledge with vector databases, embeddings, or Retrieval-Augmented Generation (RAG) architectures for operational intelligence
- Experience with prompt engineering and evaluating LLM outputs in the reliability workflow
Skills
- Python, Bash, Java, or Ruby
- Spring environment
- Cloud-based infrastructure (Azure, AWS, GCP, etc.)
- Observability tools (Datadog, Prometheus, Grafana, etc.)
- CI/CD pipelines and infrastructure-as-code (Terraform, Helm, Jenkins, GitLab)
- Kubernetes and container orchestration (EKS/AKS/GKE)
- Chaos engineering tools (Gremlin, Chaos Monkey)
- Product-facing services with high traffic scale
- Large Language Models (LLM)-based tools
- Machine learning techniques
- Vector databases, embeddings, or RAG architectures
- Prompt engineering and evaluating LLM outputs
Benefits
- Annual base pay range: $116,475 - $174,712.50 USD