Principal Engineer Software, Dev & Production Infrastructure (Chronosphere)

Job Summary

Chronosphere, a Palo Alto Networks company, is the observability platform built for control in the modern, containerized world. Chronosphere empowers customers to focus on the data and insights that matter by reducing complexity, optimizing costs, and remediating issues faster. Chronosphere reduces data volumes and associated costs by 84% on average while saving developers thousands of hours. Recognized as a leader by major analyst firms, Chronosphere is trusted by the world's most innovative brands, including DoorDash, Affirm, and Zillow.

About the Role

This role is remote, but distance is no barrier to impact. Our hybrid teams collaborate across geographies to solve big problems, stay close to our customers, and grow together. You will be part of a culture that values trust, accountability, and shared success where your work truly matters.

Two Impact Areas

• Production Engineering (Reliability & Scale)
• The Core Focus: Ensuring all live components of our global platform are optimized, highly reliable, and battle-tested.
• Key Responsibilities:
• Solve complex distributed systems problems at the largest scales in the world.
• Scale our production infrastructure and architecture globally using Kubernetes, GCP, and AWS.
• Use Go to build and operate highly scalable, resilient backend services.
• Develop the internal tools necessary to monitor, alert, and optimize our live production footprint.
• Dev Infrastructure (Developer Velocity & Platforms)
• The Core Focus: Building the platform and tooling that enables developer velocity and software reliability for the entire Chronosphere engineering organization.
• Key Responsibilities:
• Architect & Build: Design high-scale developer tooling, dynamic testing environments, and CI/CD pipelines in a 100% modern, containerized microservices ecosystem.
• Infrastructure as Code (IaC): Treat infrastructure as a first-class citizen by defining and managing entire ephemeral environments using declarative IaC (Terraform).
• Drive Systemic Quality: Identify and eliminate systemic bottlenecks across the development lifecycle through architectural changes, advanced tooling, and near-real-time telemetry processing.

Qualifications

• Experience: 8–10+ years of relevant experience in high-scale infrastructure, production engineering, systems architecture, or developer platform roles.
• Language Agnosticism: 7 + years of experience in at least one backend language (e.g., Go, Python, Java, C++, C#, or Rust).
• Cloud-Native & Distributed Systems: 5 + years of deep expertise building and debugging systems that deal with CAP theorem trade-offs, eventual consistency, and distributed tracing. Robust experience working with AWS or GCP and navigating Kubernetes clusters.
• Systems Expertise: Deep knowledge of Linux internals, process management, resource isolation, and networking protocols (OSI model, load balancing, service meshes).
• Execution & Quality: A proven track record of estimating complex work effectively, debugging distributed architectures via logs/traces, and proactively identifying design edge cases.

Preferred Qualifications

• Active engagement or experience with open-source communities.
• Experience or active interest in using AI coding assistants (like Cursor or Claude) to accelerate engineering velocity and automate boilerplate tasks.

Pay & Benefits

The compensation offered for this position will depend on qualifications, experience, and work location. For candidates who receive an offer at the posted level, the starting base salary (for non-sales roles) or base salary + commission target (for sales/commissioned roles) is expected to be the annual range listed below. The offered compensation may also include restricted stock units and a bonus. A description of our employee benefits may be found here. $147,000.00 - $237,500.00/yr

Our Commitment

We are committed to providing reasonable accommodations for all qualified individuals with a disability. If you require assistance or accommodation due to a disability or special need, please contact us at accommodations@paloaltonetworks.com.

Equal Opportunity Employer

Palo Alto Networks is an equal opportunity employer. We celebrate diversity in our workplace, and all qualified applicants will receive consideration for employment without regard to age, ancestry, color, family or medical care leave, gender identity or expression, genetic information, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran status, race, religion, sex (including pregnancy), sexual orientation, or other legally protected characteristics.

Immigration Sponsorship

Yes