Principal Software Engineer - Spark

Job Description

At Cloudera, we empower people to transform complex data into clear and actionable insights. With as much data under management as the hyperscalers, we're the preferred data partner for the top companies in almost every industry. Powered by the relentless innovation of the open source community, Cloudera advances digital transformation for the world’s largest enterprises.

About the Role

We are seeking a Principal Staff Engineer with a strong technical background in the data infrastructure space to lead the Cloudera Data Engineering experience for all customers using Cloudera Data Engineering Spark, Airflow, and Lakehouse. This high-impact IC role offers the opportunity to shape the future of Cloudera’s Data Engineering and Lakehouse products across multiple cloud environments, impacting thousands of customers worldwide.

Responsibilities

• Drive the multi-year technical roadmap and architectural vision for Cloudera Data Engineering.
• Gain deep technical knowledge across the data services technical stack, with a focus on Spark, Airflow, Iceberg, and apply this expertise in your daily work.
• Foster engineering excellence through technical mentorship, design reviews, and architectural guidance.
• Collaborate with product, engineering, and cross-functional partners, leading the delivery of several large, critical features in Cloudera’s data engineering experience.
• Work on large-scale distributed systems, ranging from hundreds to thousands of nodes in production clusters.
• Bring passion for programming, clean coding practices, attention to detail, and a strong focus on quality.

Requirements

• Relevant studies / BS or MS in Computer Science or related field
• 10+ years of experience as a Software Engineer in the data infrastructure space
• Strong understanding of at least one of the following languages: Java, Scala, C++, Python, GoLang.
• Passionate about programming, clean coding habits, attention to detail, and focus on quality
• Deep expertise in distributed data processing systems and cloud-native architectures
• Excellent communication and collaboration skills
• Experience with containerization (Kubernetes, Docker)
• Experience with using/developing Apache Spark/Airflow or other related technologies
• Experience with public cloud (AWS/Azure/GCP) and/or private cloud (OpenShift/Rancher)

Qualifications

• An open-minded attitude, desire to learn new things and build great products
• Contributed to open-source projects
• Strong understanding of modern Lakehouse architectures, open table formats, and metadata/catalog services
• Experience with large-scale, distributed systems design and development with an understanding of scaling, performance, and scheduling
• Solid experience with at least one cloud service (AWS, Azure, GCP, OpenShift)

Skills

• Programming
• Clean Coding Practices
• Attention to Detail
• Quality Focus
• Distributed Data Processing Systems
• Cloud-Native Architectures
• Communication and Collaboration
• Containerization
• Apache Spark/Airflow
• Public Cloud (AWS/Azure/GCP)
• Private Cloud (OpenShift/Rancher)

Benefits

• Generous PTO Policy
• Support Work-Life Balance with Unplugged Days
• Flexible WFH Policy
• Mental & Physical Wellness Programs
• Phone and Internet Reimbursement Program
• Access to Continued Career Development
• Comprehensive Benefits and Competitive Packages
• Paid Volunteer Time
• Employee Resource Groups

Pay

The expected base salary range for this role in California is $270 - $320k. The salary will vary depending on your job-related skills, experience and location.

Schedule

N/A