Distinguished Engineer, Apache Spark

What You'll Be Doing

• Lead the architecture, design and implementation of accelerated Apache Spark and related big-data frameworks
• Engage open source communities (including Apache Spark, RAPIDS, Apache Iceberg, Delta Lake and UCX) for technical discussion and contribution, and engage new communities where we may not have a strong presence yet
• Work with NVIDIA partners to deploy GPU enabled data analytics solutions in public cloud or on-premises clusters
• Present technical solutions at industry conferences and meetups
• Collaborate with distributed systems teams to define solutions to distributed processing problems challenges at large scale
• Provide recommendations and feedback to teams regarding decisions surrounding topics such as infrastructure, continuous integration and testing strategy
• Build, test and optimize CUDA/C++ libraries across different platforms
• Build automation and tools that will increase the productivity of teams developing distributed systems
• Mentor members of the engineering team

What We Need To See

• BS, MS, or PhD in Computer Science, Computer Engineering, or closely related field (or equivalent experience)
• 17+ years of work or research experience in software development
• Prior experience in delivering complex software projects as a lead architect
• Outstanding technical skills in designing and implementing high-quality distributed systems
• Excellent programming skills in C++, Java, and/or Scala
• Highly motivated with strong interpersonal skills and communication skills
• 5+ years working experience with key open source big-data projects as a contributor or committer to Apache Spark, Apache Hadoop, Apache Flink, Apache Kafka, Apache Hive, Apache Arrow, Delta Lake
• Excellent knowledge about distributed system schedulers: Kubernetes, Hadoop YARN, Apache Spark
• Able to delve into a new area and quickly come up to speed
• Able to work with teams across boundaries and geographies

Ways To Stand Out From The Crowd

• Working experience in designing and developing columnar query engines
• Committership at major open source projects (such as Apache Spark, Apache Hadoop, Apache Flink)
• Working experience with acceleration libraries (CUDA, RAPIDS, UCX)

Pay & Benefits

Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary range is 320,000 USD - 488,750 USD. You will also be eligible for equity and benefits.