Senior Software Engineer - Distributed Data Systems

About the role

As a software engineer on the Runtime team at Databricks, you will be building the next generation distributed data storage and processing systems that can outperform specialized SQL query engines in relational query performance, yet provide the expressiveness and programming abstractions to support diverse workloads ranging from ETL to data science.

Responsibilities

• Develop the de facto open source standard framework for big data, such as Apache Spark™.
• Provide reliable and high performance services and client libraries for storing and accessing humongous amount of data on cloud storage backends, e.g., AWS S3, Azure Blob Store.
• Achieve higher level abstractions and guarantees, including ACID transactions and time travel, to simplify the complexity of real-world data engineering architecture, as in Delta Lake.
• Make it simple and possible to orchestrate and operate tens of thousands of data pipelines, as in Delta Pipelines.
• Build the next generation query optimizer and execution engine that's fast, tuning free, scalable, and robust.

Requirements

• BS (or higher) in Computer Science, related technical field or equivalent practical experience.
• Comfortable working towards a multi-year vision with incremental deliverables.
• Motivated by delivering customer value and impact.
• 5+ years of production level experience in either Java, Scala or C++.
• Strong foundation in algorithms and data structures and their real-world use cases.
• Experience with distributed systems, databases, and big data systems (Apache Spark, Hadoop).

Qualifications

While not required, candidates with experience in performance engineering, distributed systems, and/or data engineering will be given preference.

Skills

• Experience with distributed systems, databases, and big data systems (Apache Spark, Hadoop).
• Strong foundation in algorithms and data structures and their real-world use cases.
• Experience with performance engineering.

Benefits

Details on specific benefits are available upon request.

Pay

The pay range(s) for this role is $120,000 - $200,000 per year, depending on experience and qualifications. Actual compensation packages are based on several factors that are unique to each candidate, including but not limited to job-related skills, depth of experience, relevant certifications and training, and specific work location. Databricks anticipates utilizing the full width of the range. The total compensation package for this position may also include eligibility for annual performance bonus, equity, and the benefits listed above.

Schedule

Full-time, remote position.

Contact

If you have questions about this role, please contact [Contact Information].