Staff Software Engineer - Database Engine Internals
About the role
The mission at Databricks is to simplify the entire data lifecycle from ingestion to ETL, BI, and ML/AI using a unified platform. We aim to replace traditional data warehouse architectures with a new pattern called Lakehouse, as outlined in the CIDR 2021 paper. This shift addresses challenges like data staleness, reliability, cost, lock-in, and limited use-cases. Key to this vision is a decoupled query engine and structured storage system that outperforms specialized data warehouses while maintaining the flexibility of general-purpose systems.
Responsibilities
- Design and implement next-generation query compilation and optimization algorithms
- Develop distributed query execution and scheduling mechanisms
- Build vectorized execution engines for improved performance
- Ensure data security and manage resources efficiently
- Create efficient storage structures and automatic physical data optimization techniques
Requirements
A passion for database systems, storage systems, distributed systems, language design, or performance optimization. Experience working towards a multi-year vision with incremental deliverables. Motivated by delivering customer value and impact. 8+ years of experience in a related system (preferred).
Qualifications
PhD in databases or distributed systems is optional.
Skills
Experience with database systems, storage systems, distributed systems, language design, and performance optimization.
Benefits
Details on benefits vary by location. Visit our page for more information.
Pay
The pay range for this role is $192,000—$260,000 USD. Actual compensation packages are based on several factors including job-related skills, depth of experience, relevant certifications and training, and specific work location.
Schedule
N/A