Senior Software Engineer - Database Engine Internals
About the role
The mission at Databricks is to simplify the entire data lifecycle from ingestion to ETL, BI, and ML/AI with a unified platform. We aim to replace traditional data warehouse architectures with a new pattern called Lakehouse, as outlined in the CIDR 2021 paper. This shift addresses challenges like data staleness, reliability, cost, lock-in, and limited use-cases.
Responsibilities
- Design and implement next-generation query engines and storage systems that outperform specialized data warehouses while maintaining the flexibility of general-purpose systems.
- Work on distributed query execution and scheduling, vectorized execution engines, data security, resource management, transaction coordination, efficient storage structures, and automatic physical data optimization.
Requirements
A passion for database systems, storage systems, distributed systems, language design, or performance optimization. Experience working towards a multi-year vision with incremental deliverables and a drive to deliver customer value and impact. 5+ years of experience working in a related system (preferred).
Qualifications
PhD in databases or distributed systems is optional.
Skills
Experience with query compilation and optimization, distributed systems, language design, and performance optimization.
Benefits
Details on benefits vary by location. Visit our page for more information.
Pay
The pay range for this role is $166,000—$225,000 USD. Actual compensation packages are based on several factors including job-related skills, depth of experience, relevant certifications and training, and specific work location.
Schedule
N/A