Lead Data Architect

About the role

The role of Data Architect at Optiver Chicago supports and advances the data capabilities of the local research team and contributes to the broader Lakehouse architecture vision. Key responsibilities include designing, building, and maintaining reliable ETL/ELT pipelines using Spark, Structured Streaming, Databricks, and in-house high-performance tools. The role involves optimizing and productionizing research workflows with a strong focus on scalability, resilience, and performance tuning. Collaboration with power users to develop and share reusable patterns, templates, and onboarding pathways is also part of the job.

Responsibilities

• Design, build, and maintain reliable ETL/ELT pipelines using Spark, Structured Streaming, Databricks, and in-house high-performance tools
• Optimize and productionize research workflows with a strong focus on scalability, resilience, and performance tuning
• Collaborate with power users to develop and share reusable patterns, templates, and onboarding pathways
• Define, document, and enforce data engineering best practices
• Mentor junior engineers and drive a culture of continuous learning and DataOps excellence

Requirements

Immediate impact on the data systems powering world-class research and real-time trading decisions. Unique opportunity to shape Lakehouse engineering in the United States and influence global data architecture. High autonomy to own complex workflows and template “how-to” solutions across teams. Close collaboration with quant researchers and traders to unlock predictive insights and trading alpha. Partnership with best-in-class engineers across Chicago, Amsterdam, and Sydney.

Qualifications

• 5+ years of hands-on experience in data engineering, delivering robust pipelines at scale
• Advanced Python skills and deep experience with Apache Spark and the Databricks platform
• Familiarity with Delta Lake, streaming data systems (e.g., Kafka), and distributed compute environments
• Solid understanding of cloud-native data architectures (preferably AWS) and infrastructure cost optimization principles
• Proficiency in relational databases (e.g., PostgreSQL) and modern orchestration tools
• Bonus: Experience with system-level languages (e.g., C++, Rust) and exposure to MLOps or MLflow
• Proven ability to lead projects independently and deliver outcomes in fast-paced environments
• Clear communicator who collaborates well with researchers, traders, and engineers alike
• Enthusiastic mentor and strong advocate for engineering rigor and platform scalability
• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related technical discipline

Skills

The ideal candidate should have hands-on experience in data engineering, advanced Python skills, and deep experience with Apache Spark and the Databricks platform. Familiarity with Delta Lake, streaming data systems, and distributed compute environments is essential. Solid understanding of cloud-native data architectures and infrastructure cost optimization principles is required. Proficiency in relational databases and modern orchestration tools is necessary. Bonus points for experience with system-level languages, exposure to MLOps or MLflow, and proven ability to lead projects independently.

Benefits

• 401(k) match up to 50%
• Comprehensive health, mental, dental, vision, disability, and life coverage
• 25 paid vacation days alongside market holidays
• Extensive office perks, including breakfast, lunch and snacks, regular social events, clubs, sporting leagues and more

Pay

The expected base salary for this position is [insert actual base salary range here]. Offers will ultimately be determined based on experience, education, skill set, and performance in the interview process. This position will also be eligible for a discretionary bonus (if determined by Optiver) and Optiver’s benefits package with the benefits listed above.

Schedule

[Insert actual schedule details if provided]