Staff Software Engineer, Batch and Realtime Streaming

Responsibilities

• Arcitect, design, and implement core components for a Batch and Realtime Streaming platform, including data ingestion pipelines, storage systems, serving layers, and API interfaces
• Ship new features by collaborating across research, legal, trading, finance operations data, and infra teams for trading systems
• Collaborate with ML researchers and data scientists to understand their workflows and design intuitive interfaces (APIs, SDKs, UIs) for seamless feature discovery, access, and reuse
• Ensure data quality, consistency, and lineage for features, building robust mechanisms for versioning, monitoring, and governance
• Optimize data pipelines and storage for high performance, scalability, and reliability, considering both batch and real-time use cases
• Drive adoption of the feature store across teams by producing documentation, onboarding materials, and developer support
• Mentor junior engineers and contribute to team best practices and technical excellence

Requirements

• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field (or equivalent experience)
• 10+ years of software engineering experience, with a strong background in distributed computing and data infrastructure
• Experience in programming languages such as Python and Go
• Understanding of database technologies (Postgres, MySQL, Cassandra, DuckDB) and experience with APIs (REST/gRPC)
• Strong problem-solving skills, with a focus on delivering high-quality, maintainable, and well-documented solutions
• Excellent communication and collaboration skills; ability to work closely with both engineering and research teams

Preferred Qualifications

• Experience building large-scale data pipelines and storage systems (e.g., Airflow, Spark, Ray, Iceberg, etc)
• Exposure to modern Python data science tooling. (pandas, polars, dask, duckdb, etc)
• Experience with monitoring and observability tools for distributed systems (e.g., Prometheus, Grafana, ELK Stack)
• Prior experience working with feature stores (e.g., Feast, Hopsworks, or custom solutions) is highly desirable