Software Development Engineer in Test

What You Will Do

• Develop end-to-end, integration, and unit test suites covering options pricing models (Black-Scholes, binomial trees), order lifecycle, and P&L calculations
• Test and validate AI/ML models used in signal generation, volatility forecasting, and trade execution — including drift detection, model regression, and output boundary testing
• Build evaluation harnesses for LLM-powered tools used internally (e.g., trade summarization, risk Q&A, alert triage) to assess accuracy, hallucination rates, and latency
• Simulate realistic market scenarios including high-volatility events, expiry dates, and corporate actions to stress-test system behavior
• Validate FIX protocol messaging, OMS/EMS integrations, and exchange connectivity (CBOE, ISE, etc.)
• Collaborate with quants to write test cases that verify Greeks (delta, gamma, vega, theta) and pricing accuracy under various market conditions
• Build performance and load testing harnesses to validate sub-millisecond latency requirements
• Design data quality pipelines to validate training data, feature stores, and model inputs for correctness and consistency
• Participate in code reviews and advocate for testability in system design
• Own CI/CD pipeline quality gates, including ML model promotion gates (shadow mode, A/B, champion/challenger)
• Investigate production incidents and translate findings into regression tests

What SpiderRock Is Looking For

• 4+ years of SDET or QA Engineering experience, with at least 2 years in financial services or trading systems
• Proficiency in Python and/or Java/C++ for test automation
• Strong understanding of options trading concepts — calls/puts, expiry, strike, Greeks, volatility surfaces
• Experience testing real-time, event-driven systems (Kafka, FIX, WebSockets)
• Familiarity with ML concepts — model training, inference, overfitting, feature importance, and evaluation metrics (precision, recall, AUC)
• Experience testing ML pipelines end-to-end: data ingestion → feature engineering → model output → downstream consumption
• Solid fundamentals in data structures, algorithms, and distributed systems
• Familiarity with SQL and time-series databases (kdb+, InfluxDB, TimescaleDB)
• Experience with CI/CD tools (Jenkins, GitLab CI, GitHub Actions)
• Ability to read and reason about quantitative models and pricing logic
• Experience writing LLM evaluation frameworks — prompt regression testing, output scoring, and consistency checks across model versions
• Familiarity with MLflow, Weights & Biases, or SageMaker for model lifecycle tracking and test integration
• Knowledge of AI governance and model risk management frameworks (SR 11-7 or equivalent) relevant to financial institutions
• Knowledge of regulatory requirements (FINRA, SEC, CFTC) and audit trail testing including AI-assisted decision logging
• Exposure to co-location or FPGA-based trading infrastructure
• Familiarity with chaos engineering and fault injection testing
• Prior experience with kdb+/q for tick data validation