Principal Software Architect - Data Platform

About SecurityScorecard

SecurityScorecard is the global leader in cybersecurity ratings, with over 12 million companies continuously rated, operating in 64 countries. Founded in 2013 by security and risk experts Dr. Alex Yampolskiy and Sam Kassoumeh and funded by world‑class investors, SecurityScorecard’s patented rating technology is used by over 25 000 organizations for self‑monitoring, third‑party risk management, board reporting, and cyber‑insurance underwriting. Headquartered in New York City, the company has been recognized by Inc Magazine as a “Best Workplace,” by Crain’s NY as a “Best Places to Work in NYC,” and as one of the 10 hottest SaaS startups in New York for two years in a row. SecurityScorecard was also named to Fast Company’s 2023 list of the World’s Most Innovative Companies and to the Achievers 50 Most Engaged Workplaces in 2023.

About the Role

SecurityScorecard is hiring a Principal Software Architect to lead the system design of its data platform. The platform ingests internet‑scale measurement data, processes it across streaming, micro‑batch, and batch paths, stores it affordably and queryably, and serves analytics fast enough for real‑time risk exploration. Data quality is critical because the ratings directly affect underwriting and scoring decisions. This individual contributor role reports to the Chief Architect and partners closely with engineering leadership, Product, and Data Science.

What You’ll Do

• Own the system design for the data platform end‑to‑end, from ingestion through to the serving layer.
• Define service boundaries and data contracts between producers and consumers, including schema ownership, compatibility rules, and handling of breaking changes.
• Design the lakehouse: table format, partitioning strategy, schema evolution, compaction, and metadata growth at scale.
• Architect the analytical serving layer for three workload classes with conflicting demands—low‑latency, high‑concurrency queries for customers; ad‑hoc exploration for internal analytics and Data Science; and bulk delivery to external feeds and partners—ensuring isolation so one workload never degrades another.
• Set direction on languages and frameworks in the data stack.
• Engineer data quality and observability into the platform rather than bolting them on: validation and quarantine paths, freshness and completeness SLOs, drift detection, and lineage/metadata generated by the pipeline itself.
• Design for correctness and reproducibility in the ratings pipeline, including backfills and historical restatement when scoring logic changes.
• Write Technical Design Reviews (TDRs), design docs, and standards that set data architecture direction across teams, and embed that intent into repositories.
• Review TDRs from across engineering, providing substantive feedback on architecture and risk.
• Partner with AI & Front‑End Architects on data access patterns, and mentor senior and staff engineers on data system design.

Required Qualifications

• 10+ years of software or data engineering experience, including significant time architecting large‑scale data platforms.
• Deep expertise in stream and batch processing at scale with Kafka, Flink, and Spark (or close equivalents), and clear judgment about which path a given workload belongs in.
• Strong Python and PySpark, solid Java for Flink stream processing, and enough Scala to read and reason about an existing Spark codebase.
• Hands‑on experience designing lakehouse storage in production: columnar formats such as Parquet, open table formats such as Iceberg, and the associated partitioning, compaction, and schema‑evolution decisions.
• Experience architecting OLAP and analytical serving layers (ClickHouse, Druid, Pinot, BigQuery, Snowflake or similar).
• Strong distributed‑systems fundamentals as they apply to data: exactly‑once vs. at‑least‑once semantics, ordering, backpressure, late and out‑of‑order data, and pipeline failure modes.
• Track record of building data quality, contracts, and observability as engineered system properties (assertions, schema enforcement, lineage in code).
• Experience owning a large‑scale data migration, preserving history and correctness through a cutover.
• Proven ability to influence without authority—presenting technical direction to skeptical engineers, earning genuine buy‑in, and giving rigorous design‑review feedback on systems you didn’t build.
• Strong technical writing and mentorship: TDRs, design docs, decision records that teams can act on without hand‑holding, and a history of raising the technical bar around you.
• Comfort operating as a senior individual contributor, driving outcomes through prototyping and technical credibility.

Preferred Qualifications

• Experience building the data layer underneath ML or LLM systems: feature stores, vector stores, or retrieval pipelines.
• Experience with internet‑scale scan, telemetry, or observability data, or a cybersecurity industry background.
• Track record of reducing platform cost at scale through storage tiering, query governance, or compute right‑sizing.

Our Tech Stack

Our platform runs on Node.js and TypeScript with a React micro‑frontend architecture and PostgreSQL and ClickHouse for storage. We use Kafka for event streaming, and our infrastructure runs on AWS with Kubernetes, Terraform, Helm, and ArgoCD. Stream processing runs on Flink in Java, while batch and micro‑batch run on Spark. Some high‑performance pipeline components are written in C++. ClickHouse serves our analytical workloads. While you don’t need to have used every tool, you should be comfortable reasoning across this stack and making principled architectural trade‑offs.

Benefits

• Competitive salary and stock options.
• Health benefits.
• Unlimited PTO.
• Parental leave.
• Tuition reimbursements.
• Additional country‑specific perks.

Compensation

The estimated total compensation range for this position is $240,000 – $300,000 (base plus bonus). Actual compensation is based on factors such as skills, qualifications, experience, and market considerations.