Principal Software Architect - Data Platform

About SecurityScorecard

SecurityScorecard is the global leader in cybersecurity ratings, continuously rating over 12 million companies across 64 countries. Founded in 2013, the company’s patented rating technology is used by more than 25 000 organizations for self‑monitoring, third‑party risk management, board reporting, and cyber‑insurance underwriting. Headquartered in New York City, SecurityScorecard has been recognized by Inc Magazine, Crain’s NY, Fast Company, and the Achievers 50 Most Engaged Workplaces award.

About the Role

SecurityScorecard is hiring a Principal Software Architect to lead the system design of its data platform. The role is an individual contributor reporting to the Chief Architect, working alongside a Principal Architect for AI and a Principal Front‑end Architect, and partnering closely with engineering leadership, Product, and Data Science.

Responsibilities

• Own the end‑to‑end system design for the data platform, from ingestion through the serving layer.
• Define service boundaries and data contracts between producers and consumers, including schema ownership, compatibility rules, and handling of breaking changes.
• Design the lakehouse architecture: table format, partitioning strategy, schema evolution, compaction, and metadata growth at scale.
• Architect the analytical serving layer for three workload classes—low‑latency high‑concurrency queries, ad‑hoc internal analytics, and bulk delivery to external feeds—ensuring isolation so one workload never degrades another.
• Set direction on languages and frameworks in the data stack.
• Engineer data quality and observability into the platform (validation, quarantine paths, freshness/completeness SLOs, drift detection, and automated lineage/metadata).
• Design for correctness and reproducibility in the ratings pipeline, including backfills and historical restatements when scoring logic changes.
• Write Technical Design Reviews (TDRs), design docs, and standards that guide data architecture across teams, and embed that intent directly into repositories.
• Review TDRs from across engineering, providing substantive feedback on architecture and risk.
• Partner with AI and Front‑End Architects on data access patterns.
• Mentor senior and staff engineers on data system design.

Required Qualifications

• 10+ years of software or data engineering experience, with significant time architecting large‑scale data platforms.
• Deep expertise in stream and batch processing at scale using Kafka, Flink, Spark, or close equivalents, and clear judgment about workload placement.
• Strong Python and PySpark skills; solid Java for Flink stream processing; enough Scala to read and reason about an existing Spark codebase.
• Hands‑on experience designing lakehouse storage in production (Parquet, Iceberg, partitioning, compaction, schema evolution).
• Experience architecting OLAP and analytical serving layers (ClickHouse, Druid, Pinot, BigQuery, Snowflake, or similar).
• Strong distributed‑systems fundamentals as they apply to data (exactly‑once vs. at‑least‑once semantics, ordering, backpressure, late/out‑of‑order data, pipeline failure modes).
• Track record of building data quality, contracts, and observability as engineered system properties (assertions, schema enforcement, lineage in code).
• Experience owning a large‑scale data migration while preserving history and correctness through cutover.
• Proven ability to influence without authority—presenting technical direction to skeptical engineers and earning genuine buy‑in.
• Strong technical writing and mentorship: TDRs, design docs, decision records, and raising the technical bar around you.
• Comfort operating as a senior individual contributor, driving outcomes through prototyping and technical credibility.

Preferred Qualifications

• Experience building the data layer underneath ML or LLM systems (feature stores, vector stores, retrieval pipelines).
• Experience with internet‑scale scan, telemetry, or observability data, or a cybersecurity industry background.
• Track record of reducing platform cost at scale through storage tiering, query governance, or compute right‑sizing.

Tech Stack

The platform runs on Node.js and TypeScript with a React micro‑frontend architecture and PostgreSQL/ClickHouse for storage. Core data components include Kafka for event streaming, Flink (Java) for stream processing, Spark (PySpark/Scala) for batch and micro‑batch, and ClickHouse for analytical workloads. Infrastructure is on AWS using Kubernetes, Terraform, Helm, and ArgoCD. Some high‑performance pipeline components are written in C++. The team is actively expanding its AI/ML infrastructure.

Benefits

• Competitive salary and stock options.
• Health benefits.
• Unlimited PTO, parental leave, and tuition reimbursements.
• Additional country‑specific perks.

Pay

The estimated total compensation range for this position is $240,000 – $300,000 (base plus bonus). Actual compensation will vary based on factors such as skills, qualifications, experience, and affordability.